Video signal encoding / decoding method and apparatus therefor

Abstract

The present invention provides a video signal encoding / decoding method and an apparatus for the same. The video decoding method of the present invention comprises the steps of: determining whether to apply a merge motion difference encoding method to a current block; generating a merge candidate list of the current block; determining a merge candidate of the current block based on the merge candidate list; and deriving a motion vector of the current block based on the merge candidate.

Description

Technical Field

[0001] This invention relates to a video signal encoding / decoding method and an apparatus for the method. Background Technology

[0002] With the trend of increasingly larger display panels, there is a growing need for higher-quality video services. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, research is actively underway to improve video compression rates. As a representative example, in 2009, the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013, and its compression performance is approximately twice that of H.264 / AVC. However, with the rapid development of high-definition video services, the limitations of HEVC have gradually become apparent. Summary of the Invention

[0003] Technical problems to be solved The purpose of this invention is to provide a method for refining motion vectors derived from merging candidates based on offset vectors when encoding / decoding video signals, and an apparatus for performing the method.

[0004] The purpose of this invention is to provide a method for transmitting an offset vector using a signal during the encoding / decoding of a video signal, and an apparatus for performing the method.

[0005] The technical problems to be solved by the present invention are not limited to those mentioned above, and those skilled in the art to which the present invention pertains will clearly understand other technical problems not mentioned through the following description.

[0006] Technical solution The video signal decoding method according to the present invention includes the following steps: determining whether to apply a merging motion interpolation coding method to the current block; generating a merging candidate list for the current block; determining merging candidates for the current block based on the merging candidate list; and deriving the motion vector of the current block based on the merging candidates. Specifically, when the merging motion interpolation coding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived from the merging candidates. When the maximum number of merging candidates included in the merging candidate list is greater than one, a merging candidate for the current block is selected based on index information decoded from the bitstream indicating one of the merging candidates. When the maximum number is one, the merging candidate is determined without decoding the index information.

[0007] In the video signal decoding method according to the present invention, the magnitude of the offset vector can be determined based on first index information, wherein the first index information indicates one of a plurality of motion offset magnitude candidates.

[0008] In the video signal decoding method according to the present invention, at least one of the maximum or minimum values ​​of the motion offset size candidates can be set differently depending on the value of the flag indicating the number of motion offset size candidates.

[0009] In the video signal decoding method according to the present invention, the flag can be transmitted as a signal at the image level.

[0010] In the video signal decoding method according to the present invention, at least one of the maximum or minimum values ​​of the motion offset size candidate can be set differently depending on the motion vector accuracy of the current block.

[0011] In the video signal decoding method according to the present invention, the magnitude of the offset vector can be obtained by shifting the value represented by the motion offset magnitude candidate specified by the first index information.

[0012] In the video signal decoding method according to the present invention, the direction of the offset vector can be determined based on second index information, the second index information being used to indicate one of a plurality of vector direction candidates.

[0013] The video signal encoding method of the present invention includes the following steps: determining whether to apply a merging motion interpolation encoding method to the current block; generating a merging candidate list for the current block; determining merging candidates for the current block based on the merging candidate list; and deriving the motion vector of the current block based on the merging candidates. Specifically, when the merging motion interpolation encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived from the merging candidates. When the maximum number of merging candidates included in the merging candidate list is more than one, index information indicating the merging candidates of the current block in the merging candidates is encoded. When the maximum number is one, the encoding of the index information is omitted.

[0014] The video decoding apparatus of the present invention includes an inter-frame prediction unit, which is configured to: determine whether to apply a merged motion interpolation coding method to a current block; generate a merged candidate list for the current block; determine merged candidates for the current block based on the merged candidate list; and derive motion vectors for the current block based on the merged candidates. When the merged motion interpolation coding method is applied to the current block, the motion vectors for the current block are derived by adding an offset vector to the motion vectors derived from the merged candidates. When the maximum number of merged candidates included in the merged candidate list is greater than one, a merged candidate for the current block is selected based on index information decoded from the bitstream indicating one of the merged candidates. When the maximum number is one, the merged candidate is determined without decoding the index information.

[0015] The video encoding apparatus of the present invention includes an inter-frame prediction unit, which is configured to: determine whether to apply a merged motion interpolation coding method to the current block; generate a merge candidate list for the current block; determine merge candidates for the current block based on the merge candidate list; and derive the motion vector of the current block based on the merge candidates. When the merged motion interpolation coding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived from the merge candidates. When the maximum number of merge candidates included in the merge candidate list is more than one, index information indicating the merge candidates of the current block is encoded. When the maximum number is one, the encoding of the index information is omitted.

[0016] The video decoder of the present invention includes a processor and a memory storing a computer program. The processor is used to execute the computer program to perform the video decoding method described above.

[0017] The video encoder of the present invention includes a processor and a memory storing a computer program. The processor is used to execute the computer program to perform the video encoding method described above.

[0018] The computer-readable storage medium of the present invention stores an executable program, which, when executed by a processor, implements the above-described video decoding method or video encoding method.

[0019] The features briefly outlined above are merely exemplary embodiments of the invention as described in the detailed description to follow, and do not limit the scope of the invention.

[0020] Invention Effects According to the present invention, the efficiency of inter-frame prediction can be improved by refining the motion vectors of the merged candidates based on the offset vector.

[0021] According to the present invention, the efficiency of inter-frame prediction can be improved by adaptively determining the magnitude and direction of the offset vector.

[0022] The effects that can be obtained in this invention are not limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art through the following description. Attached Figure Description

[0023] Figure 1 This is a block diagram of a video encoder according to an embodiment of the present invention.

[0024] Figure 2 This is a block diagram of a video decoder according to an embodiment of the present invention.

[0025] Figure 3 This is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0026] Figure 4 This is a diagram illustrating the various partitioning types of coded blocks.

[0027] Figure 5 This is a diagram illustrating an example of how coding tree units are divided.

[0028] Figure 6 It is a diagram showing the shape of the basic data unit.

[0029] Figure 7 and Figure 8 This is a diagram illustrating an example of dividing a coded block into multiple sub-blocks.

[0030] Figure 9 This is a flowchart illustrating an inter-frame prediction method according to an embodiment of the present invention.

[0031] Figure 10 It is a diagram showing the nonlinear motion of an object.

[0032] Figure 11 This is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0033] Figure 12 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0034] Figure 13 This is a diagram showing an example of the affine vectors of a sub-block under a 4-parameter motion model.

[0035] Figure 14 It is a flowchart demonstrating the process of exporting motion information of the current block using the merge mode.

[0036] Figure 15 This is a diagram showing the candidate blocks used to derive the merge candidates.

[0037] Figure 16 This is a diagram showing the location of the reference sample.

[0038] Figure 17 This is a diagram showing an example of a candidate block used to derive merge candidates.

[0039] Figure 18 This is a diagram illustrating an example of changing the position of a reference sample.

[0040] Figure 19 This is a diagram illustrating an example of changing the position of a reference sample.

[0041] Figure 20 It is a flowchart illustrating the process of updating the list of inter-frame motion information.

[0042] Figure 21 This is a diagram illustrating an embodiment of updating the inter-frame merging candidate list.

[0043] Figure 22 This is a diagram illustrating an example of updating the index of a stored inter-frame merge candidate.

[0044] Figure 23 This is a diagram showing the positions of the sub-blocks.

[0045] Figure 24 A diagram showing an example of generating a list of inter-frame motion information for each inter-frame prediction mode is provided.

[0046] Figure 25 This is a diagram illustrating an example of adding inter-frame merge candidates included in the long-term motion information list to the merge candidate list.

[0047] Figure 26 This is a diagram illustrating an example of performing redundancy checks only on some merge candidates.

[0048] Figure 27 This is a diagram illustrating an example of omitting redundancy checks for a specific merge candidate.

[0049] Figure 28 This is a graph showing the offset vector based on the values ​​of distance_idx, which indicates the magnitude of the offset vector, and direction_idx, which indicates the direction of the offset vector.

[0050] Figure 29 This is a graph showing the offset vector based on the values ​​of distance_idx, which indicates the magnitude of the offset vector, and direction_idx, which indicates the direction of the offset vector.

[0051] Figure 30 This is a diagram illustrating the partitioning pattern of coded blocks when the triangulation technique is applied.

[0052] Figure 31 This is a diagram showing examples of different settings for the offset vector of each sub-unit.

[0053] Figure 32 This is a diagram showing the motion vector candidates that can be used for fine-grained merging.

[0054] Figure 33 This is a diagram illustrating the configuration for merging the refined offset list.

[0055] Figure 34 and Figure 35 This is a graph showing the offset vectors specified by the merge offset candidates.

[0056] Figure 36 This is a diagram showing the candidate blocks used to derive motion vector prediction candidates.

[0057] Figure 37 This is a graph showing motion vector candidates that can be set as fine motion vector prediction candidates.

[0058] Figure 38 This is a diagram showing the configuration of the prediction vector refinement offset list. Detailed Implementation

[0059] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0060] Video encoding and decoding are performed on a block-by-block basis. For example, encoding / decoding processes such as transform, quantization, prediction, loop filtering, or reconstruction can be performed on encoded blocks, transform blocks, or prediction blocks.

[0061] Hereinafter, the block to be encoded / decoded will be referred to as the "current block". For example, depending on the current encoding / decoding process step, the current block can represent an encoded block, a transform block, or a prediction block.

[0062] Additionally, as used herein, the term "unit" refers to a basic unit used to perform a specific encoding / decoding process, and "block" can be understood as representing a sample array of a predetermined size. Unless otherwise stated, "block" and "unit" are used interchangeably. For example, in the embodiments described later, encoding block and encoding unit can be understood to have the same meaning.

[0063] Figure 1 This is a block diagram of a video encoder according to an embodiment of the present invention.

[0064] Reference Figure 1 The video encoding device 100 may include an image segmentation unit 110, a prediction unit 120 and 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy coding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0065] Figure 1 The components shown are illustrated individually to illustrate the distinct functionalities of the video encoding device and do not imply that each component is composed of separate hardware or a single software component. That is, for ease of explanation, the components are arranged such that at least two components are combined into one, or one component is divided into multiple components, thereby performing functions. Such embodiments of integrated components and embodiments of separated components are also within the scope of this invention, provided they do not depart from its spirit.

[0066] Furthermore, some structural elements are not essential structural elements for performing the essential functions of this invention, but rather optional structural elements used only to improve performance. This invention can be implemented by including only the components necessary for realizing the essence of the invention, excluding the structural elements used only to improve performance, and structures including only the essential structural elements, excluding the optional structural elements used only to improve performance, are also within the scope of this invention.

[0067] The image partitioning unit 110 can divide the input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transformation unit (TU), or a coding unit (CU). The image partitioning unit 110 divides an image into a combination of multiple coding units, prediction units, and transformation units, and can select a combination of coding units, prediction units, and transformation units to encode the image based on a predetermined criterion (e.g., a cost function).

[0068] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quadtree structure can be used. A video or the largest coding unit can be used as the root, and the coding unit can be divided into additional coding units with a number of child nodes equivalent to the number of coding units in the division. Coding units that are no longer divided according to certain constraints become leaf nodes. That is, when it is assumed that a coding unit can only be divided into squares, a coding unit can be divided into a maximum of four other coding units.

[0069] In the embodiments of the present invention, the encoding unit may mean a unit that performs encoding, or it may mean a unit that performs decoding.

[0070] Prediction units within a coding unit can be divided into at least one shape of the same size, such as squares or rectangles, or a prediction unit within a coding unit can be divided into units with different shapes and / or sizes than another prediction unit.

[0071] Intra-prediction can be performed when the prediction unit for intra-prediction based on the coding unit is not the smallest coding unit, without having to divide it into multiple prediction units N×N.

[0072] Prediction units 120 and 125 may include an inter-frame prediction unit 120 that performs inter-frame prediction and an intra-frame prediction unit 125 that performs intra-frame prediction. It can be determined whether inter-frame prediction or intra-frame prediction is used for the prediction unit, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) is determined based on each prediction method. In this case, the processing unit performing the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction unit may determine the prediction method and prediction mode, and the transformation unit may perform the prediction. The residual value (residual block) between the generated prediction block and the original block can be input to the transformation unit 130. Furthermore, the prediction mode information, motion vector information, etc., used for prediction, along with the residual value, can be encoded in the entropy coding unit 165 and transmitted to the decoder. When using a specific coding mode, the original block can also be directly encoded and transmitted to the decoder without generating a prediction block through the prediction units 120 and 125.

[0073] The inter-frame prediction unit 120 can predict prediction units based on information from at least one image preceding or following the current image. In some cases, it can also predict prediction units based on information from a portion of the encoded region within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0074] The reference image interpolation unit receives reference image information from memory 155 and can generate pixel information of integer pixels or fractional pixels from the reference image. For luminance pixels, in order to generate pixel information of fractional pixels in units of 1 / 4 pixels, an 8-tap DCT-based interpolation filter with different filter coefficients can be used. For chrominance signals, in order to generate pixel information of fractional pixels in units of 1 / 8 pixels, a 4-tap DCT-based interpolation filter with different filter coefficients can be used.

[0075] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as the Full Search-based Block Matching Algorithm (FBMA), the Three-Step Search (TSS), and the New Three-Step Search Algorithm (NTS). Motion vectors can have values ​​in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. Different motion prediction methods can be used in the motion prediction unit to predict the current prediction unit. These methods include skipping, merging, Advanced Motion Vector Prediction (AMVP), and Intra Block Copying.

[0076] The intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block, which serves as pixel information within the current image. When the neighboring block of the current prediction unit is a block that has already undergone inter-frame prediction, and the reference pixel is a pixel that has undergone inter-frame prediction, the reference pixel included in the block that has undergone inter-frame prediction can be used as the reference pixel information for the surrounding block that has undergone intra-frame prediction. That is, when a reference pixel is unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0077] In intra-frame prediction, the prediction mode can have an angular prediction mode that uses reference pixel information in the prediction direction and a non-angular mode that does not use direction information when performing prediction. The mode used to predict luminance information and the mode used to predict chrominance information can be different. To predict chrominance information, either the intra-frame prediction mode information used for predicting luminance information or the predicted luminance signal information can be applied.

[0078] When performing intra-frame prediction, if the size of the prediction unit is the same as the size of the transform unit, intra-frame prediction can be performed based on pixels to the left, upper left, and upper right of the prediction unit. However, when performing intra-frame prediction, if the size of the prediction unit is different from the size of the transform unit, intra-frame prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-frame prediction using an N×N partition only for the smallest coding unit can be applied.

[0079] Intra-prediction methods can generate prediction blocks after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel based on the prediction mode. The type of AIS filter used for the reference pixel may vary. To perform intra-prediction, the intra-prediction mode of the current prediction unit can be predicted from the intra-prediction modes of prediction units existing in the vicinity of the current prediction unit. When using mode information predicted from surrounding prediction units to predict the prediction mode of the current prediction unit, if the intra-prediction modes of the current prediction unit and those of the surrounding prediction units are the same, predetermined flag information can be used to convey information indicating that the prediction modes of the current prediction unit and those of the surrounding prediction units are the same. If the prediction modes of the current prediction unit and those of the surrounding prediction units are different, entropy coding can be performed to encode the prediction mode information of the current block.

[0080] Furthermore, residual blocks including residual information can be generated, the residual information being the difference between the prediction unit that performs prediction based on the prediction unit generated in prediction units 120 and 125 and the original block of the prediction unit. The generated residual blocks can be input to the transformation unit 130.

[0081] In the transform unit 130, a transform method such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST) can be used to transform the residual block, which includes residual information between the original block and the prediction units generated by the prediction units 120 and 125. The DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on the intra-frame prediction mode information of the prediction units used to generate the residual block. Transformation of the residual block can also be skipped. A flag indicating whether to skip the transformation of the residual block can be encoded. Transformation skipping is allowed for residual blocks with a size below a threshold, or for luma or chroma components (4:4:4 format or below).

[0082] The quantization unit 135 can quantize the values ​​that have been transformed into the frequency domain in the transformation unit 130. The quantization coefficients can be changed according to the importance of the block or video. The values ​​calculated in the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0083] The rearrangement unit 160 can rearrange the coefficient values ​​of the quantized residual values.

[0084] The rearrangement unit 160 can transform 2D block shape coefficients into 1D vector form using a coefficient scanning method. For example, the rearrangement unit 160 can use a zig-zag scan method to scan the DC coefficients and even the coefficients in the high-frequency domain, and transform them into 1D vector form. Depending on the size of the transform unit and the intra-frame prediction mode, instead of zig-zag scanning, vertical scanning along the column direction and horizontal scanning along the row direction can also be used to scan the 2D block shape coefficients. That is, the choice between zig-zag scanning, vertical scanning, and horizontal scanning can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0085] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. For example, entropy coding can use various coding methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0086] The entropy coding unit 165 can encode various information such as residual coefficient information, block type information, prediction mode information, partitioning unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding units originating from the rearrangement unit 160 and the prediction units 120 and 125.

[0087] The coefficient values ​​of the coding units input from the rearrangement unit 160 can be entropy encoded in the entropy coding unit 165.

[0088] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the multiple values ​​quantized in the quantization unit 135, and perform inverse transform on the values ​​transformed in the transform unit 130. The residual values ​​generated in the inverse quantization unit 140 and the inverse transform unit 145 can be merged with the prediction units predicted by the motion prediction unit, motion compensation unit and intra-frame prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0089] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0090] Deblocking filters remove block distortion generated in the reconstructed image due to the boundaries between blocks. To determine whether to perform deblocking, the number of pixels in the columns or rows included in the block can be used to decide whether to apply a deblocking filter to the current block. When applying a deblocking filter to a block, a strong or weak filter can be applied depending on the desired deblocking intensity. Furthermore, during the use of deblocking filters, horizontal and vertical filtering can be processed simultaneously when performing vertical and horizontal filtering.

[0091] The offset correction unit can correct the offset between the video being deblocked and the original video on a pixel-by-pixel basis. To perform offset correction on a specific image, the following methods can be used: after dividing the pixels included in the video into a predetermined number of regions, determine the region to be offset and apply the offset to the corresponding region, or apply the offset by taking into account the edge information of each pixel.

[0092] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered reconstructed image and the original video. After dividing the pixels in the video into predetermined groups, filtering can be performed differently for each group by determining a filter to be used for the corresponding group. Information related to whether adaptive loop filtering is applied, along with luminance signals, can be transmitted per coding unit (CU). The shape and filter coefficients of the adaptive loop filter to be applied can vary depending on the block. Furthermore, it is possible to apply the same type (fixed type) of adaptive loop filter regardless of the characteristics of the block to which it is applied.

[0093] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the prediction units 120 and 125 when performing inter-frame prediction.

[0094] Figure 2 This is a block diagram of a video decoder according to an embodiment of the present invention.

[0095] Reference Figure 2The video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0096] When inputting a video stream from a video encoder, the input stream can be decoded by following the reverse steps of the video encoder.

[0097] The entropy decoding unit 210 can perform entropy decoding in the reverse order of the entropy encoding steps performed in the entropy encoding unit of the video encoder. For example, corresponding to the methods performed in the video encoder, various methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied.

[0098] The entropy decoding unit 210 can decode information related to intra-frame prediction and inter-frame prediction performed by the encoder.

[0099] The rearrangement unit 215 can perform rearrangement based on a method used in the encoding unit to rearrange the bitstream that has been entropily decoded by the entropy decoding unit 210. Rearrangement can be performed by reconstructing multiple coefficients represented in 1D vector form into 2D block-shaped coefficients. The rearrangement unit 215 receives information related to the coefficient scan performed in the encoding unit and can perform rearrangement by performing a reverse scan based on the scan order performed in the corresponding encoding unit.

[0100] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged blocks.

[0101] The inverse transform unit 225 can perform inverse discrete cosine transform and inverse discrete sine transform on the quantization result performed by the video encoder. These inverse discrete cosine transforms and inverse discrete sine transforms are inverse transforms of the transforms performed in the transform unit, i.e., inverse transforms of the discrete cosine transform and discrete sine transform. The DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform unit 225 may not perform the inverse transform. The inverse transform can be performed based on the transmission unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, a transform method (e.g., DCT or DST) can be selectively performed based on multiple pieces of information such as the prediction method, the size of the current block, and the prediction direction.

[0102] Prediction units 230 and 235 can generate prediction blocks based on information related to prediction block generation provided by entropy decoding unit 210 and previously decoded block or image information provided by memory 245.

[0103] As described above, when intra-prediction is performed in the same manner as in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra-prediction is performed on the prediction unit based on the pixels to its left, the pixels to its upper left, and the pixels above it. If the size of the prediction unit is different from the size of the transform unit, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using only N×N partitioning for the smallest coding unit can also be applied.

[0104] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit receives various information input from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method. It classifies prediction units according to the current coding unit and determines whether the prediction unit is performing inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 can use the information required for inter-frame prediction of the current prediction unit provided by the video encoder and perform inter-frame prediction on the current prediction unit based on information included in at least one of the previous or subsequent images of the current image to which the current prediction unit belongs. Alternatively, inter-frame prediction can also be performed based on information from a portion of the reconstructed region within the current image to which the current prediction unit belongs.

[0105] In order to perform inter-frame prediction, it is possible to determine, based on the coding unit, which of the following modes of motion prediction method is used for the prediction units included in the corresponding coding unit: Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), or Intra-Block Copy Mode.

[0106] The intra-prediction unit 235 can generate prediction blocks based on pixel information within the current image. When the prediction unit is one that has already performed intra-prediction, intra-prediction can be performed based on the intra-prediction mode information of the prediction unit provided by the video encoder. The intra-prediction unit 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra-smoothing filter is the part that performs filtering on the reference pixels of the current block, and whether to apply the filter can be determined according to the prediction mode of the current prediction unit. Adaptive intra-smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided by the video encoder and the adaptive intra-smoothing filter information. If the prediction mode of the current block is a mode that does not perform adaptive intra-smoothing filtering, then the adaptive intra-smoothing filter may not be applied.

[0107] For the reference pixel interpolation unit, if the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on pixel values ​​interpolated from reference pixels, then reference pixels with integer or fractional pixel units can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, then interpolation of reference pixels is not required. If the prediction mode of the current block is DC mode, then the DC filter can generate prediction blocks by filtering.

[0108] The reconstructed blocks or images can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0109] Information related to whether to apply a deblocking filter to a corresponding block or image can be received from the video encoder, as well as information regarding whether to apply strong or weak filtering when applying the deblocking filter. Information related to the deblocking filter provided by the video encoder can be received from the video decoder's deblocking filter, and deblocking filtering can be performed on the corresponding block at the video decoder.

[0110] The offset correction unit can perform offset correction on the reconstructed video based on the type and amount of offset correction used during video encoding.

[0111] The ALF can be applied to the coding unit based on information provided by the encoder, such as whether the ALF is applied and ALF coefficient information. This ALF information can be provided by including it in a specific parameter set.

[0112] The memory 245 stores the reconstructed image or block, such that the image or block can be used as a reference image or reference block, and the reconstructed image can be provided to the output unit.

[0113] Figure 3 This is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0114] The largest coding block can be defined as the coding tree block. An image can be divided into multiple coding tree units (CTUs). The coding tree unit is the largest coding unit and can also be called the largest coding unit (LCU). Figure 3 An example of dividing an image into multiple coding tree units is shown.

[0115] The size of a coding tree unit can be defined at the image level or the sequence level. Therefore, information representing the size of the coding tree unit can be transmitted via signals using either an image parameter set or a sequence parameter set.

[0116] For example, the size of the coding tree unit for the entire image within the sequence can be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image can be set to 128×128, and the size of the coding tree unit in the second image can be set to 256×256.

[0117] Coded blocks can be generated by dividing the coding tree into units. A coded block represents the basic unit used for encoding / decoding processing. For example, prediction or transformation can be performed on different coded blocks, or prediction coding modes can be determined on different coded blocks. The prediction coding mode represents the method for generating the predicted image. For example, prediction coding modes can include intra-frame prediction, inter-frame prediction, current picture referencing (CPR, or intra-block copy (IBC)), or combined prediction. For a coded block, at least one of the prediction coding modes—intra-frame prediction, inter-frame prediction, current picture referencing, or combined prediction—can be used to generate the prediction block associated with the coded block.

[0118] Information indicating the predictive coding mode of the current block can be transmitted via signals in the bitstream. For example, this information could be a 1-bit flag indicating whether the predictive coding mode is intra-frame or inter-frame. Current image reference or combined prediction can be used only if the predictive coding mode of the current block is determined to be inter-frame.

[0119] The current image reference is used to set the current image as the reference image and obtain the prediction block of the current block from the encoded / decoded region within the current image. Here, the current image means the image that includes the current block. Information indicating whether the current image reference is applied to the current block can be sent via signals in the bitstream. For example, this information could be a 1-bit flag. When the flag is true, the prediction coding mode of the current block can be determined as the current image reference; when the flag is false, the prediction mode of the current block can be determined as inter-frame prediction.

[0120] Alternatively, the predictive coding mode for the current block can be determined based on a reference image index. For example, when the reference image index points to the current image, the predictive coding mode for the current block can be determined as current image reference. When the reference image index points to another image instead of the current image, the predictive coding mode for the current block can be determined as inter-frame prediction. That is, current image reference is a prediction method that uses information from already encoded / decoded regions within the current image, and inter-frame prediction is a prediction method that uses information from other encoded / decoded images.

[0121] Combinatorial prediction represents a coding mode composed of two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, when applying combinatorial prediction, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or the current image reference, and a second prediction block can be generated based on another. If a first and a second prediction block are generated, a final prediction block can be generated by averaging or weighted summing the first and second prediction blocks. Information indicating whether combinatorial prediction is applied can be transmitted via a signal in the bitstream. This information can be a 1-bit flag.

[0122] Figure 4 This is a diagram illustrating the various partitioning types of coded blocks.

[0123] A coded block can be divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. Furthermore, the divided coded blocks can be further divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0124] Quadtree partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block can be divided into four square partitions (see reference). Figure 4 (a) 'SPLIT_QT' in section (a).

[0125] Binary tree partitioning refers to a partitioning technique that divides the current block into two blocks. The process of partitioning the current block along a vertical direction (i.e., using a vertical line that crosses the current block) is called vertical binary tree partitioning, and the process of partitioning the current block along a horizontal direction (i.e., using a horizontal line that crosses the current block) is called horizontal binary tree partitioning. After binary tree partitioning, the current block can be divided into two non-square partitions. Figure 4 In section (b), "SPLIT_BT_VER" represents the result of the vertical binary tree partitioning, and Figure 4 In section (c), “SPLIT_BT_HOR” indicates the result of the horizontal binary tree partitioning.

[0126] Ternary tree partitioning refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along a vertical direction (i.e., using two vertical lines crossing the current block) is called vertical ternary tree partitioning, and the process of dividing the current block into three blocks along a horizontal direction (i.e., using two horizontal lines crossing the current block) is called horizontal ternary tree partitioning. After ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. Figure 4 In part (d), "SPLIT_TT_VER" represents the result of the vertical ternary tree partitioning, and Figure 4 In section (e), “SPLIT_TT_HOR” indicates the result of the horizontal ternary tree partitioning.

[0127] The number of partitions in a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or image level. Therefore, the maximum partitioning depth of a coding tree unit can vary depending on different sequences or images.

[0128] Alternatively, the maximum partitioning depth can be determined individually for each of the multiple partitioning techniques. For example, the maximum partitioning depth allowed for quadtree partitioning can be different from the maximum partitioning depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0129] The encoder can transmit information via signals from the bitstream representing at least one of the partition shape or partition depth of the current block. The decoder can determine the partition shape and partition depth of the coding tree unit based on the information parsed from the bitstream.

[0130] Figure 5 This is a diagram illustrating an example of how coding tree units are divided.

[0131] The process of dividing coding blocks using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning is called multitree partitioning.

[0132] The coded blocks generated by applying a multi-way tree partitioning to the coded block can be called multiple downstream coded blocks. When the partitioning depth of the coded block is k, the partitioning depth of the multiple downstream coded blocks is set to k+1.

[0133] On the other hand, for multiple coding blocks with a partitioning depth of k+1, the coding block with a partitioning depth of k can be called the upstream coding block.

[0134] The partition type of the current coding block can be determined based on at least one of the partition shape of the upstream coding block or the partition type of the adjacent coding blocks. The adjacent coding blocks are adjacent to the current coding block and can include at least one of the current coding block's upper adjacent block, left adjacent block, or adjacent block to its upper left corner. The partition type can include at least one of whether to partition into a quadtree, whether to partition into a binary tree, the binary tree partition direction, whether to partition into a ternary tree, or the ternary tree partition direction.

[0135] To determine the shape of the coded block partition, information indicating whether the coded block has been partitioned can be sent via a signal in the bitstream. This information is a 1-bit flag "split_cu_flag," and when the flag is true, it indicates that the coded block has been partitioned using a multi-way tree partitioning technique.

[0136] When "split_cu_flag" is true, information indicating whether the coded block has been partitioned by a quadtree can be sent via a signal in the bitstream. This information is a 1-bit flag "split_qt_flag". When this flag is true, the coded block can be divided into 4 blocks.

[0137] For example, in Figure 5 The example shown illustrates how the coding tree unit is partitioned by a quadtree to generate four coding blocks with a partition depth of 1. Furthermore, the example illustrates applying quadtree partitioning again to the first and fourth coding blocks generated as a result of the quadtree partitioning. Ultimately, four coding blocks with a partition depth of 2 can be generated.

[0138] Furthermore, a code block with a partition depth of 3 can be generated by applying a quadtree partition to the code block with a partition depth of 2 again.

[0139] When a quadtree partitioning is not applied to the coded block, it can be determined whether to perform a binary tree partitioning or a ternary tree partitioning on the coded block by considering at least one of the following: the size of the coded block, whether the coded block is located at an image boundary, the maximum partitioning depth, or the partitioning shape of adjacent blocks. When it is determined whether to perform a binary tree partitioning or a ternary tree partitioning on the coded block, information indicating the partitioning direction can be transmitted via a signal in the bitstream. This information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partitioning direction (vertical or horizontal) can be determined based on this flag. Additionally, information indicating whether a binary tree partitioning or a ternary tree partitioning is applied to the coded block can be transmitted via a signal in the bitstream. This information can be a 1-bit flag "mtt_split_cu_binary_flag". The binary tree partitioning or ternary tree partitioning can be determined based on this flag.

[0140] For example, in Figure 5 The example shown illustrates the application of a vertical binary tree partitioning to a coded block with a partitioning depth of 1, the application of a vertical ternary tree partitioning to the left coded block in the resulting coded block, and the application of a vertical binary tree partitioning to the right coded block.

[0141] When implementing devices for encoding or decoding video, there is a problem that hardware performance may be insufficient to handle regions larger than a threshold. For example, there may be a problem where, when hardware performance allows for processing up to 4096 samples simultaneously, 64×64 data units would be redundantly accessed and processed, and data could not be processed simultaneously for regions with more than 4096 samples. As mentioned above, the basic unit of data processing can be defined as a pipelined basic data unit (virtual processing data unit, VPDU, hereinafter referred to as basic data unit).

[0142] Basic data units can be categorized into square, non-square, or non-rectangular types.

[0143] Figure 6 It is a diagram showing the shape of the basic data unit.

[0144] The basic data unit can include samples that are equal to or less than the maximum number of samples that can be processed simultaneously. For example, as in Figure 6 In the example shown in (a), a 64×64 square block can be used as the basic data unit. Alternatively, a non-square block can be used as the basic data unit. For example, as in Figure 6 In the examples shown in (b) or 6(c), a 32×128 block or a 64×32 block can be set as the basic data unit.

[0145] Although not shown, basic data units such as triangles, L-shapes, or polygons can be defined.

[0146] Information for determining basic data units can be transmitted via a bitstream using signals. This information can be used to determine at least one of the size or shape of the basic data unit. Based on this information, it can be determined whether non-square basic data units are permitted.

[0147] Alternatively, at least one of the sizes or shapes of the basic data units can be predefined in the encoder and decoder.

[0148] The size of the basic data unit can be considered to determine whether the type of coded block partitioning is allowed. For example, when the resulting coded block is larger than the basic data unit, such partitioning may not be allowed. Alternatively, when the resulting non-square coded block is larger than the basic data unit, such partitioning may not be allowed. For example, binary or ternary tree partitioning may not be allowed when the width or height of the coded block exceeds a threshold, or when the number of samples included in the coded block exceeds a threshold. Therefore, the encoding of information related to binary or ternary tree partitioning can be omitted.

[0149] Alternatively, it can be configured to require that coded blocks larger than the basic data unit be partitioned. Alternatively, it can be configured to require that coded blocks larger than the basic data unit be partitioned into a binary tree or a ternary tree. Therefore, for coded blocks larger than the basic data unit, although the flag split_flag indicating whether the coded block has been partitioned is not encoded, the value of the flag can be derived as 1.

[0150] As another example, a coding block larger than a basic data unit can be divided into multiple sub-blocks. Here, the sub-blocks can be set as prediction units for basic units used for prediction, or as transformation units for basic units used for transformation and / or quantization. In this case, dividing the coding block into multiple prediction units can be defined as VPDU prediction unit partitioning, and dividing the coding block into multiple transformation units can be defined as VPDU transformation unit partitioning.

[0151] At least one of VPDU prediction unit partitioning and VPDU transform unit partitioning can be applied to a coding block. The partitioning type of a coding block partitioned according to the application of VPDU prediction units can be set to the same as the partitioning type of a coding block partitioned according to the application of VPDU transform units.

[0152] When VPDU prediction unit partitioning is applied only to coding blocks, prediction is performed for each sub-block, but transforms and / or quantization can be performed on the coding block. In this case, prediction modes such as prediction coding mode, intra-frame prediction mode, or inter-frame prediction mode can be determined for the coding block.

[0153] When only VPDU transform units are applied to a coding block, prediction is performed on the sub-blocks, but transform and / or quantization can be performed on each sub-block.

[0154] Figure 7 and Figure 8 This is a diagram illustrating an example of dividing a coded block into multiple sub-blocks.

[0155] Figure 7 This is a diagram illustrating the partitioning pattern when only square basic data units are allowed. Figure 8 This is a diagram illustrating the partitioning patterns when square and non-square basic data units are allowed.

[0156] Assuming only square basic data units are allowed, in Figure 7 (a) and Figure 7 In (b), CU0 and CU3 are defined as two different VPDUs, and CU1 is defined as four different VPDUs. Therefore, CU0 and CU3 can be divided into two sub-blocks, and CU1 can be divided into four sub-blocks.

[0157] Assuming that square and non-square basic data units are allowed, in Figure 8 (a) and Figure 8 In (b), CU0 and CU3 can be defined as one VPDU, and CU1 can be defined as using two different VPDUs. Therefore, CU0 and CU3 are not divided into sub-blocks, while CU1 can be divided into two sub-blocks.

[0158] In this case, CU1 can be divided into square sub-blocks or non-square sub-blocks. For example, based on the horizontal lines dividing CU1 vertically, CU1 can be divided into two square sub-blocks. Alternatively, CU1 can be divided into two non-square sub-blocks based on the vertical lines dividing CU1 horizontally.

[0159] When multiple partitioning type candidates exist for a coded block, information indicating any one of the multiple partitioning type candidates can be transmitted via signals in the bitstream. For example, the information can indicate whether the coded block is divided into square sub-blocks or whether the coded block is divided into non-square sub-blocks.

[0160] Alternatively, dividing a coded block into square sub-blocks can be set to have a higher priority than dividing a coded block into non-square sub-blocks. For example, dividing a coded block into non-square sub-blocks can be allowed when the coded block cannot be divided into square sub-blocks.

[0161] Alternatively, the partitioning type of the coding block can be determined based on the partitioning type of the parent node coding block. For example, when partitioning the parent node coding block based on a ternary tree, it can be set to partition the coding block into square sub-blocks. On the other hand, when partitioning the parent node coding block based on a binary tree or ternary tree, it can be set to partition the coding block into non-square sub-blocks.

[0162] Inter-frame prediction refers to using information from previous images to predict the predictive coding mode of the current block. For example, a block in a previous image that is at the same position as the current block (hereinafter referred to as a collocated block) can be set as the predicted block for the current block. Hereinafter, the predicted block generated based on the block at the same position as the current block is called a collocated prediction block.

[0163] On the other hand, if an object that existed in a previous image has moved to a different location in the current image, the object's motion can be used to effectively predict the current block. For example, if the direction and size of the object's movement can be known by comparing the previous and current images, the object's motion information can be considered to generate a predicted block (or predicted image) for the current block. Hereinafter, the predicted block generated using motion information can be referred to as a motion prediction block.

[0164] A residual block can be generated by subtracting a prediction block from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using a motion prediction block instead of a corresponding prediction block, thereby improving the compression performance of the residual block.

[0165] As mentioned above, the process of generating prediction blocks using motion information can be called motion-compensated prediction. In most inter-frame predictions, prediction blocks can be generated based on motion-compensated prediction.

[0166] Motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weighted index. The motion vector represents the direction and magnitude of the object's movement. The reference image index specifies the reference image for the current block among a list of reference images. The prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 and L1 prediction). Motion information in either the L0 or L1 direction can be used based on the prediction direction of the current block. The bidirectional weighted index specifies the weights applied to the L0 prediction block and the weights applied to the L1 prediction block.

[0167] Figure 9 This is a flowchart illustrating an inter-frame prediction method according to an embodiment of the present invention.

[0168] Reference Figure 9 The inter-frame prediction method includes: determining the inter-frame prediction mode of the current block (S901); obtaining motion information of the current block according to the determined inter-frame prediction mode (S902); and performing motion compensation prediction of the current block based on the obtained motion information (S903).

[0169] Inter-frame prediction modes represent various techniques used to determine the motion information of the current block, and can include inter-frame prediction modes using translational motion information and inter-frame prediction modes using affine motion information. For example, inter-frame prediction modes using translational motion information can include merging mode and advanced motion vector prediction mode, while inter-frame prediction modes using affine motion information can include affine merging mode and affine motion vector prediction mode. Based on the inter-frame prediction mode, the motion information of the current block can be determined based on neighboring blocks adjacent to the current block or information parsed from the bitstream.

[0170] The following section details the inter-frame prediction method using affine motion information.

[0171] Figure 10 It is a diagram showing the nonlinear motion of an object.

[0172] The motion of objects within a video may be non-linear. For example, as in... Figure 10 In the example shown, non-linear motion of the object may occur, such as camera zooming, rotation, affine transformations, etc. When non-linear motion of the object occurs, it is not possible to effectively represent the motion of the object using translational motion vectors. Therefore, in the parts where non-linear motion of the object occurs, affine motion can be used instead of translational motion, thereby improving coding efficiency.

[0173] Figure 11 This is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0174] Whether to apply affine motion-based inter-frame prediction techniques to the current block can be determined based on information parsed from the bitstream. Specifically, whether to apply affine motion-based inter-frame prediction techniques to the current block can be determined based on at least one of a flag indicating whether to apply an affine merging mode or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0175] When an inter-frame prediction technique based on affine motion is applied to the current block, the affine motion model of the current block can be determined (S1101). The affine motion model can be determined by at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent the affine motion, while the 4-parameter affine motion model uses 4 parameters to represent the affine motion.

[0176] Equation 1 represents the case where affine motion is expressed using 6 parameters. Affine motion represents translational motion within a predetermined region defined by an affine seed vector.

[0177] Equation 1

[0178]

[0179] While using six parameters to represent affine motion allows for the representation of complex motions, the increased number of bits required to encode each parameter reduces encoding efficiency. Therefore, affine motion can also be represented using four parameters. Equation 2 illustrates the case of representing affine motion using four parameters.

[0180] Equation 2

[0181]

[0182] Information used to determine the affine motion model for the current block can be encoded and transmitted via a bitstream using signals. For example, this information could be a 1-bit flag, "affine_type_flag". A value of 0 indicates the application of a 4-parameter affine motion model, and a value of 1 indicates the application of a 6-parameter affine motion model. The flag can be encoded in units of stripes, tiles, or blocks (e.g., coded blocks or coded tree units). When the flag is transmitted via signals at the strip level, the affine motion model determined at that strip level can be applied to all blocks belonging to that strip.

[0183] Alternatively, the affine motion model of the current block can be determined based on the affine inter-frame prediction mode of the current block. For example, when applying the affine merging mode, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when applying the affine motion vector prediction mode, the information used to determine the affine motion model of the current block can be encoded and transmitted via a signal through the bitstream. For example, when applying the affine motion vector prediction mode to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0184] Next, the affine seed vector of the current block can be exported (S1102). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be exported. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block can be exported. The motion vectors at the control points can be called affine seed vectors. Control points can include at least one of the top-left, top-right, or bottom-left corners of the current block.

[0185] Figure 12 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0186] In a 4-parameter affine motion model, two related affine seed vectors can be derived from the top left, top right, or bottom left corners. For example, as in... Figure 12 In the example shown in (a), when the 4-parameter affine motion model is selected, the affine vectors can be derived using the affine seed vector SV0 of the top-left corner of the current block (e.g., the top-left sample (x0, y0)) and the affine seed vector sv1 of the top-right corner of the current block (e.g., the top-right sample (x1, y1)). Alternatively, the affine seed vector associated with the bottom-left corner can be used instead of the affine seed vector associated with the top-left corner, or vice versa.

[0187] In a 6-parameter affine motion model, affine seed vectors related to the top-left, top-right, and bottom-left corners can be derived. For example, as in... Figure 12 In the example shown in (b), when the 6-parameter affine motion model is selected, the affine vectors can be derived using the affine seed vector SV0 of the top left corner of the current block (e.g., the top left sample (x0, y0)), the affine seed vector sv1 of the top right corner of the current block (e.g., the top right sample (x1, y1)), and the affine seed vector sv2 of the bottom left corner of the current block (e.g., the bottom left sample (x2, y2)).

[0188] In the embodiments described later, under the 4-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments using the first and second affine seed vectors described later, at least one of the first and second affine seed vectors can be replaced by the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0189] Furthermore, under the 6-parameter affine motion model, the affine seed vectors of the upper left control point, upper right control point, and lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiments using the first, second, and third affine seed vectors described later, at least one of the first, second, and third affine seed vectors can be replaced by the affine seed vector of the lower right control point (the fourth affine seed vector).

[0190] The affine vector for each sub-block can be derived using an affine seed vector (S1103). Here, the affine vector represents the translational motion vector derived based on the affine seed vector. The affine vector of a sub-block can be referred to as the affine sub-block motion vector or the sub-block motion vector.

[0191] Figure 13 This is a diagram showing an example of the affine vectors of a sub-block under a 4-parameter motion model.

[0192] The affine vector of a sub-block can be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine sub-block vector.

[0193] Equation 3

[0194] In Equation 3, (x, y) represents the position of the sub-block. The position of the sub-block refers to the position of the reference sample included within it. The reference sample can be the sample located at the top left corner of the sub-block, or at least one sample located at the center of the x-axis or y-axis coordinate system. (x0, y0) represents the position of the first control point, and (sv... 0x sv 0y Let (x1, y1) represent the first affine seed vector. Additionally, (x1, y1) represents the position of the second control point, and (sv... 1x sv 1y ) represents the second affine seed vector.

[0195] When the first control point and the second control point correspond to the top left corner and the top right corner of the current block, respectively, x1-x0 can be set to the same value as the width of the current block.

[0196] Subsequently, motion compensation prediction for each sub-block can be performed using the affine vector of each sub-block (S1104). After performing motion compensation prediction, prediction blocks associated with each sub-block can be generated. The prediction blocks of the sub-blocks can be set as the prediction blocks of the current block.

[0197] Next, we will explain in detail the inter-frame prediction method that uses translational motion information.

[0198] Motion information for the current block can be derived from the motion information of other blocks. These other blocks can be those that are prioritized for inter-frame prediction encoding / decoding compared to the current block. Setting the motion information of the current block to be the same as that of other blocks is defined as a merging mode. Furthermore, setting the motion vectors of other blocks to the predicted values ​​of the motion vectors of the current block is defined as a motion vector prediction mode.

[0199] Figure 14 It is a flowchart demonstrating the process of exporting motion information of the current block using the merge mode.

[0200] Merging candidates for the current block can be exported (S1401). Merging candidates for the current block can be exported from blocks that were encoded / decoded using inter-frame prediction before the current block.

[0201] Figure 15 This is a diagram showing the candidate blocks used to derive the merge candidates.

[0202] Candidate blocks can include at least one of the following: neighboring blocks containing samples adjacent to the current block, or non-neighboring blocks containing samples not adjacent to the current block. Hereinafter, the samples used to determine candidate blocks will be designated as reference samples. Furthermore, reference samples adjacent to the current block will be referred to as neighboring reference samples, and reference samples not adjacent to the current block will be referred to as non-neighboring reference samples.

[0203] Adjacent reference samples can be included in the adjacent column of the leftmost column of the current block or the adjacent row of the topmost row of the current block. For example, if the coordinates of the top-left sample of the current block are (0, 0), then at least one of the following blocks—a block including a reference sample at position (-1, H-1), a block including a reference sample at position (W-1, -1), a block including a reference sample at position (W, -1), a block including a reference sample at position (-1, H), or a block including a reference sample at position (-1, -1)—can be used as candidate blocks. Referring to the accompanying drawings, adjacent blocks with indices 0 to 4 can be used as candidate blocks.

[0204] A non-adjacent reference sample refers to a sample whose x-axis distance or y-axis distance to the reference sample adjacent to the current block has a predefined value. For example, a block containing a reference sample whose x-axis distance to the left reference sample is a predefined value, a block containing a non-adjacent sample whose y-axis distance to the upper reference sample is a predefined value, or a block containing non-adjacent samples whose x-axis and y-axis distances to the upper-left reference sample are both predefined values ​​can be used as a candidate block. The predefined value can be an integer such as 4, 8, 12, 16, etc. Referring to the accompanying drawings, at least one of the blocks with indices from 5 to 26 can be used as a candidate block.

[0205] Samples that are not on the same vertical, horizontal, or diagonal line as adjacent reference samples can be set as non-adjacent reference samples.

[0206] Figure 16 This is a diagram showing the location of the reference sample.

[0207] As in Figure 16 In the example shown, the x-coordinate of the upper non-adjacent reference sample can be set to be different from the x-coordinate of the upper adjacent reference sample. For example, when the position of the upper adjacent reference sample is (W-1, -1), the position of the upper non-adjacent reference sample that is N away from the upper adjacent reference sample along the y-axis can be set to ((W / 2)-1, -1-N), and the position of the upper non-adjacent reference sample that is 2N away from the upper adjacent reference sample along the y-axis can be set to (0, -1-2N). That is, the position of the non-adjacent reference sample can be determined based on the position of the adjacent reference sample and the distance between the adjacent reference samples.

[0208] In the following text, a candidate block containing an adjacent reference sample is called a neighboring block, and a block containing a non-adjacent reference sample is called a non-adjacent block.

[0209] When the distance between the current block and a candidate block is greater than or equal to a threshold, the candidate block can be set as unusable as a merging candidate. The threshold can be determined based on the size of the coding tree unit. For example, the threshold can be set to the height of the coding tree unit (ctu_height), or the height of the coding tree unit plus or minus an offset value (e.g., ctu_height ± N). The offset value N is a predefined value in the encoder and decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0210] If the difference between the y-axis coordinate of the current block and the y-axis coordinate of the samples included in the candidate block is greater than a threshold, the candidate block can be determined as unsuitable for merging.

[0211] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block can be set as unsuitable for merging. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, candidate blocks that include the reference sample can be set as unsuitable for merging.

[0212] If the upper boundary of the current block is adjacent to the upper boundary of a coding tree unit, multiple candidate blocks will be determined as unsuitable for merging, which will reduce the encoding / decoding efficiency of the current block. To resolve this issue, candidate blocks can be configured such that the number of candidate blocks above the current block is greater than the number of candidate blocks to the left of the current block.

[0213] Figure 17 This is a diagram showing an example of a candidate block used to derive merge candidates.

[0214] As in Figure 17 In the example shown, the block above the current block (N blocks above it) and the block to the left of the current block (M blocks to its left) can be set as candidate blocks. In this case, by setting M to be greater than N, the number of left candidate blocks can be set to be greater than the number of top candidate blocks.

[0215] For example, the difference between the y-axis coordinate of the reference sample within the current block and the y-axis coordinate of the block above which can be used as a candidate block can be set to no more than N times the height of the current block. Additionally, the difference between the x-axis coordinate of the reference sample within the current block and the x-axis coordinate of the block to the left of which can be used as a candidate block can be set to no more than M times the width of the current block.

[0216] For example, in Figure 17 The example shown illustrates setting blocks belonging to the two blocks above the current block and blocks belonging to the five blocks to the left of the current block as candidate blocks.

[0217] As another example, when a candidate block does not belong to the same coding tree unit as the current block, a merge candidate can be derived by using a block that belongs to the same coding tree unit as the current block, or a block that contains a reference sample adjacent to the boundary of the coding tree unit, instead of the candidate block.

[0218] Figure 18 This is a diagram illustrating an example of changing the position of a reference sample.

[0219] When a reference sample is included in a coding tree unit that is different from the current block, and the reference sample is not adjacent to the boundary of the coding tree unit, a candidate block reference sample can be determined by using a reference sample adjacent to the boundary of the coding tree unit instead of the reference sample.

[0220] For example, in Figure 18 (a) and Figure 18In the example shown in (b), when the upper boundary of the current block touches the upper boundary of the coding tree unit, the reference sample above the current block belongs to a coding tree unit different from the current block. A reference sample belonging to a coding tree unit different from the current block that is not adjacent to the upper boundary of the coding tree unit can be replaced with a sample adjacent to the upper boundary of the coding tree unit.

[0221] For example, such as Figure 18 In the example shown in (a), the reference sample at position 6 is replaced with the sample at position 6' located at the upper boundary of the coding tree unit, as follows: Figure 18 As shown in example (b), the reference sample at position 15 is replaced by the sample at position 15', which is located at the upper boundary of the coding tree unit. In this case, the y-coordinate of the replacement sample can be changed to that of an adjacent position in the coding tree unit, and the x-coordinate of the replacement sample can be set to be the same as that of the reference sample. For example, the sample at position 6' can have the same x-coordinate as the sample at position 6, and the sample at position 15' can have the same x-coordinate as the sample at position 15.

[0222] Alternatively, the x-coordinate of the replacement sample can be set by adding or subtracting the offset value from the x-coordinate of the reference sample. For example, when the x-coordinates of adjacent and non-adjacent reference samples above the current block are the same, the x-coordinate of the replacement sample can be set by adding or subtracting the offset value from the x-coordinate of the reference sample. This is to prevent the replacement sample used to replace a non-adjacent reference sample from being in the same position as other non-adjacent or adjacent reference samples.

[0223] Figure 19 This is a diagram illustrating an example of changing the position of a reference sample.

[0224] When replacing a reference sample that is included in a different coding tree unit than the current block and is not adjacent to the boundary of the coding tree unit with a sample located at the boundary of the coding tree unit, the x-coordinate of the replacement sample can be set by adding or subtracting the offset value to the x-coordinate of the reference sample.

[0225] For example, in Figure 19 In the example shown, the reference sample at position 6 and the reference sample at position 15 can be replaced with the sample at position 6' and the sample at position 15', respectively, having the same y-coordinate as the row adjacent to the upper boundary of the coding tree unit. In this case, the x-coordinate of the sample at position 6' can be set to a value where the difference between its x-coordinate and the x-coordinate of the reference sample at position 6 is W / 2, and the x-coordinate of the sample at position 15' can be set to a value where the difference between its x-coordinate and the x-coordinate of the reference sample at position 15 is W-1.

[0226] Unlike Figure 18 and Figure 19In the example shown, the y-coordinate of the row above the top row of the current block or the y-coordinate of the upper boundary of the coding tree unit can also be set to the y-coordinate of the replacement sample.

[0227] Although not illustrated, the sample replacing the reference sample can also be determined based on the left boundary of the coding tree unit. For example, when the reference sample is not included in the same coding tree unit as the current block and is not adjacent to the left boundary of the coding tree unit, the reference sample can be replaced with a sample adjacent to the left boundary of the coding tree unit. In this case, the replacement sample can have the same y-coordinate as the reference sample, or it can have a y-coordinate obtained by adding or subtracting an offset value to the y-coordinate of the reference sample.

[0228] Then, the block containing the replacement sample can be set as a candidate block, and the merge candidates for the current block can be derived based on the candidate blocks.

[0229] Merge candidates can also be derived from temporally adjacent blocks included in images different from the current block. For example, merge candidates can be derived from co-located blocks included in co-located images.

[0230] The motion information of the merged candidate can be set to be the same as that of the candidate block. For example, at least one of the motion vector, reference image index, prediction direction, or bidirectional weighted index of the candidate block can be set as the motion information of the merged candidate.

[0231] A list of merge candidates, including merge candidates, can be generated (S1402). The merge candidates can be classified into adjacent merge candidates derived from adjacent blocks adjacent to the current block, and non-adjacent merge candidates derived from non-adjacent blocks.

[0232] The indices of multiple merge candidates within the merge candidate list can be assigned in a predetermined order. For example, the index assigned to an adjacent merge candidate can have a smaller value than the index assigned to a non-adjacent merge candidate. Alternatively, based on Figure 15 or Figure 17 The index shown for each block can be assigned to each merge candidate.

[0233] When the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates can be selected (S1403). At this time, information indicating whether the motion information of the current block is derived from adjacent merge candidates can be sent via a signal in the code stream. The information can be a 1-bit flag. For example, the syntax element isAdjancentMergeFlag indicating whether the motion information of the current block is derived from adjacent merge candidates can be sent via a signal in the code stream. When the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block can be derived based on adjacent merge candidates. On the other hand, when the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block can be derived based on non-adjacent merge candidates.

[0234] Table 1 shows the syntax table including the syntax element isAdjancentMergeFlag.

[0235] Table 1

[0236] Information specifying any one of multiple merge candidates can be sent via signals in the bitstream. For example, information indicating the index of any merge candidate included in the merge candidate list can be sent via signals in the bitstream.

[0237] When isAdjacentMergeflag is 1, the syntax element merge_idx can be signaled to determine which of the adjacent merge candidates is being merged. The maximum value of the syntax element merge_idx can be set to a value that is 1 greater than the difference between the number of adjacent merge candidates.

[0238] When isAdjacentMergeflag is 0, the syntax element NA_merge_idx can be signaled to determine any of the non-adjacent merge candidates. The syntax element NA_merge_idx indicates the value obtained by subtracting the index of the non-adjacent merge candidate from the number of adjacent merge candidates. The decoder can select a non-adjacent merge candidate by adding the number of adjacent merge candidates to the index determined by NA_merge_idx.

[0239] When the number of merge candidates in the merge candidate list is less than a threshold, merge candidates included in the inter-frame motion information list can be added to the merge candidate list. The threshold can be a value calculated from the maximum number of merge candidates the merge candidate list can include, or the maximum number of merge candidates minus an offset. The offset can be an integer such as 1 or 2. The inter-frame motion information list can include merge candidates derived based on blocks encoded / decoded prior to the current block.

[0240] The inter-frame motion information list includes merging candidates derived from blocks encoded / decoded based on inter-frame prediction within the current image. For example, the motion information of the merging candidates included in the inter-frame motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on inter-frame prediction. The motion information may include at least one of motion vectors, reference image indexes, prediction directions, or bidirectional weighted indexes.

[0241] For ease of explanation, the merging candidates included in the inter-frame motion information list are referred to as inter-frame merging candidates.

[0242] The maximum number of merge candidates that can be included in the inter-frame motion information list can be predefined in the encoder and decoder. For example, the maximum number of merge candidates that can be included in the inter-frame motion information list can be 1, 2, 3, 4, 5, 6, 7, 8 or greater (e.g., 16).

[0243] Alternatively, information representing the maximum number of merge candidates in the inter-frame motion information list can be transmitted via a signal in the bitstream. This information can be transmitted at the sequence level, image level, or strip level.

[0244] Alternatively, the maximum number of merged candidates for the inter-frame motion information list can be determined based on the image size, the strip size, or the size of the coding tree unit.

[0245] The inter-frame motion information list can be initialized at the level of images, stripes, tiles, blocks, coding tree units, or coding tree unit lines (rows or columns). For example, the inter-frame motion information list is also initialized during strip initialization, and it may not include any merge candidates.

[0246] Alternatively, information indicating whether to initialize the inter-frame motion information list can be sent via a signal in the bitstream. This information can be sent at the strip, tile, brick, or block level. A pre-configured inter-frame motion information list can be used before the information indicates initialization of the inter-frame motion information list.

[0247] Alternatively, information related to inter-frame merge candidates can be signaled via an image parameter set or a strip header. Even if the strip is initialized, the inter-frame motion information list can include initial inter-frame merge candidates. Thus, inter-frame merge candidates can be used for the first block encoded / decoded within a strip.

[0248] The blocks are encoded / decoded according to the encoding / decoding order, and multiple blocks encoded / decoded based on inter-frame prediction can be set as inter-frame merging candidates in sequence according to the encoding / decoding order.

[0249] Figure 20It is a flowchart illustrating the process of updating the list of inter-frame motion information.

[0250] When performing inter-frame prediction on the current block (S2001), inter-frame merging candidates can be derived based on the current block (S2002). The motion information of the inter-frame merging candidates can be set to be the same as the motion information of the current block.

[0251] When the inter-frame motion information list is empty (S2003), inter-frame merging candidates derived from the current block can be added to the inter-frame motion information list (S2004).

[0252] When the inter-frame merge candidate is already included in the inter-frame motion information list (S2003), a redundancy check can be performed on the motion information of the current block (or the inter-frame merge candidate derived from the current block) (S2005). The redundancy check is used to determine whether the motion information of the inter-frame merge candidates stored in the inter-frame motion information list is the same as the motion information of the current block. Redundancy checks can be performed on all inter-frame merge candidates stored in the inter-frame motion information list. Alternatively, redundancy checks can be performed on inter-frame merge candidates whose index is above or below a threshold among the inter-frame merge candidates stored in the inter-frame motion information list.

[0253] If inter-frame merge candidates with the same motion information as the current block are not included, inter-frame merge candidates derived from the current block can be added to the inter-frame motion information list (S2008). Whether inter-frame merge candidates are the same can be determined based on whether their motion information (e.g., motion vectors and / or reference image indexes, etc.) is the same.

[0254] In this case, when the maximum number of inter-frame merge candidates has been stored in the inter-frame motion information list (S2006), the earliest inter-frame merge candidate is deleted (S2007), and inter-frame merge candidates derived based on the current block can be added to the inter-frame motion information list (S2008).

[0255] Multiple inter-frame merge candidates can be identified based on their indices. When adding an inter-frame merge candidate derived from the current block to the inter-frame motion information list, the candidate is assigned the lowest index (e.g., 0), and the indices of already stored candidates can be incremented by 1. In this case, when the maximum number of inter-frame merge candidates is stored in the inter-frame motion information list, the candidate with the highest index is removed.

[0256] Alternatively, when adding inter-frame merge candidates derived from the current block to the inter-frame motion information list, the inter-frame merge candidate can be assigned the largest index. For example, if the number of inter-frame merge candidates already stored in the inter-frame motion information list is less than the maximum value, the inter-frame merge candidate can be assigned an index with the same value as the number of stored inter-frame merge candidates. Alternatively, if the number of inter-frame merge candidates already stored in the inter-frame motion information list is equal to the maximum value, the inter-frame merge candidate can be assigned an index that is 1 less than the maximum value. Furthermore, the inter-frame merge candidate with the smallest index is removed, and the indices of the remaining stored inter-frame merge candidates are each reduced by 1.

[0257] Figure 21 This is a diagram illustrating an embodiment of updating the inter-frame merging candidate list.

[0258] Assume that inter-frame merge candidates derived from the current block are added to the inter-frame merge candidate list, and the largest index is assigned to the inter-frame merge candidate. Also, assume that the inter-frame merge candidate list already stores the maximum number of inter-frame merge candidates.

[0259] When adding the inter-frame merge candidate HmvpCand[n+1] exported from the current block to the inter-frame merge candidate list HmvpCandList, the inter-frame merge candidate HmvpCand[0] with the smallest index is removed from the stored inter-frame merge candidates, and the indices of the remaining inter-frame merge candidates are decreased by 1 respectively. Alternatively, the index of the inter-frame merge candidate HmvpCand[n+1] exported from the current block can be set to the maximum value (in... Figure 21 In the example shown, n).

[0260] If an inter-frame merge candidate that is the same as the inter-frame merge candidate derived from the current block is already stored (S2005), the inter-frame merge candidate derived from the current block may not be added to the inter-frame motion information list (S2009).

[0261] Alternatively, as inter-frame merge candidates derived from the current block are added to the inter-frame motion information list, previously stored inter-frame merge candidates that are identical to those candidates can also be removed. In this case, the indexes of the previously stored inter-frame merge candidates will be updated.

[0262] Figure 22 This is a diagram illustrating an example of updating the index of a stored inter-frame merge candidate.

[0263] When the index of a stored inter-frame merge candidate that is the same as the inter-frame merge candidate mvCand derived based on the current block is hIdx, deleting the stored inter-frame merge candidate can reduce the index of each inter-frame merge candidate with an index greater than hIdx by 1. For example, in Figure 22The example shown illustrates removing HmvpCand[2], which is identical to mvCand, from the inter-frame motion information list HvmpCandList, and reducing the indices of HmvpCand[3] to HmvpCand[n] by 1.

[0264] Furthermore, inter-frame merge candidate mvCands derived based on the current block can be added to the end of the inter-frame motion information list.

[0265] Alternatively, the index of a stored inter-frame merge candidate that is assigned to the same inter-frame merge candidate derived based on the current block can be updated. For example, the index of a stored inter-frame merge candidate can be changed to the minimum or maximum value.

[0266] Motion information of blocks included in a predetermined region can be set to not be added to the inter-frame motion information list. For example, inter-frame merge candidates derived from the motion information of blocks included in the merge processing region cannot be added to the inter-frame motion information list. Since the encoding / decoding order of the blocks included in the merge processing region is not defined, it is inappropriate to use the motion information of any of these blocks for inter-frame prediction of other blocks. Therefore, inter-frame merge candidates derived from the blocks included in the merge processing region may not be added to the inter-frame motion information list.

[0267] When performing motion compensation prediction using sub-block units, inter-frame merging candidates can be derived from the motion information of representative sub-blocks within the current block. For example, when using sub-block merging candidates for the current block, inter-frame merging candidates can be derived from the motion information of representative sub-blocks within the sub-block.

[0268] The motion vector of a sub-block can be derived in the following order. First, any of the merge candidates included in the merge candidate list of the current block can be selected, and the initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. Then, by adding the initial shift vector to the positions (xSb, ySb) of the reference samples (e.g., the top-left sample or the middle sample) of each sub-block within the coded block, a shifted sub-block with reference sample positions (xColSb, yColSb) can be derived. Equation 4 below shows the equation used to derive the shifted sub-block.

[0269] Equation 4

[0270] Next, the motion vector of the co-position block corresponding to the center position of the sub-block including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0271] A representative sub-block can mean a sub-block that includes the top-left sample or the center sample of the current block.

[0272] Figure 23 This is a diagram showing the positions of the sub-blocks.

[0273] Figure 23 (a) shows an example of setting the child block located to the upper left of the current block as the representative child block. Figure 23 (b) illustrates an example of setting the sub-block located at the center of the current block as the representative sub-block. When performing motion compensation prediction on a sub-block basis, inter-frame merge candidates for the current block can be derived based on the motion vectors of sub-blocks that include the top-left sample of the current block or sub-blocks that include the center sample of the current block.

[0274] Based on the inter-frame prediction mode of the current block, it can also be determined whether the current block should be used as an inter-frame merging candidate. For example, blocks encoded / decoded based on an affine motion model can be set as non-inter-frame merging candidates. Thus, even if the current block is encoded / decoded using inter-frame prediction, the inter-frame prediction motion information list will not be updated based on the current block if the current block's inter-frame prediction mode is affine prediction mode.

[0275] Alternatively, inter-frame merge candidates can be derived from at least one sub-block vector within the sub-blocks included in the block being encoded / decoded based on an affine motion model. For example, an inter-frame merge candidate can be derived using a sub-block located to the upper left, center, or upper right of the current block. Alternatively, the average of the sub-block vectors of multiple sub-blocks can be used as the motion vector for the inter-frame merge candidate.

[0276] Alternatively, inter-frame merge candidates can be derived based on the average of the affine seed vectors of the blocks encoded / decoded using an affine motion model. For example, the average of at least one of the first, second, or third affine seed vectors of the current block can be set as the motion vector of the inter-frame merge candidate.

[0277] Alternatively, the inter-frame motion information list can be configured for different inter-frame prediction modes. For example, at least one of the following can be defined: an inter-frame motion information list for blocks encoded / decoded via intra-block copying, an inter-frame motion information list for blocks encoded / decoded based on a translational motion model, or an inter-frame motion information list for blocks encoded / decoded based on an affine motion model. Any one of the multiple inter-frame motion information lists can be selected depending on the inter-frame prediction mode of the current block.

[0278] Figure 24 A diagram showing an example of generating a list of inter-frame motion information for each inter-frame prediction mode is provided.

[0279] When encoding / decoding a block based on a non-affine motion model, inter-frame merge candidate `mvCand` derived from the block can be added to the inter-frame non-affine motion information list `HmvpCandList`. Conversely, when encoding / decoding a block based on an affine motion model, inter-frame merge candidate `mvAfCand` derived from the block can be added to the inter-frame affine motion information list `HmvpAfCandList`.

[0280] The affine seed vector of a block can be stored in an inter-frame merge candidate derived from the block encoded / decoded based on the affine motion model. Thus, the inter-frame merge candidate can be used as a merge candidate for deriving the affine seed vector of the current block.

[0281] In addition to the described list of inter-frame motion information, another list of inter-frame motion information can be defined. Besides the described list of inter-frame motion information (hereinafter referred to as the first inter-frame motion information list), a long-term motion information list (hereinafter referred to as the second inter-frame motion information list) can also be defined. The long-term motion information list includes long-term merging candidates.

[0282] When both the first and second inter-frame motion information lists are empty, inter-frame merge candidates can be added to the second inter-frame motion information list first. Only after the maximum number of available inter-frame merge candidates in the second inter-frame motion information list has been reached can inter-frame merge candidates be added to the first inter-frame motion information list.

[0283] Alternatively, an inter-frame merge candidate can be added to both the second inter-frame motion information list and the first inter-frame motion information list.

[0284] In this case, the already configured second inter-frame motion information list may no longer be updated. Alternatively, the second inter-frame motion information list may be updated when the decoded region is above a predetermined ratio of the stripes. Alternatively, the second inter-frame motion information list may be updated every N coding tree unit rows.

[0285] On the other hand, the first inter-frame motion information list can be updated whenever a block is generated using inter-frame prediction for encoding / decoding. However, inter-frame merge candidates added to the second inter-frame motion information list can also be set not to be used to update the first inter-frame motion information list.

[0286] Information for selecting either a first inter-frame motion information list or a second inter-frame motion information list can be transmitted via a signal in the bitstream. When the number of merge candidates included in the merge candidate list is less than a threshold, merge candidates included in the inter-frame motion information list indicated by the information can be added to the merge candidate list.

[0287] Alternatively, the list of inter-frame motion information can be selected based on the size and shape of the current block, the inter-frame prediction mode, whether bidirectional prediction is enabled or disabled, whether motion vectors are refined or not, or whether triangulation is enabled or not.

[0288] Alternatively, if the number of merge candidates included in the merge candidate list is still less than the maximum number of merges even after adding the inter-frame merge candidates included in the first inter-frame motion information list, then the inter-frame merge candidates included in the second inter-frame motion information list can be added to the merge candidate list.

[0289] Figure 25 This is a diagram illustrating an example of adding inter-frame merge candidates included in the long-term motion information list to the merge candidate list.

[0290] If the number of merge candidates in the merge candidate list is less than the maximum number, inter-frame merge candidates included in the first inter-frame motion information list HmvpCandList can be added to the merge candidate list. Even if the number of merge candidates in the merge candidate list is still less than the maximum number after adding inter-frame merge candidates included in the first inter-frame motion information list, then inter-frame merge candidates included in the long-term motion information list HmvpLTCandList can be added to the merge candidate list.

[0291] Table 2 illustrates the process of adding inter-frame merging candidates, which are included in the long-term motion information list, to the merge candidate list.

[0292] Table 2

[0293] Inter-frame merge candidates can be configured to include additional information besides motion information. For example, the size, shape, or partitioning information of storage blocks can be added to the inter-frame merge candidates. When constructing the merge candidate list for the current block, only inter-frame merge candidates with the same or similar size, shape, or partitioning information as the current block are used in the inter-frame merge candidate list, or inter-frame merge candidates with the same or similar size, shape, or partitioning information as the current block are preferentially added to the merge candidate list.

[0294] Alternatively, inter-frame motion information lists can be generated for different block sizes, shapes, or partitioning information. Multiple inter-frame motion information lists corresponding to the shape, size, or partitioning information of the current block can be used to generate a merge candidate list for the current block.

[0295] If the number of merge candidates in the current block's merge candidate list is less than a threshold, inter-frame merge candidates included in the inter-frame motion information list can be added to the merge candidate list. The addition process is performed in ascending or descending order of the index. For example, the inter-frame merge candidate with the largest index can be added to the merge candidate list.

[0296] When adding inter-frame merge candidates included in the inter-frame motion information list to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidates and multiple merge candidates already stored in the merge candidate list.

[0297] For example, Table 3 shows the process of adding inter-frame merge candidates to the merge candidate list.

[0298] Table 3

[0299] Redundancy checks can also be performed only on some of the inter-frame merge candidates included in the inter-frame motion information list. For example, redundancy checks can be performed only on inter-frame merge candidates with indices above or below a threshold. Alternatively, redundancy checks can be performed only on the N merge candidates with the largest indices or the N merge candidates with the smallest indices.

[0300] Alternatively, redundancy checks can be performed only on some of the merge candidates already stored in the merge candidate list. For example, redundancy checks can be performed only on merge candidates with an index above or below a threshold, or on merge candidates derived from a block at a specific location. A specific location may include at least one of the current block's left neighbor, top neighbor, top-right neighbor, or bottom-left neighbor.

[0301] Figure 26 This is a diagram illustrating an example of performing redundancy checks only on some merge candidates.

[0302] When adding an inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the two merge candidates with the largest indices, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1]. Here, NumMerge represents the number of available spatial and temporal merge candidates.

[0303] Unlike the example shown in the figure, when adding an inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can also be performed between the inter-frame merge candidate and the two merge candidates with the smallest index. For example, it can be verified whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, a redundancy check can be performed only on merge candidates derived from a specific location. For example, a redundancy check can be performed only on at least one of the merge candidates derived from the adjacent block to the left of the current block or the merge candidate derived from the adjacent block above the current block. When there is no merge candidate derived from a specific location in the merge candidate list, the inter-frame merge candidate can be added to the merge candidate list without performing a redundancy check.

[0304] If a merge candidate that is identical to the first inter-frame merge candidate is found, the redundancy check of the merge candidate that is identical to the first inter-frame merge candidate can be omitted when performing a redundancy check on the second inter-frame merge candidate.

[0305] Figure 27 This is a diagram illustrating an example of omitting redundancy checks for a specific merge candidate.

[0306] When adding the inter-frame merge candidate HmvpCand[i] at index i to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the merge candidates already stored in the merge candidate list. In this case, if a merge candidate mergeCandList[j] identical to the inter-frame merge candidate HmvpCand[i] is found, the inter-frame merge candidate HmvpCand[i] will not be added to the merge candidate list, and a redundancy check can be performed between the inter-frame merge candidate HmvpCand[i-1] at index i-1 and the merge candidate. In this case, the redundancy check between the inter-frame merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0307] For example, in Figure 27 In the example shown, HmvpCand[i] is determined to be the same as mergeCandList[2]. Therefore, HmvpCand[i] is not added to the merge candidate list, and a redundancy check can be performed on HmvpCand[i-1]. In this case, the redundancy check between HvmpCand[i-1] and mergeCandList[2] can be omitted.

[0308] When the number of merge candidates in the current block's merge candidate list is less than a threshold, in addition to inter-frame merge candidates, at least one of pairwise merge candidates or zero merge candidates may be included. Pairwise merge candidates are those whose motion vectors are averaged from two or more merge candidates, while zero merge candidates are those whose motion vectors are zero.

[0309] Merge candidates for the current block can be added in the following order.

[0310] Spatial merge candidate - Temporal merge candidate - Inter-frame merge candidate - (Inter-frame affine merge candidate) - Paired merge candidate - Zero merge candidate.

[0311] Spatial merge candidates refer to merge candidates derived from at least one of adjacent or non-adjacent blocks, while temporal merge candidates refer to merge candidates derived from a previous reference image. The inter-frame affine merge candidate column represents inter-frame merge candidates derived from blocks encoded / decoded using an affine motion model.

[0312] The inter-frame motion information list can also be used in advanced motion vector prediction mode. For example, if the number of motion vector prediction candidates included in the current block's motion vector prediction candidate list is less than a threshold, the inter-frame merging candidates included in the inter-frame motion information list are set as motion vector prediction candidates related to the current block. Specifically, the motion vectors of the inter-frame merging candidates are set as motion vector prediction candidates.

[0313] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list for the current block is selected, the selected candidate is set as the motion vector prediction value for the current block. After decoding the motion vector residual value for the current block, the motion vector for the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0314] The candidate list for motion vector prediction of the current block can be constructed in the following order.

[0315] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Inter-frame decoding region merging candidate - (Inter-frame decoding region affine merging candidate) - Zero motion vector prediction candidate.

[0316] Spatial motion vector prediction candidates refer to motion vector prediction candidates derived from at least one of neighboring or non-neighboring blocks, while temporal motion vector prediction candidates refer to motion vector prediction candidates derived from a previous reference image. The inter-frame affine merging candidate column represents inter-frame motion vector prediction candidates derived from blocks encoded / decoded using an affine motion model. Zero motion vector prediction candidates represent candidates with a motion vector value of 0.

[0317] When a merge candidate is selected for the current block, the motion vector of the selected merge candidate is set as the initial motion vector, and motion compensation prediction for the current block can be performed using the motion vector derived by adding or subtracting the offset vector from the initial motion vector. Deriving a new motion vector by adding or subtracting the offset vector from the motion vector of the merge candidate can be defined as a merge motion difference coding method.

[0318] Information indicating whether the merge offset encoding method is used can be transmitted via signals in the bitstream. This information can be a 1-bit flag, `merge_offset_vector_flag`. For example, a value of 1 for `merge_offset_vector_flag` indicates that the merge motion interpolation encoding method is applied to the current block. When the merge motion interpolation encoding method is applied to the current block, the motion vector of the current block can be derived by adding or subtracting the offset vector from the motion vector of the merge candidate. A value of 0 for `merge_offset_vector_flag` indicates that the merge motion interpolation encoding method is not applied to the current block. When the merge offset encoding method is not applied, the motion vector of the merge candidate can be set as the motion vector of the current block.

[0319] The flag can only be signaled when the value of the skip flag indicating whether to apply the skip mode or the value of the merge flag indicating whether to apply the merge mode. For example, when the value of skip_flag indicating whether to apply the skip mode to the current block is 1, or when the value of merge_flag indicating whether to apply the merge mode to the current block is 1, merge_offset_vector_flag can be encoded and signaled.

[0320] When it is determined that the merge offset encoding method will be applied to the current block, at least one of the following can be signaled: information specifying any of the merge candidates included in the merge candidate list, information indicating the magnitude of the offset vector, and information indicating the direction of the offset vector.

[0321] Information for determining the maximum number of merge candidates that can be included in the merge candidate list can be sent via a bitstream using signals. For example, the maximum number of merge candidates that can be included in the merge candidate list can be set to an integer of 6 or less.

[0322] When it is determined that the merge offset encoding method will be applied to the current block, only a preset maximum number of merge candidates can be set as the initial motion vector of the current block. That is, the number of merge candidates that the current block can use can be adaptively determined depending on whether the merge offset encoding method is applied. For example, when the value of `merge_offset_vector_flag` is set to 0, the maximum number of merge candidates that the current block can use can be set to M, while when the value of `merge_offset_vector_flag` is set to 1, the maximum number of merge candidates that the current block can use can be set to N. Here, M represents the maximum number of merge candidates that can be included in the merge candidate list, and N represents an integer equal to or less than M.

[0323] For example, when M is 6 and N is 2, the two merge candidates with the smallest indices in the merge candidate list can be set as available for the current block. Therefore, the motion vector of the merge candidate with index 0 or the motion vector of the merge candidate with index 1 can be set as the initial motion vector for the current block. When M and N are the same (e.g., when M and N are 2), all merge candidates in the merge candidate list can be set as available for the current block.

[0324] Alternatively, whether adjacent blocks can be used as merge candidates can be determined based on whether the merge motion interpolation coding method is applied to the current block. For example, when the value of merge_offset_vector_flag is 1, at least one of the adjacent blocks adjacent to the top-right corner, the top-left corner, and the bottom-left corner of the current block can be set as unusable as merge candidates. Therefore, when the merge motion interpolation coding method is applied to the current block, the motion vector of at least one of the adjacent blocks adjacent to the top-right corner, the top-left corner, and the bottom-left corner of the current block cannot be set as the initial motion vector. Alternatively, when the value of merge_offset_vector_flag is 1, the temporal adjacent blocks of the current block can be set as unusable as merge candidates.

[0325] When the merge motion difference encoding method is applied to the current block, it can be configured not to use at least one of the pairwise merge candidates and the zero merge candidate. Therefore, when the value of merge_offset_vector_flag is 1, even if the number of merge candidates included in the merge candidate list is less than the maximum number, at least one of the pairwise merge candidates or the zero merge candidate may not be added to the merge candidate list.

[0326] The motion vector of a merge candidate can be set as the initial motion vector of the current block. In this case, when there are multiple merge candidates available for the current block, information specifying any one of the multiple merge candidates can be sent via signaling in the bitstream. For example, when the maximum number of merge candidates that the merge candidate list can include is greater than one, information indicating any one of the multiple merge candidates, `merge_idx`, can be sent via signaling in the bitstream. That is, under the merge offset coding method, a merge candidate can be specified using the information `merge_idx` to specify any one of the multiple merge candidates. The initial motion vector of the current block can be set as the motion vector of the merge candidate indicated by `merge_idx`.

[0327] On the other hand, when the number of merge candidates available for the current block is one, the signaling of information specifying the merge candidate can be omitted. For example, when the maximum number of merge candidates that the merge candidate list can include is no greater than one, the signaling of the merge_idx information specifying the merge candidate can be omitted. That is, under the merge offset encoding method, when a merge candidate is included in the merge candidate list, the encoding of the merge_idx information specifying the merge candidate can be omitted, and the initial motion vector can be determined based on the merge candidates included in the merge candidate list. The motion vector of the merge candidate can be set as the initial motion vector of the current block.

[0328] As another example, after determining the merge candidates for the current block, it can be determined whether to apply the merge motion interpolation coding method to the current block. For example, when the maximum number of merge candidates that the merge candidate list can include is greater than 1, the information merge_idx, which specifies any of the merge candidates, can be signaled. After selecting a merge candidate based on merge_idx, the merge_offset_vector_flag indicating whether to apply the merge motion interpolation coding method to the current block can be decoded. Table 4 is a diagram illustrating the syntax table according to the above embodiment.

[0329] Table 4

[0330] As another example, after determining the merge candidates for the current block, it can be determined whether to apply the merge motion interpolation (MIO) method to the current block only if the index of the determined merge candidate is less than the maximum number of merge candidates that can be used when applying the merge motion interpolation method. For example, the merge_offset_vector_flag, which indicates whether to apply the merge motion interpolation method to the current block, can only be encoded and signaled if the value of the index information merge_idx is less than N. When the value of the index information merge_idx is equal to or greater than N, the encoding of merge_offset_vector_flag can be omitted. If the encoding of merge_offset_vector_flag is omitted, it can be determined that the merge motion interpolation method has not been applied to the current block.

[0331] Alternatively, after determining the merge candidates for the current block, it is possible to consider whether the determined merge candidates have bidirectional or unidirectional motion information to determine whether to apply the merge motion difference encoding method to the current block. For example, the merge_offset_vector_flag indicating whether to apply the merge motion difference encoding method to the current block is encoded and signaled only if the value of the index information merge_idx is less than N and the merge candidate selected by the index information has bidirectional motion information. Alternatively, the merge_offset_vector_flag indicating whether to apply the merge motion difference encoding method to the current block is encoded and signaled only if the value of the index information merge_idx is less than N and the merge candidate selected by the index information has unidirectional motion information.

[0332] Alternatively, the decision to apply the merge motion interpolation coding method can be based on at least one of the following: the size of the current block, the shape of the current block, and whether the current block is in contact with the boundary of a coding tree unit. When at least one of the following conditions is not met, the encoding of the merge_offset_vector_flag indicating whether to apply the merge motion interpolation coding method to the current block can be omitted.

[0333] When a merge candidate is selected, its motion vector can be set as the initial motion vector of the current block. Then, information indicating the magnitude and direction of the offset vector can be decoded to determine the offset vector. The offset vector can have a horizontal or vertical component.

[0334] The information indicating the magnitude of the offset vector can be index information indicating any of the motion offset magnitude candidates. For example, the index information distance_idx indicating any of the motion offset magnitude candidates can be transmitted via a signal in the bitstream. Table 5 shows the binarization of the index information distance_idx and the value of the variable DistFromMergeMV used to determine the magnitude of the offset vector based on distance_idx.

[0335] Table 5

[0336] The size of the offset vector can be derived by dividing the variable DistFromMergeMV by a preset value. Equation 5 shows an example of determining the size of the offset vector.

[0337] Equation 5

[0338] According to Equation 5, the value obtained by dividing the variable DistFromMergeMV by 4 or by shifting the variable DistFromMergeMV to the left by 2 can be set as the size of the offset vector.

[0339] You can use more or fewer motion offset size candidates compared to the examples shown in Table 5, or you can set the number of motion vector offset size candidates to be different from the examples shown in Table 5. For example, the magnitude of the horizontal or vertical component of the offset vector can be set to no more than 2 sample distances. Table 6 shows the binarization of the index information distance_idx and the value of the variable DistFromMergeMV used to determine the magnitude of the offset vector based on distance_idx.

[0340] Table 6

[0341] Alternatively, the number of candidate values ​​for the motion vector offset size can be set differently based on the motion vector precision. For example, when the motion vector precision of the current block is fractional pixels, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 1, 2, 4, 8, 16, etc. Here, fractional pixels include at least one of 1 / 16 pixel, one-eighth pixel, one-quarter pixel, or half pixel. On the other hand, when the motion vector precision of the current block is integer pixels, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 4, 8, 16, 32, 64, etc. That is, the table used to determine the variable DistFromMergeMV can be set differently depending on the motion vector precision of the current block.

[0342] For example, when the motion vector precision of the current block or merge candidate is a quarter pixel, the variable DistFromMergeMV, represented by distance_idx, can be derived using Table 5. On the other hand, when the motion vector precision of the current block or merge candidate is an integer pixel, the value of the variable DistFromMergeMV can be derived by taking N times (e.g., 4 times) the value of the variable DistFromMergeMV indicated by distance_idx in Table 5.

[0343] Information used to determine motion vector precision can be transmitted via signals in the bitstream. For example, this information can be transmitted at the sequence level, image level, slice level, or block level. Therefore, the number of motion offset size candidates can be set differently based on the motion vector precision-related information transmitted via signals in the bitstream. Alternatively, motion vector precision can be determined based on the merging candidates for the current block. For example, the motion vector precision for the current block can be set to the same as the motion vector precision of the merging candidates.

[0344] Alternatively, information for determining the search range of the offset vector can be signaled via the bitstream. At least one of the following can be determined based on the search range: the number of motion offset size candidates, the minimum of the motion offset size candidates, and the maximum of the motion offset size candidates. For example, a flag `merge_offset_vector_flag` for determining the search range of the offset vector can be signaled via the bitstream. This information can be signaled via a sequence header, image header, or stripe header.

[0345] For example, when the value of merge_offset_extend_range_flag is 0, the size of the offset vector can be set to no more than 2. Therefore, the maximum value of DistFromMergeMV can be set to 8. On the other hand, when the value of merge_offset_extend_range_flag is 1, the size of the offset vector can be set to no more than 32 sample distances. Therefore, the maximum value of DistFromMergeMV can be set to 128.

[0346] The size of the offset vector can be determined using a flag indicating whether its size is greater than a threshold. For example, the flag `distance_flag` indicating whether the offset vector's size is greater than a threshold can be signaled via the bitstream. The threshold can be 1, 2, 4, 8, or 16. For example, a `distance_flag` of 1 indicates that the offset vector's size is greater than 4. Conversely, a `distance_flag` of 0 indicates that the offset vector's size is 4 or less.

[0347] When the size of the offset vector is greater than the threshold, the difference between the offset vector size and the threshold can be derived using the index information distance_idx. Alternatively, when the size of the offset vector is less than or equal to the threshold, the size of the offset vector can be determined using the index information distance_idx. Table 7 is a syntax table illustrating the encoding process of distance_flag and distance_idx.

[0348] Table 7

[0349] Equation 6 shows an example of using distance_flag and distance_idx to derive the variable DistFromMergeMV for determining the size of the offset vector.

[0350] Equation 6

[0351] In Equation 6, the value of distance_flag can be set to 1 or 0. The value of distance_idx can be set to 1, 2, 4, 8, 16, 32, 64, 128, etc. N represents a coefficient determined by the threshold. For example, when the threshold is 4, N can be set to 16.

[0352] The information indicating the direction of the offset vector can be index information indicating any of the vector direction candidates. For example, the index information direction_idx indicating any of the vector direction candidates can be transmitted as a signal via the bitstream. Table 8 shows the binarization of the index information direction_idx and the direction of the offset vector according to direction_idx.

[0353] Table 8

[0354] In Table 8, sign[0] indicates the horizontal direction, and sign[1] indicates the vertical direction. +1 indicates that the x-component or y-component of the offset vector is positive (+), and -1 indicates that the x-component or y-component of the offset vector is negative (-). Equation 7 shows an example of determining the offset vector based on its magnitude and direction.

[0355] Equation 7

[0356] In Equation 7, offsetMV[0] indicates the vertical component of the offset vector, and offsetMV[1] indicates the horizontal component of the offset vector.

[0357] Figure 28 This is a graph showing the offset vector based on the values ​​of distance_idx, which indicates the magnitude of the offset vector, and direction_idx, which indicates the direction of the offset vector.

[0358] As in Figure 28 In the example shown, the magnitude and direction of the offset vector can be determined based on the values ​​of distance_idx and direction_idx. The maximum size of the offset vector can be set to not exceed a threshold. Here, the threshold can have a value predefined by the encoder and decoder. For example, the threshold could be 32 sample distances. Alternatively, the threshold can be determined based on the magnitude of the initial motion vector. For example, the horizontal threshold can be set based on the magnitude of the horizontal component of the initial motion vector, and the vertical threshold can be set based on the magnitude of the vertical component of the initial motion vector.

[0359] When the merging candidate has bidirectional motion information, the L0 motion vector of the merging candidate can be set as the L0 initial motion vector of the current block, and the L1 motion vector of the merging candidate can be set as the L1 initial motion vector of the current block. In this case, the L0 offset vector and L1 offset vector can be determined by taking into account the output order difference (hereinafter referred to as L0 difference) between the L0 reference image of the merging candidate and the current image and the output order difference (hereinafter referred to as L1 difference) between the L1 reference image of the merging candidate and the current image.

[0360] First, when the L0 and L1 differences have the same sign, the L0 and L1 offset vectors can be set to be the same. On the other hand, when the L0 and L1 differences have different signs, the L1 offset vector can be set in the opposite direction to the L0 offset vector.

[0361] The sizes of the L0 offset vector and the L1 offset vector can be set to be the same. Alternatively, the size of the L1 offset vector can be determined by scaling the L0 offset vector based on the L0 difference and the L1 difference.

[0362] For example, Equation 8 shows the L0 offset vector and L1 offset vector when the signs of the L0 difference and L1 difference are the same.

[0363] Equation 8

[0364] In Equation 8, offsetMVL0[0] indicates the horizontal component of the L0 offset vector, and offsetMVL0[1] indicates the vertical component of the L0 offset vector. offsetMVL1[0] indicates the horizontal component of the L1 offset vector, and offsetMVL1[1] indicates the vertical component of the L1 offset vector.

[0365] Equation 9 shows the L0 offset vector and L1 offset vector when the signs of the L0 difference and L1 difference are different.

[0366] Equation 9

[0367] More than four vector direction candidates can also be defined. Tables 9 and 10 show examples of defining eight vector direction candidates.

[0368] Table 9

[0369] Table 10

[0370] In Tables 9 and 10, an absolute value greater than 0 for sign[0] and sign[1] indicates that the offset vector is in the diagonal direction. When using Table 9, the magnitudes of the x-axis and y-axis components of the diagonal offset vector are set to abs(offsetMV), while when using Table 10, the magnitudes of the x-axis and y-axis components of the diagonal offset vector are set to abs(offsetMV / 2).

[0371] Figure 29 This is a graph showing the offset vector based on the values ​​of distance_idx, which indicates the magnitude of the offset vector, and direction_idx, which indicates the direction of the offset vector.

[0372] Figure 29 (a) is an example of applying Table 9, and Figure 29 (b) is an example of applying Table 10.

[0373] Information for determining at least one of the number or size of vector direction candidates can be transmitted via signals in the bitstream. For example, the flag `merge_offset_direction_range_flag` for determining vector direction candidates can be transmitted via signals in the bitstream. The flag can be transmitted via signals at the sequence level, image level, or strip level. For example, when the flag value is 0, the four vector direction candidates exemplified in Table 8 can be used. On the other hand, when the flag value is 1, the eight vector direction candidates exemplified in Table 9 or Table 10 can be used.

[0374] Alternatively, at least one of the number or size of vector direction candidates can be determined based on the magnitude of the offset vector. For example, when the value of the variable DistFromMergeMV, used to determine the magnitude of the offset vector, is equal to or less than a threshold, the eight vector direction candidates exemplified in Table 9 or Table 10 can be used. On the other hand, when the value of the variable DistFromMergeMV is greater than the threshold, the four vector direction candidates exemplified in Table 8 can be used.

[0375] Alternatively, at least one of the number or size of vector direction candidates can be determined based on the values ​​of the x-component MVx and the y-component MVy of the initial motion vector. For example, when the difference or absolute value of the difference between MVx and MVy is less than or equal to a threshold, the eight vector direction candidates exemplified in Table 9 or Table 10 can be used. On the other hand, when the difference or absolute value of the difference between MVx and MVy is greater than a threshold, the four vector direction candidates exemplified in Table 8 can be used.

[0376] The motion vector of the current block can be derived by adding the offset vector to the initial motion vector. Equation 10 shows an example of determining the motion vector of the current block.

[0377] Equation 10

[0378] In Equation 10, mvL0 indicates the L0 motion vector of the current block, and mvL1 indicates the L1 motion vector of the current block. mergeMVL0 indicates the initial L0 motion vector of the current block (i.e., merging candidate L0 motion vectors), and mergeMVL1 indicates the initial L1 motion vector of the current block. [0] indicates the horizontal component of the motion vector, and [1] indicates the vertical component of the motion vector.

[0379] Even when performing inter-frame prediction for each sub-unit after dividing the coding block into multiple sub-units, the merged motion difference coding method can be applied. Here, performing inter-frame prediction on a sub-unit basis can include at least one of Advanced Temporal Motion Vector Prediction (ATMVP), Spatiotemporal Motion Vector Prediction (STMVP), and triangle partitioning techniques.

[0380] For example, in the ATMVP method, the initial motion vector can be derived as follows.

[0381] First, the initial shift vector can be derived using the motion vectors of the merge candidates derived from the adjacent blocks adjacent to the coded block. Additionally, the initial shift vector can be used to derive the shift blocks of the sub-blocks included in the coded block. Equation 11 shows the positions of the shift blocks.

[0382] Equation 11

[0383] In Equation 11, (xColSb, yColSb) represents the position of the top-left sample of the shift block, and (xSb, ySb) represents the position of the top-left sample of the sub-block. shVector represents the shift vector.

[0384] When determining the shift block, the motion vector of the co-position block located at the same position as the shift block in the co-position image can be set as the motion vector of the sub-block. That is, the motion vector of the co-position block of the sample included at position (xColSb, yColSb) in the co-position block can be set as the motion vector of the sub-block of the sample included at position (xSb, ySb).

[0385] When applying the triangular partitioning technique, a coding block can be divided into triangular sub-units. For example, the coding block can be divided into two sub-units by connecting the top-left and bottom-right diagonals or by connecting the top-right and bottom-left diagonals.

[0386] Figure 30 This is a diagram illustrating the partitioning pattern of coded blocks when the triangulation technique is applied.

[0387] Motion information for each triangular sub-unit can be specified by a merge candidate. To do this, index information indicating any of the merge candidates can be sent for each sub-unit. For example, the index information `merge_1st_idx` for the first sub-unit can specify the merge candidate for the first sub-unit, and the index information `merge_2nd_idx` for the second sub-unit can specify the merge candidate for the second sub-unit.

[0388] The initial motion vector for each sub-unit can be determined individually. For example, when applying an affine motion model to a coding block, the affine vector of the sub-block derived from the affine seed vector of the coding block can be set as the initial motion vector of the sub-block. The motion vector of each sub-block can be derived by adding or subtracting the offset vector from the initial motion vector.

[0389] When applying the merged motion difference coding method to a coding block divided into multiple sub-units, multiple sub-units can be configured to use the same offset vector. That is, the initial motion vector of each sub-unit in multiple sub-units can be changed using the same offset vector.

[0390] Alternatively, the coded block can be divided into multiple sub-units, and the offset vector of each sub-unit can be determined individually. Therefore, the offset vector of at least one sub-unit can be set to be different from the offset vector of another sub-unit.

[0391] Figure 31 This is a diagram showing examples of different settings for the offset vector of each sub-unit.

[0392] As in Figure 31 In the example shown, information indicating the magnitude of the offset vector, distance_idx, and information indicating the direction of the offset vector, direction_idx, can be encoded for each sub-cell and sent as signals.

[0393] Alternatively, the magnitude of the offset vector for all sub-units can be set to be the same, and the direction of the offset vector can be set individually for these sub-units. For example, the value of distance_idx, which is signaled at the coding level, can be shared among the sub-units, and direction_idx can be encoded and signaled for each sub-unit.

[0394] Alternatively, the direction of the offset vector for all sub-units can be set to be the same, and the magnitude of the offset vector can be set individually for these sub-units. For example, the value of direction_idx, which is signaled at the coding level, can be shared among the sub-units, and distance_idx can be encoded and signaled for each sub-unit.

[0395] The motion difference coding method can be applied only to some sub-units among multiple sub-units generated by dividing the coding block. For example, when the current block is divided into a first sub-unit and a second sub-unit, the motion vector of the first sub-unit can be set to be the same as the motion vector of the merging candidate, and the motion vector of the second sub-unit can be derived by adding the offset vector to the motion vector of the merging candidate.

[0396] Instead of sending signals to determine the offset vector, the decoder can derive the offset vector. Specifically, the offset vector can be derived using the average of the horizontal and vertical gradients of the predicted samples included in the sub-block.

[0397] Here, gradients can be derived based on the difference between a reconstructed sample corresponding to a predicted sample in a reference image and its neighboring samples. For example, a horizontal gradient can indicate the difference between a reconstructed sample and its neighboring left and / or right reconstructed samples, and a vertical gradient can indicate the difference between a reconstructed sample and its neighboring upper and / or lower reconstructed samples.

[0398] Among the merge candidates included in the merge candidate list, merge candidates with motion vectors derived by adding or subtracting an offset vector from the motion vector of a reference merge candidate included in the merge candidate list can be added to the merge candidate list. Merge candidates with motion vectors derived by adding or subtracting an offset vector from the motion vector of a reference merge candidate can be referred to as refined merge candidates.

[0399] The motion information, except for the motion vectors of the fine-grained merged candidates, can be set to be the same as the motion information of the reference merged candidates.

[0400] Figure 32 This is a diagram showing the motion vector candidates that can be used for fine-grained merging.

[0401] When the motion vector of the reference merging candidate is (MvLX[0], MvLX[1]), the motion vector of the fine merging candidate can be derived by adding or subtracting an offset from at least one of the x-component or y-component of the motion vector of the reference merging candidate. For example, the motion vector of the fine merging candidate can be set to (MvLX[0]+M, MvLX[1]), (MvLX[0]-M, MvLX[1]), (MvLX[0], MvLX[1]+M), or (MvLX[0], MvLX[1]-M). M represents the magnitude of the offset vector.

[0402] A reference merge candidate can be a merge candidate with a predefined index value in the merge candidate list. For example, the merge candidate with the smallest index value (i.e., the merge candidate with index value 0) or the merge candidate with the largest index value in the merge candidate list can be set as the reference merge candidate. Alternatively, the inter-frame merge candidate with the smallest index value or the inter-frame merge candidate with the largest index value in the inter-frame motion information list can be set as the reference merge candidate.

[0403] Alternatively, the merge candidate with the smallest index value among the merge candidates that have bidirectional motion information can be set as the reference merge candidate. That is, when searching candidate blocks sequentially, the first bidirectional merge candidate found can be set as the reference merge candidate.

[0404] Basic merge candidates can be selected based on the size of the current block, the shape of the current block, or whether the current block touches the boundary of a coding tree unit. For example, when the current block is a square shape or a non-square shape where the height is greater than the width, the merge candidate with index 0 or a merge candidate derived from the adjacent block above the current block can be set as a reference merge candidate. When the current block is a non-square shape where the width is greater than the height, the merge candidate with index 1 or a merge candidate derived from the adjacent block to the left of the current block can be set as a reference merge candidate.

[0405] Alternatively, information specifying a reference merge candidate can be transmitted via signaling through the bitstream. This information may be index information for any of the merge candidates included in the specified merge candidate list.

[0406] Information indicating whether to use a fine-grained merge candidate can be sent via a signal in the bitstream. This information can be a 1-bit flag. When the flag is 1, a fine-grained merge candidate generated based on a reference merge candidate can be added to the merge candidate list. Conversely, when the flag is 0, the merge candidate list may not include a fine-grained merge candidate.

[0407] Alternatively, fine-grained merge candidates may be added to the merge candidate list when the number of merge candidates already added is less than the maximum number of merge candidates that the merge candidate list can include. Here, the added merge candidates may include at least one of spatial merge candidates, temporal merge candidates, inter-frame merge candidates, or paired merge candidates. For example, fine-grained merge candidates may be added to the merge candidate list when the number of at least one of spatial merge candidates, temporal merge candidates, and inter-frame merge candidates included in the merge candidate list is less than or equal to a threshold.

[0408] Alternatively, fine-grained merge candidates can be used when the number of merge candidates added to the merge candidate list is greater than or equal to a threshold.

[0409] The maximum number of merge candidates that can be included in the merge candidate list can be set differently depending on whether fine-grained merge candidates are used. For example, when fine-grained merge candidates are not used, the maximum number of merge candidates that can be included in the merge candidate list can be set to N, while when fine-grained merge candidates are used, the maximum number of merge candidates that can be included in the merge candidate list can be set to N + n.

[0410] The index of a fine-grained merge candidate can be greater than the index of a merge candidate already added to the merge candidate list. For example, Table 11 shows an example of configuring the merge candidate list.

[0411] Table 11

[0412] In Table 11, mergeCand[X] represents the merge candidate with index X. MvLX[0] represents the x-component motion vector of the reference merge candidate, and MvLX[1] represents the y-component motion vector of the reference merge candidate. For example, when the reference merge candidate is mergeCand[0], MvLX[0] and MvLX[1] can represent the motion vector of mergeCand[0].

[0413] The size M of the offset vector can be predefined in the encoder and decoder. For example, the size M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0414] Alternatively, information for determining the offset vector can be transmitted via signaling in the bitstream. This information can be transmitted at the sequence level, image level, strip level, or block level. For example, the offset vector can be determined using at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector, as described above.

[0415] As shown in the example in Table 11, at least one fine merge candidate derived from the reference merge candidate can be added to the merge candidate list. If there is a merge candidate among the added merge candidates that has the same motion information as the fine merge candidate, the fine merge candidate may not be added to the merge candidate list. For example, when the fine merge candidate derived from the reference merge candidate mergeCand[0] is the same as any one of mergeCand[1] to mergeCand[5], the fine merge candidate may not be added to the merge candidate list.

[0416] Alternatively, when a merge candidate with the same motion information as the fine merge candidate exists, the fine merge candidate can be re-derived by changing the offset vector, or the merge candidate with the same motion information as the fine merge candidate can be reset as the fine merge candidate. For example, if the motion information of the fine merge candidate mergeCand[6] derived from the reference merge candidate mergeCand[0] is the same as the motion information of the merge candidate mergeCand[2], the motion vector of the fine merge candidate mergeCand[6] can be changed to the value obtained by adding or subtracting the offset vector from the motion vector of the merge candidate[2]. For example, the motion vector of mergeCand[6] can be changed from (mergeCand[0]_mxLx[0]+ M, mergeCand[0]_mvLx[1]) to (mergeCand[2]_mxLx[0]+ M, mergeCand[2]_mvLx[1]). Here, mergeCand[X]_mvLx represents the motion vector of the merge candidate with index X.

[0417] As another example, a refined merge offset list that includes at least one merge offset candidate can be used to determine the offset vector. The refined merge offset list can be used to determine the offset vector when the merge candidate specified by the index information of the current block is a reference merge candidate. Alternatively, the motion vector of the current block can be derived by adding or subtracting the offset vector from the motion vector of the merge candidate. The reference merge candidate can be a merge candidate with a predefined index value in the merge candidate list. For example, the merge candidate with the smallest index value (i.e., the merge candidate with index value 0) or the merge candidate with the largest index value in the merge candidate list can be set as the reference merge candidate. Alternatively, the inter-frame merge candidate with the smallest index value or the inter-frame merge candidate with the largest index value in the inter-frame motion information list can be set as the reference merge candidate.

[0418] Figure 33 This is a diagram illustrating the configuration for merging the refined offset list.

[0419] exist Figure 33 In this context, we assume the reference merge candidate is the merge candidate with index 6.

[0420] If the index of the merge candidate specified by merge_idx, which indicates any of the merge candidates, is not 6, then the motion vector of the merge candidate can be set as the motion vector of the current block.

[0421] On the other hand, when the index of the merge candidate specified by the index information merge_idx is 6, the offset vector can be derived using the merge refinement offset list. The index information MrgOffset_idx, specifying any one of the merge offset candidates included in the merge refinement offset list, can be sent via a signal in the bitstream.

[0422] When an offset vector is specified, the motion vector of the current block can be derived by adding or subtracting the offset vector from the motion vector of the reference merge candidate.

[0423] The merge refined offset list can include at least one merge offset candidate. For example, the number of merge offset candidates included in the merge refined offset list can be 4, 8, or 16.

[0424] Figure 34 and Figure 35 This is a graph showing the offset vectors specified by the merge offset candidates.

[0425] Figure 34 The illustration shows an example where the number of merge offset candidates is 8, and Figure 35 The illustration shows an example where the number of merge offset candidates is 16.

[0426] As in Figure 34 In the example shown in (a), the offset vector represented by the merged offset candidate can be set such that the absolute values ​​of the motion vectors in the horizontal direction and / or the absolute values ​​of the motion vectors in the vertical direction can have fixed values. Alternatively, as Figure 35 In the example shown, the offset vector represented by the merge offset candidate whose index is less than a threshold can be set to a first value, which is the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction, and the offset vector indicated by other merge offset candidates can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a second value.

[0427] Alternatively, such as Figure 34 As shown in (b), the offset vector represented by the merged offset candidate can be set such that the sum of the absolute values ​​of the motion vectors in the horizontal direction and the absolute values ​​of the motion vectors in the vertical direction can have a fixed value.

[0428] Multiple reference merge candidates can be set. For example, the two merge candidates with the smallest index among those included in the merge candidate list can be set as reference merge candidates. Therefore, when the index of the merge candidate specified by the index information merge_idx is 0 or 1, the offset vector can be derived using the merge refinement offset list. Alternatively, the merge candidate with the smallest index among those included in the merge candidate list and the merge candidate with the largest index among those included in the inter-frame merge candidate list can be set as reference merge candidates.

[0429] In advanced motion vector prediction mode, the motion vector of the current block can be derived by adding the motion difference vector to the motion prediction vector. The motion prediction vector of the current block can be determined based on a list of motion vector prediction candidates that includes at least one motion prediction vector candidate. For example, any one of the motion prediction vector candidates can be set as the motion prediction vector of the current block.

[0430] Motion vector prediction candidates can be derived based on at least one of the spatial neighboring blocks or the temporal neighboring blocks of the current block.

[0431] Figure 36 This is a diagram showing the candidate blocks used to derive motion vector prediction candidates.

[0432] Spatial adjacent blocks can include the upper adjacent block above the current block and the left adjacent block to the left of the current block. The upper adjacent block can include one of the following blocks: block B0 containing the sample at position (xCb+CbW, yCb-1), block B1 containing the sample at position (xCb+CbW-1, yCb-1), block B2 containing the sample at position (xCb-1, yCb-1), or block B3 containing the sample at position (xCb, yCb-1). Here, (xCb, yCb) represents the position of the upper left sample of the current block, and CbW represents the width of the current block. The left adjacent block can include one of the following blocks: block A0 containing the sample at position (xCb-1, yCb+CbH), block A1 containing the sample at position (xCb-1, yCb+CbH-1), or block A2 containing the sample at position (xCb-1, yCb). Where CbH represents the height of the current block.

[0433] The temporally adjacent blocks may include at least one of blocks C0 and C1, where block C0 includes a sample located at the center of a block that has the same position and size as the current block in the co-located block, and block C1 includes a sample adjacent to the lower right corner of that block.

[0434] The maximum number of motion vector prediction candidates that can be included in the motion vector prediction candidate list is 2. The order in which the motion vector prediction candidates are derived is as follows.

[0435] 1. When at least one of the left adjacent block A0 or the left adjacent block A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0436] 2. When at least one of the upper neighboring block B0, upper neighboring block B1, or upper neighboring block B2 is available, the motion vector of the available block is set as a motion vector prediction candidate.

[0437] 3. When a temporally adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0438] 4. Set the zero motion vector as a candidate for motion vector prediction.

[0439] Alternatively, when the number of motion vector prediction candidates derived in the order of 1 to 3 is less than 2, the motion vectors included in the inter-frame motion information list can be set as motion vector prediction candidates. When the inter-frame motion information list is available, motion vector prediction candidates can be derived in the following order.

[0440] 1. When at least one of the left adjacent block A0 or the left adjacent block A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0441] 2. When at least one of the upper neighboring block B0, upper neighboring block B1, or upper neighboring block B2 is available, the motion vector of the available block is set as a motion vector prediction candidate.

[0442] 3. When a temporally adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0443] 4. Set the motion vectors included in the inter-frame motion information list as motion vector prediction candidates.

[0444] 5. Set the zero motion vector as a candidate for motion vector prediction.

[0445] Motion vector prediction candidates that have motion vectors derived by adding or subtracting an offset vector from the motion vectors of reference motion vector prediction candidates can be added to the motion vector prediction candidate list. Motion vector prediction candidates that have motion vectors derived by adding or subtracting an offset vector from the motion vectors of reference motion vector prediction candidates can be referred to as refined motion vector prediction candidates.

[0446] Figure 37 This is a graph showing motion vector candidates that can be set as fine motion vector prediction candidates.

[0447] When the motion vector of the reference motion vector prediction candidate is (MvpLX[0], MvpLX[1]), the motion vector of the fine motion vector prediction candidate can be derived by adding or subtracting an offset from at least one of the x-component or y-component of the motion vector of the reference motion vector prediction candidate. For example, the motion vector of the fine motion vector prediction candidate can be set to (MvpLX[0] + M, MvpLX[1]), (MvpLX[0] - M, MvpLX[1]), (MvpLX[0], MvpLX[1] + M) or (MvpLX[0], MvpLX[1] - M). M represents the magnitude of the offset vector.

[0448] The size M of the offset vector can be predefined in the encoder and decoder. For example, the size M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0449] Alternatively, information for determining the offset vector can be transmitted via signaling in the bitstream. This information can be transmitted at the sequence level, image level, strip level, or block level. For example, the offset vector can be determined using at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector, as described above.

[0450] A reference motion vector prediction candidate can be a motion vector prediction candidate with a predetermined index value in the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, a motion vector prediction candidate with an index value of 0 or a motion vector prediction candidate with an index value of 1 can be set as a reference motion vector prediction candidate.

[0451] As another example, the offset vector can be determined using a merged, refined offset list that includes at least one predicted vector offset candidate. The offset vector can be determined using the predicted vector refined offset list when the motion vector prediction candidate specified by the index information of the current block is a reference motion vector prediction candidate. Alternatively, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector from the motion vector prediction candidate. The reference motion vector prediction candidate can be a motion vector prediction candidate with a predetermined index value in the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, the motion vector prediction candidate with the smallest index value or the motion vector prediction candidate with the largest index value can be set as the reference motion vector prediction candidate.

[0452] When using a refined list of predicted vector offsets to compute the offset vector, the maximum number of predicted vector candidates that the predicted vector candidate list can include can be set to a value greater than 2.

[0453] Figure 38 This is a diagram showing the configuration of the prediction vector refinement offset list.

[0454] exist Figure 38 In this context, we assume that the reference prediction vector candidate is the prediction vector candidate with index 2.

[0455] When the index of a prediction vector candidate specified by the index information AMVPcand_idx indicating any of the prediction vector candidates is not 2, the motion vector of the prediction vector candidate can be set as the motion prediction vector of the current block.

[0456] On the other hand, when the index of the prediction vector candidate specified by the index information AMVPcand_idx is 2, the offset vector can be derived using the prediction vector refinement offset list. The index information AMVPOffset_idx, specifying any one of the prediction vector offset candidates included in the prediction vector refinement offset list, can be sent via a signal in the bitstream.

[0457] When an offset vector is specified, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector from the motion vector of the reference prediction vector candidate.

[0458] Even when encoding blocks based on an affine motion model, motion vector refinement techniques can be used. For example, when applying an affine advanced motion vector prediction mode, the affine seed vector of the encoding block can be derived by adding the affine seed difference vector to the affine seed prediction vector. Here, the affine seed prediction vector can be derived based on the affine seed vectors of spatially or temporally adjacent blocks of the encoding block. The affine seed difference vector can be determined based on information transmitted from the bitstream via signals. In this case, the same affine seed difference vector can be applied to all control points. Alternatively, information for determining the affine seed vector can be transmitted via signals for each control point.

[0459] When deriving the affine vectors of sub-blocks based on the affine seed vector of a coded block, the affine vectors can be set as the initial motion vectors, and then the offset vectors can be derived. The motion vectors of each sub-block can be derived by adding or subtracting the offset vectors from the initial motion vectors.

[0460] Instead of sending signals to determine the offset vector, the decoder can derive the offset vector. Specifically, the offset vector can be derived using the average of the horizontal and vertical gradients of the predicted samples included in the sub-block.

[0461] Intra-frame prediction uses encoded / decoded reconstructed samples from the surrounding blocks to predict the current block. In this case, intra-frame prediction of the current block can use reconstructed samples before the loop filter is applied.

[0462] Intra-prediction techniques include matrix-based intra-prediction and general intra-prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra-prediction technique for the current block can be signaled via the bitstream. This information may be a 1-bit flag. Alternatively, the intra-prediction technique for the current block can be determined based on at least one of the intra-prediction techniques of the current block's position, size, shape, or neighboring blocks. For example, when the current block crosses an image boundary, the current block is set not to apply matrix-based intra-prediction.

[0463] Matrix-based intra-frame prediction is a method that obtains the predicted block for the current block by performing matrix multiplication between the matrices stored in the encoder and decoder and the reconstructed samples surrounding the current block. Information specifying any one of a plurality of stored matrices can be transmitted via a signal in the bitstream. The decoder can then determine the matrix used for intra-frame prediction of the current block based on this information and the size of the current block.

[0464] Intra-frame prediction is a method that uses either non-angular intra-frame prediction mode or angular intra-frame prediction mode to obtain the prediction block associated with the current block.

[0465] The residual image can be derived by subtracting the predicted image from the original image. In this case, when the residual image is transformed into the frequency domain, even if high-frequency components are removed, the subjective image quality of the video is not significantly degraded. Therefore, reducing the value of high-frequency components or setting the value of high-frequency components to 0 can improve compression efficiency without causing significant visual distortion. Reflecting these characteristics, the current block can be transformed to decompose the residual image into 2D frequency components. This transformation can be performed using transformation techniques such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST).

[0466] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the DCT- or DST-based transformation can be defined as the primary transformation, and the process of transforming the block again using the primary transformation can be called the secondary transformation.

[0467] The main transform can be performed using any of a number of transform kernel candidates. For example, the main transform can be performed using any of DCT2, DCT8, or DCT7.

[0468] Different transform kernels can be used for the horizontal and vertical directions. Information representing combinations of horizontal and vertical transform kernels can also be transmitted via a bitstream signal.

[0469] The execution units for the primary and secondary transformations will differ. For example, a primary transformation can be performed on an 8×8 block, and a secondary transformation can be performed on 4×4 sub-blocks within the transformed 8×8 block. In this case, the transformation coefficients of the remaining regions where the secondary transformation is not performed can also be set to 0.

[0470] Alternatively, a primary transformation can be performed on a 4×4 block, and a secondary transformation can be performed on an 8×8 region of the 4×4 block that includes the transformation.

[0471] Information indicating whether a secondary transformation is to be performed can be sent via signals through the bitstream.

[0472] The decoder can perform the inverse of the second inverse transform (second inverse transform), and can perform the inverse of the main transform (first inverse transform) on its result. The residual signal of the current block can be obtained as the result of the second inverse transform and the first inverse transform.

[0473] Quantization is used to reduce the energy of the block, and the quantization process involves dividing the transformation coefficients by a specific constant. This constant can be derived from quantization parameters, which can be defined as values ​​between 1 and 63.

[0474] If transform and quantization are performed in the encoder, the decoder can obtain the residual block through inverse quantization and inverse transform. The decoder then adds the predicted block and the residual block together to obtain the reconstructed block of the current block.

[0475] If a reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss during quantization and encoding. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

[0476] Embodiments described with a focus on the decoding or encoding process are also included within the scope of this invention. Variations of multiple embodiments described in a predetermined order, in a different order than those described, are also included within the scope of this invention.

[0477] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention, and they can be performed simultaneously or in a different order as needed. Furthermore, in the above embodiments, the structural elements constituting the block diagrams (e.g., units, modules, etc.) can also be implemented as hardware devices or software, and multiple structural elements can be combined to implement a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can individually or in combination include program instructions, data files, data structures, etc. Examples of computer-readable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy optical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. The hardware device can be configured to operate as one or more software modules to perform the processing according to the invention, and vice versa.

[0478] [Industrial Applicability] This invention can be applied to electronic devices that encode / decode video.

Claims

1. A method for receiving a bitstream, characterized in that, include: Receive the bitstream, and Perform a video decoding method to decode the bitstream to generate a video or image; The video decoding method includes the following steps: Determine whether to apply the merged motion difference coding method to the current block; Generate a list of candidate blocks to be merged; The merge candidates for the current block are determined based on the merge candidate list; and The motion vector of the current block is derived based on the merged candidate, wherein, When the merging motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding the offset vector to the motion vector derived from the merging candidate. When the maximum number of merging candidates included in the merging candidate list is more than 1, the merging candidate of the current block is selected based on the index information of the merging candidate list that is decoded from the bitstream, and when the maximum number is 1, the merging candidate is determined without decoding the index information. The magnitude of the offset vector is determined based on first index information, which indicates one of a plurality of motion offset size candidates; wherein at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the value of a flag indicating the range of the motion offset size candidates, or at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the motion vector precision of the current block.

2. The method for receiving a code stream according to claim 1, wherein, The flag is transmitted as a signal at the image level.

3. The method for receiving a code stream according to claim 1, wherein, The magnitude of the offset vector is obtained by shifting the value represented by the motion offset magnitude candidate specified by the first index information.

4. The method for receiving a code stream according to claim 1, wherein, The direction of the offset vector is determined based on the second index information, which is used to indicate one of a plurality of vector direction candidates.

5. A method for transmitting a code stream, characterized in that, include: Perform a video encoding method to generate a bitstream; as well as Transmit the bitstream; The video encoding method includes the following steps: Determine whether to apply the merged motion difference coding method to the current block; Generate a list of candidate blocks to be merged; The merge candidates for the current block are determined based on the merge candidate list; and The motion vector of the current block is derived based on the merged candidate, wherein, When the merging motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding the offset vector to the motion vector derived based on the merging candidate. When the maximum number of merging candidates included in the merging candidate list is more than 1, the index information indicating the merging candidate of the current block in the merging candidate list is encoded, and when the maximum number is 1, the encoding of the index information is omitted. The first index information is encoded to specify a motion offset size candidate among a plurality of motion offset size candidates that indicates the size of the offset vector; wherein, a flag indicating the range of the motion offset size candidates is encoded, wherein at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the value of the flag, or at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the motion vector precision of the current block.

6. The method for transmitting a code stream according to claim 5, wherein, The flag is encoded at the image level.

7. The method for transmitting a code stream according to claim 5, wherein, The motion offset size candidate has a value derived by shifting the size of the offset vector.

8. The method for transmitting a code stream according to claim 5, wherein, The second index information is encoded, which is used to specify a vector direction candidate that indicates the direction of the offset vector among a plurality of vector direction candidates.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an executable program and a bitstream, which, when executed by one or more processors, implements the steps of the method for receiving a bitstream as described in any one of claims 1 to 4 to receive and decode the bitstream.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an executable program and a bitstream, which, when executed by one or more processors, implements the steps of the method for transmitting the bitstream as described in any one of claims 5 to 8 to transmit the bitstream.

Images