Video signal encoding and decoding method, and apparatus therefor

Abstract

An image decoding method, according to the present invention, comprises the steps of: determining whether a merge offset vector encoding method is applicable to a current block; generating a merge candidate list for the current block; specifying a merge candidate for the current block on the basis of the merge candidate list; and deriving a motion vector for the current block on the basis of the merge candidate.

Description

Video signal encoding / decoding method and apparatus for the same

[0001] The present invention relates to a video signal encoding / decoding method and an apparatus for the same.

[0002] With the trend of display panels becoming increasingly larger, there is a growing demand for video services with higher image quality. The biggest challenge facing high-definition video services is the significant increase in data volume; to address this, active research is underway to improve video compression ratios. A representative example is the formation of the Joint Collaborative Team on Video Coding (JCT-VC) in 2009 by the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T). JCT-VC proposed High Efficiency Video Coding (HEVC), a video compression standard with approximately twice the compression performance of H.264 / AVC, which was approved on January 25, 2013. However, with the rapid advancement of high-definition video services, the performance of HEVC is gradually revealing its limitations.

[0003] The present invention aims to provide a method for refining motion vectors derived from merge candidates based on offset vectors in encoding / decoding a video signal, and an apparatus for performing said method.

[0004] The present invention aims to provide a signaling method for an offset vector and an apparatus for performing said method in encoding / decoding a video signal.

[0005] The technical problems to be solved by the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below.

[0006] A video signal decoding / encoding method according to the present invention comprises the steps of determining whether a merge offset vector encoding method is applied to a current block, generating a merge candidate list for the current block, specifying a merge candidate for the current block based on the merge candidate list, and deriving a motion vector for the current block based on the merge candidate. In this case, when the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate, and when the maximum number of merge candidates that the merge candidate list can include is multiple, the merge candidate of the current block is selected based on index information decoded from a bitstream pointing to any one of the merge candidates, and when the maximum number is one, the merge candidate can be determined without decoding the index information.

[0007] In the video signal decoding / encoding method according to the present invention, the size of the offset vector may be determined based on first index information specifying any one of the vector size candidates.

[0008] In the video signal decoding / encoding method according to the present invention, at least one of the maximum or minimum values ​​of the vector size candidates may be set differently depending on the value of a flag indicating the range of the vector size candidates.

[0009] In the video signal decoding / encoding method according to the present invention, the flag can be signaled through a picture parameter set.

[0010] In the video signal decoding / encoding method according to the present invention, at least one of the maximum or minimum values ​​of the vector magnitude candidates may be set differently depending on the motion vector precision for the current block.

[0011] In the video signal decoding / encoding method according to the present invention, the size of the offset vector can be obtained by applying a bit shifting operation to the value pointed to by the vector size candidate specified by the first index information.

[0012] In the video signal decoding / encoding method according to the present invention, the direction of the offset vector may be determined based on second index information that specifies one of the vector direction candidates.

[0013] The features briefly summarized above regarding the present invention are merely exemplary aspects of the detailed description of the present invention that follows and do not limit the scope of the present invention.

[0014] According to the present invention, the inter prediction efficiency can be improved by refining the motion vector of a merge candidate based on an offset vector.

[0015] According to the present invention, the inter prediction efficiency can be improved by adaptively determining the magnitude and direction of the offset vector.

[0016] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below.

[0017] FIG. 1 is a block diagram of an image encoder (encoder) according to one embodiment of the present invention.

[0018] FIG. 2 is a block diagram of an image decoder (decoder) according to one embodiment of the present invention.

[0019] FIG. 3 is a drawing illustrating a basic coding tree unit according to one embodiment of the present invention.

[0020] Figure 4 is a diagram showing various division forms of a coding block.

[0021] Figure 5 is a diagram illustrating the division pattern of a coding tree unit.

[0022] Figure 6 is a diagram showing the form of a basic data unit.

[0023] FIGS. 7 and FIGS. 8 are diagrams illustrating examples in which a coding block is divided into multiple sub-blocks.

[0024] FIG. 9 is a flowchart of an inter prediction method according to one embodiment of the present invention.

[0025] Figure 10 is a diagram illustrating the non-linear movement of an object.

[0026] FIG. 11 is a flowchart of an inter prediction method based on affine motion according to an embodiment of the present invention.

[0027] Figure 12 is a diagram illustrating affine seed vectors for each affine motion model.

[0028] FIG. 13 is a diagram illustrating the affine vectors of sub-blocks under a 4-parameter motion model.

[0029] Figure 14 is a flowchart of the process of inducing movement information of the current block under merge mode.

[0030] FIG. 15 is a diagram illustrating candidate blocks used to derive merge candidates.

[0031] Figure 16 is a diagram showing the locations of reference samples.

[0032] Figure 17 is a diagram illustrating candidate blocks used to derive merge candidates.

[0033] Figure 18 is a diagram illustrating an example in which the position of a reference sample is changed.

[0034] Figure 19 is a diagram illustrating an example in which the position of a reference sample is changed.

[0035] FIG. 20 is a diagram illustrating the update pattern of an inter-region motion information table.

[0036] Figure 21 is a diagram showing the update pattern of the inter-region merge candidate table.

[0037] Figure 22 is a diagram showing an example in which the index of a previously stored inter-region merge candidate is updated.

[0038] Figure 23 is a diagram showing the location of the representative sub-block.

[0039] Figure 24 shows an example of how an inter-region motion information table is generated for each inter-prediction mode.

[0040] Figure 25 is a diagram showing an example in which inter-region merge candidates included in a long-term motion information table are added to a merge candidate list.

[0041] Figure 26 illustrates an example where redundancy checks are performed on only some of the merge candidates.

[0042] Figure 27 is a diagram showing an example where redundancy checks with specific merge candidates are omitted.

[0043] Figure 28 is a diagram showing an offset vector according to the values ​​of distance_idx, which represents the magnitude of the offset vector, and direction_idx, which represents the direction of the offset vector.

[0044] Figure 29 is a diagram showing an offset vector according to the values ​​of distance_idx, which represents the magnitude of the offset vector, and direction_idx, which represents the direction of the offset vector.

[0045] Figure 30 is a diagram showing the division pattern of a coding block when a triangular partitioning technique is applied.

[0046] FIG. 31 is a diagram showing an example in which the offset vectors of each sub-unit are set differently.

[0047] Figure 32 is a diagram showing motion vector candidates that a refine merge candidate can take.

[0048] FIG. 33 is a diagram illustrating the configuration of a merge refinement offset list.

[0049] Figures 34 and 35 are diagrams showing offset vectors specified by merge offset candidates.

[0050] Figure 36 is a diagram showing candidate blocks used to derive motion vector prediction candidates.

[0051] Figure 37 is a diagram showing motion vector candidates that can be set as refined motion vector prediction candidates.

[0052] Figure 38 is a diagram illustrating the configuration of a prediction vector refinement offset list.

[0053] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0054] Encoding and decoding of the image are performed in block units. For example, encoding / decoding processes such as transformation, quantization, prediction, in-loop filtering, or restoration may be performed on a coding block, a transform block, or a prediction block.

[0055] Hereinafter, the block subject to encoding / decoding will be referred to as the 'current block'. For example, the current block may represent a coding block, a transformation block, or a prediction block depending on the current encoding / decoding processing stage.

[0056] Furthermore, as used in this specification, the term 'unit' may be understood to represent a basic unit for performing a specific encoding / decoding process, and 'block' may be understood to represent a sample array of a predetermined size. Unless otherwise noted, 'block' and 'unit' may be used interchangeably. For example, in the embodiments described below, a coding block and a coding unit may be understood to have mutually equivalent meanings.

[0057] FIG. 1 is a block diagram of an image encoder (encoder) according to one embodiment of the present invention.

[0058] Referring to FIG. 1, the image encoding device (100) may include a picture splitting unit (110), a prediction unit (120, 125), a conversion unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse conversion unit (145), a filter unit (150), and a memory (155).

[0059] Each component shown in FIG. 1 is depicted independently to represent different characteristic functions of the image encoding device and does not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for the convenience of explanation, but at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform functions, and such integrated and separated embodiments of each component are included within the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0060] Furthermore, some components may not be essential components performing an essential function in the present invention, but merely optional components for enhancing performance. The present invention may be implemented by including only the components essential for realizing the essence of the present invention, excluding components used solely for performance enhancement, and a structure including only the essential components, excluding optional components used solely for performance enhancement, is also included within the scope of the rights of the present invention.

[0061] The picture segmentation unit (110) can divide an input picture into at least one processing unit. At this time, the processing unit may be a Prediction Unit (PU), a Transform Unit (TU), or a Coding Unit (CU). The picture segmentation unit (110) can divide a picture into a combination of multiple coding units, prediction units, and transformation units, and can encode the picture by selecting one combination of coding units, prediction units, and transformation units based on a predetermined criterion (e.g., a cost function).

[0062] For example, a single picture can be divided into multiple coding units. To divide coding units within a picture, a recursive tree structure such as a Quad Tree Structure can be used. A coding unit divided into other coding units, with a single image or the largest coding unit as the root, can have as many child nodes as there are divided coding units. A coding unit that can no longer be divided according to certain limits becomes a leaf node. That is, assuming that only square division is possible for a single coding unit, a single coding unit can be divided into up to four different coding units.

[0063] In the following embodiments of the present invention, the encoding unit may be used to mean a unit that performs encoding, or a unit that performs decoding.

[0064] A prediction unit may be divided into at least one shape, such as a square or rectangle, of the same size within a single encoding unit, or one of the prediction units divided within a single encoding unit may be divided such that any one prediction unit has a different shape and / or size from another prediction unit.

[0065] When generating a prediction unit that performs intra prediction based on a coding unit, if it is not the minimum coding unit, intra prediction can be performed without dividing it into multiple prediction units NxN.

[0066] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction and an intra prediction unit (125) that performs intra prediction. It may determine whether to use inter prediction or perform intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. At this time, the processing unit in which the prediction is performed and the processing unit in which the prediction method and specific details are determined may be different. For example, the prediction method and prediction mode, etc. may be determined in the prediction unit, and the prediction may be performed in the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoder. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0067] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous picture or the subsequent picture of the current picture, and in some cases, may predict a prediction unit based on information of a partially encoded area within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0068] In the reference picture interpolation unit, reference picture information is received from memory (155), and pixel information of integer pixels or less can be generated from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 4 pixel units. In the case of chrominance signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 8 pixel units.

[0069] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to calculate motion vectors. Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 or 1 / 4 pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various motion prediction methods such as the Skip method, Merge method, AMVP (Advanced Motion Vector Prediction) method, and Intra Block Copy method can be used.

[0070] The intra prediction unit (125) can generate a prediction unit based on reference pixel information around the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block that has performed inter prediction, and the reference pixel is a pixel that has performed inter prediction, the reference pixel included in the block that has performed inter prediction can be replaced with the reference pixel information of the surrounding intra prediction block. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced with at least one of the available reference pixels.

[0071] In intra-prediction, the prediction mode may include a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing the prediction. The mode for predicting luminance information and the mode for predicting chrominance information may be different, and the intra-prediction mode information used to predict luminance information or the predicted luminance signal information may be utilized to predict chrominance information.

[0072] When performing intra prediction, if the size of the prediction unit and the size of the transform unit are the same, intra prediction for the prediction unit can be performed based on the pixels to the left, the pixels at the top left, and the pixels at the top of the prediction unit. However, when performing intra prediction, if the size of the prediction unit and the size of the transform unit are different, intra prediction can be performed using reference pixels based on the transform unit. Additionally, intra prediction using NxN partitioning can be used only for the minimum encoding unit.

[0073] The intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel according to the prediction mode. The type of AIS filter applied to the reference pixel may vary. To perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of prediction units existing in the vicinity of the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted from neighboring prediction units, if the intra prediction modes of the current prediction unit and the neighboring prediction units are the same, information indicating that the prediction modes of the current prediction unit and the neighboring prediction units are the same can be transmitted using predetermined flag information; if the prediction modes of the current prediction unit and the neighboring prediction units are different, entropy coding can be performed to encode the prediction mode information of the current block.

[0074] Additionally, a residual block can be generated that includes residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit. The generated residual block can be input to the conversion unit (130).

[0075] In the transformation unit (130), the residual block containing residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST). Here, the DCT transformation core includes at least one of DCT2 or DCT8, and the DST transformation core includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on the intra-prediction mode information of the prediction unit used to generate the residual block. Transformation for the residual block may also be skipped. A flag indicating whether to skip transformation for the residual block may be encoded. Transformation skipping may be allowed for residual blocks, luminance components, or chroma components under a 4:4:4 format, where the size is below a threshold value.

[0076] The quantization unit (135) can quantize the values ​​converted into the frequency domain in the conversion unit (130). The quantization coefficient may vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) may be provided to the inverse quantization unit (140) and the reordering unit (160).

[0077] The reordering unit (160) can perform reordering of coefficient values ​​for quantized residual values.

[0078] The reordering unit (160) can convert two-dimensional block-shaped coefficients into one-dimensional vector forms through a coefficient scanning method. For example, the reordering unit (160) can convert the coefficients from DC to high-frequency range coefficients into one-dimensional vector forms by scanning using a Zig-Zag Scan method. Depending on the size of the conversion unit and the intra-prediction mode, a vertical scan that scans two-dimensional block-shaped coefficients in the column direction or a horizontal scan that scans two-dimensional block-shaped coefficients in the row direction may be used instead of the Zig-Zag Scan. That is, depending on the size of the conversion unit and the intra-prediction mode, it can be determined whether to use a Zig-Zag Scan, a vertical scan, or a horizontal scan.

[0079] The entropy encoding unit (165) can perform entropy encoding based on the values ​​calculated by the reordering unit (160). Entropy encoding can use various encoding methods, such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0080] The entropy encoding unit (165) can encode various information from the reordering unit (160) and the prediction unit (120, 125), such as residual value coefficient information of the encoding unit, block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0081] The entropy encoding unit (165) can entropy-encode the coefficient value of the encoding unit input from the rearrangement unit (160).

[0082] In the inverse quantization unit (140) and inverse transformation unit (145), the values ​​quantized in the quantization unit (135) are inverse quantized, and the values ​​transformed in the transformation unit (130) are inverse transformed. The residual value generated in the inverse quantization unit (140) and inverse transformation unit (145) can be combined with the prediction unit predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0083] The filter section (150) may include at least one of a deblocking filter, an offset correction section, and an ALF (Adaptive Loop Filter).

[0084] The deblocking filter can remove block distortion caused by boundaries between blocks in the restored picture. To determine whether to perform deblocking, the decision to apply the deblocking filter to the current block can be made based on the pixels contained in a certain number of columns or rows within the block. When applying the deblocking filter to a block, a Strong Filter or a Weak Filter can be applied depending on the required deblocking filtering strength. Additionally, when applying the deblocking filter, horizontal and vertical filtering can be processed in parallel.

[0085] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for the image that has undergone deblocking. To perform offset correction for a specific picture, a method can be used in which pixels included in the image are divided into a certain number of regions, the region to be offset is determined, and the offset is applied to that region, or a method can be used in which the offset is applied by considering the edge information of each pixel.

[0086] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered restored image and the original image. After dividing the pixels included in the image into predetermined groups, a single filter to be applied to each group can be determined, allowing for differential filtering for each group. Information regarding whether to apply ALF can be transmitted per coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. Additionally, an ALF filter of the same form (fixed form) may be applied regardless of the characteristics of the block to be applied.

[0087] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter-prediction.

[0088] FIG. 2 is a block diagram of an image decoder (decoder) according to one embodiment of the present invention.

[0089] Referring to FIG. 2, the image decoder (200) may include an entropy decoder (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0090] When a video bitstream is input to a video encoder, the input bitstream can be decoded using the reverse procedure of the video encoder.

[0091] The entropy decoding unit (210) can perform entropy decoding in the opposite procedure to that which the entropy encoding unit of the image encoder performed. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied in correspondence with the method performed in the image encoder.

[0092] The entropy decoding unit (210) can decode information related to intra-prediction and inter-prediction performed in the encoder.

[0093] The reordering unit (215) can perform reordering based on the method of reordering the entropy-decoded bitstream in the encoding unit in the entropy decoding unit (210). It can reorder by restoring the coefficients expressed in the form of a one-dimensional vector back into coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed in the encoding unit and scanning in reverse based on the scanning order performed in the encoding unit.

[0094] The inverse quantization unit (220) can perform inverse quantization based on the coefficient values ​​of the rearranged block and the quantization parameters provided by the encoder.

[0095] The inverse transform unit (225) can perform an inverse transform, i.e., an inverse DCT or an inverse DST, on the transformation, i.e., a DCT or a DST, performed by the transform unit on the quantization result performed by the image encoder. Here, the DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Alternatively, if the transformation is skipped in the image encoder, the inverse transform unit (225) may not perform an inverse transform. The inverse transform may be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transformation technique (e.g., DCT or DST) may be selectively performed according to a plurality of information such as a prediction method, the size of the current block, and the prediction direction.

[0096] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0097] As described above, when performing intra prediction identical to the operation in the video encoder, if the size of the prediction unit and the size of the transform unit are the same, intra prediction for the prediction unit is performed based on the pixels to the left, the pixels to the top left, and the pixels to the top of the prediction unit; however, if the size of the prediction unit and the size of the transform unit are different when performing intra prediction, intra prediction can be performed using reference pixels based on the transform unit. Additionally, intra prediction using NxN partitioning only for the minimum encoding unit may also be used.

[0098] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information, such as prediction unit information input from the entropy decoding unit (210), prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, distinguishes the prediction unit from the current encoding unit, and determines whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit (230) may perform inter prediction for the current prediction unit based on information included in at least one picture, either the previous picture or the subsequent picture, of the current picture containing the current prediction unit, using information required for inter prediction of the current prediction unit provided by the video encoder. Alternatively, it may perform inter prediction based on information of a partially restored area within the current picture containing the current prediction unit.

[0099] To perform inter-prediction, based on the encoding unit, it is possible to determine whether the motion prediction method of the prediction unit included in the corresponding encoding unit is Skip Mode, Merge Mode, Motion Vector Prediction Mode (AMVP Mode), or Intra-Block Copy Mode.

[0100] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, it can perform intra prediction based on the intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and can determine whether to apply the filter based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixel of the current block using the prediction mode of the prediction unit and the AIS filter information provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0101] The reference pixel interpolation unit can generate a reference pixel of an integer value or less by interpolating the reference pixel when the prediction mode of the prediction unit is a prediction unit that performs intra-prediction based on the pixel value interpolated from the reference pixel. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is DC mode.

[0102] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0103] Information regarding whether a deblocking filter has been applied to the corresponding block or picture can be received from the video encoder, and if a deblocking filter has been applied, information regarding whether a strong filter or a weak filter has been applied. The deblocking filter of the video decoder receives information related to the deblocking filter provided by the video encoder and can perform deblocking filtering on the corresponding block.

[0104] The offset correction unit can perform offset correction on the restored image based on the type of offset correction and offset value information applied to the image during encoding.

[0105] ALF can be applied to the encoding unit based on information on whether to apply ALF, ALF coefficient information, etc., provided by the encoder. This ALF information can be provided included in a specific parameter set.

[0106] The memory (245) can store the restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to the output unit.

[0107]

[0108] FIG. 3 is a drawing illustrating a basic coding tree unit according to one embodiment of the present invention.

[0109] A coding block of maximum size can be defined as a coding tree block. A picture is divided into multiple coding tree units (CTUs). A coding tree unit is a coding unit of maximum size and may also be referred to as an LCU (Largest Coding Unit). FIG. 3 shows an example in which a picture is divided into multiple coding tree units.

[0110] The size of a coding tree unit can be defined at the picture level or the sequence level. To this end, information indicating the size of the coding tree unit can be signaled through a set of picture parameters or a set of sequence parameters.

[0111] For example, the size of the coding tree unit for the entire picture in the sequence may be set to 128x128. Alternatively, the size of the coding tree unit may be determined at the picture level to be either 128x128 or 256x256. For example, the size of the coding tree unit in the first picture may be set to 128x128, and the size of the coding tree unit in the second picture may be set to 256x256.

[0112] Coding blocks can be generated by dividing a coding tree unit. A coding block represents a basic unit for encoding / decoding processing. For example, prediction or transformation may be performed per coding block, or a prediction encoding mode may be determined per coding block. Here, the prediction encoding mode represents a method for generating a prediction image. For example, the prediction encoding mode may include intra-prediction, inter-prediction, current picture referencing (CPR, or intra block copy (IBC)), or combined prediction. For a coding block, a prediction block for the coding block can be generated using at least one prediction encoding mode among intra-prediction, inter-prediction, current picture referencing, or combined prediction.

[0113] Information indicating the predictive encoding mode of the current block may be signaled through a bitstream. For example, the information may be a 1-bit flag indicating whether the predictive encoding mode is intra mode or inter mode. Only when the predictive encoding mode of the current block is determined to be inter mode may current picture reference or composite prediction be available.

[0114] The current picture reference is intended to set the current picture as the reference picture and to obtain the prediction block of the current block from an area within the current picture that has already been encoded / decoded. Here, the current picture refers to the picture containing the current block. Information indicating whether the current picture reference is applied to the current block may be signaled via a bitstream. For example, the information may be a 1-bit flag. If the flag is true, the prediction encoding mode of the current block is determined to be the current picture reference, and if the flag is false, the prediction mode of the current block may be determined to be inter-prediction.

[0115] Alternatively, the predictive coding mode of the current block can be determined based on the reference picture index. For example, if the reference picture index points to the current picture, the predictive coding mode of the current block can be determined as the current picture reference. If the reference picture index points to a picture other than the current picture, the predictive coding mode of the current block can be determined as inter-prediction. That is, the current picture reference is a prediction method that utilizes information from the region within the current picture where encoding / decoding is complete, while inter-prediction is a prediction method that utilizes information from another picture where encoding / decoding is complete.

[0116] Composite prediction represents a encoding mode that combines two or more of intra prediction, inter prediction, and current picture reference. For example, when composite prediction is applied, a first prediction block may be generated based on any one of intra prediction, inter prediction, or current picture reference, and a second prediction block may be generated based on the other. When the first prediction block and the second prediction block are generated, a final prediction block may be generated through an average operation or a weighted sum operation of the first prediction block and the second prediction block. Information indicating whether composite prediction is applied may be signaled through a bitstream. The information may be a 1-bit flag.

[0117] Figure 4 is a diagram showing various division forms of a coding block.

[0118] A coding block can be divided into multiple coding blocks based on quad tree splitting, binary tree splitting, or triple tree splitting. The divided coding blocks can also be further divided into multiple coding blocks based on quad tree splitting, binary tree splitting, or triple tree splitting.

[0119] Quadtree partitioning represents a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block can be divided into four square partitions (see 'SPLIT_QT' in (a) of Fig. 4).

[0120] Binary tree splitting refers to a splitting technique that divides the current block into two blocks. Splitting the current block into two blocks along the vertical direction (i.e., using a vertical line crossing the current block) can be called vertical binary tree splitting, and splitting the current block into two blocks along the horizontal direction (i.e., using a horizontal line crossing the current block) can be called horizontal binary tree splitting. As a result of binary tree splitting, the current block can be divided into two non-square partitions. Figure 4 (b) 'SPLIT_BT_VER' represents the result of vertical binary tree splitting, and Figure 4 (c) 'SPLIT_BT_HOR' represents the result of horizontal binary tree splitting.

[0121] Triple tree partitioning represents a partitioning technique that divides the current block into three blocks. Dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) can be referred to as vertical triple tree partitioning, and dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines crossing the current block) can be referred to as horizontal triple tree partitioning. As a result of triple tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located in the center of the current block can be twice the width / height of the other partitions. Figure 4 (d) 'SPLIT_TT_VER' represents the result of vertical triple tree partitioning, and Figure 4 (e) 'SPLIT_TT_HOR' represents the result of horizontal triple tree partitioning.

[0122] The number of divisions of a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Accordingly, the maximum partitioning depth of a coding tree unit may vary depending on the sequence or picture.

[0123] Alternatively, the maximum split depth for each of the splitting techniques can be determined individually. For example, the maximum split depth allowed for quadtree splitting may differ from the maximum split depth allowed for binary tree splitting and / or tripletree splitting.

[0124] The encoder can signal information indicating at least one of the partition type or partition depth of the current block through a bitstream. The decoder can determine the partition type and partition depth of the coding tree unit based on the information parsed from the bitstream.

[0125] Figure 5 is a diagram illustrating the division pattern of a coding tree unit.

[0126] Dividing coding blocks using partitioning techniques such as quad tree partitioning, binary tree partitioning, and / or triple tree partitioning can be referred to as multi-tree partitioning.

[0127] Coding blocks generated by applying multi-tree splitting to a coding block can be referred to as sub-coding blocks. If the splitting depth of a coding block is k, the splitting depth of the sub-coding blocks is set to k+1.

[0128] Conversely, for coding blocks with a partition depth of k+1, the coding block with a partition depth of k can be referred to as the parent coding block.

[0129] The split type of the current coding block may be determined based on at least one of the split form of the parent coding block or the split type of the neighboring coding block. Here, the neighboring coding block is adjacent to the current coding block and may include at least one of the top neighboring block, the left neighboring block, or the neighboring block adjacent to the top-left corner of the current coding block. Here, the split type may include at least one of whether to split into a quad tree, whether to split into a binary tree, the direction of the binary tree split, whether to split into a triple tree, or the direction of the triple tree split.

[0130] To determine the splitting form of a coding block, information indicating whether the coding block is split can be signaled through a bitstream. The information is a 1-bit flag 'split_cu_flag', and if the flag is true, it indicates that the coding block is split by a head tree splitting technique.

[0131] If split_cu_flag is true, information indicating whether the coding block is split into a quad tree can be signaled through a bitstream. The information is a 1-bit flag split_qt_flag, and if the flag is true, the coding block can be split into four blocks.

[0132] For example, in the example illustrated in FIG. 5, it is shown that as the coding tree unit is divided into a quad tree, four coding blocks with a division depth of 1 are generated. Additionally, it is shown that quad tree division is applied again to the first and fourth coding blocks among the four coding blocks generated as a result of the quad tree division. As a result, four coding blocks with a division depth of 2 can be generated.

[0133] In addition, by applying quad tree partitioning again to a coding block with a partition depth of 2, a coding block with a partition depth of 3 can be generated.

[0134] If quad tree splitting is not applied to the coding block, it may be determined whether to perform binary tree splitting or triple tree splitting on the coding block by considering at least one of the size of the coding block, whether the coding block is located at a picture boundary, the maximum splitting depth, or the splitting type of neighboring blocks. If it is determined that binary tree splitting or triple tree splitting is performed on the coding block, information indicating the splitting direction may be signaled via a bitstream. The information may be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it may be determined whether the splitting direction is vertical or horizontal. Additionally, information indicating whether binary tree splitting or triple tree splitting is applied to the coding block may be signaled via a bitstream. The information may be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it may be determined whether binary tree splitting or triple tree splitting is applied to the coding block.

[0135] For example, in the example illustrated in FIG. 5, a vertical binary tree split is applied to a coding block with a split depth of 1, and among the coding blocks generated as a result of the split, a vertical triple tree split is applied to the left coding block and a vertical binary tree split is applied to the right coding block.

[0136] When implementing a device for encoding or decoding images, a problem arises where processing is difficult in areas larger than a threshold value due to hardware performance limitations. For example, if hardware performance limits the simultaneous processing of up to 4096 samples, for areas with more than 4096 samples, problems may arise such as the need to process data by redundantly accessing 64x64 data units and the inability to process data simultaneously. As such, the basic unit for data processing can be defined as a pipeline-based data basic unit (Virtual Processing Data Unit, VPDU, hereinafter referred to as the data basic unit).

[0137] Basic data units can be classified into square, non-square, or non-rectangular types.

[0138] Figure 6 is a diagram showing the form of a basic data unit.

[0139] Data base units may include a number of samples equal to or less than the maximum number of samples that can be processed simultaneously. For example, as shown in the example illustrated in FIG. 6 (a), square blocks of size 64x64 may be set as data base units. Alternatively, non-square blocks may be set as data base units. For example, as shown in the example illustrated in FIG. 6 (b) or FIG. 6 (c), blocks of size 32x128 or 64x32 may be set as data base units.

[0140] Although not illustrated, triangular, L-shaped, or polygonal data basic units may be defined.

[0141] Information for determining a data base unit may be signaled via a bitstream. The information may be for determining at least one of the size or shape of the data base unit. Based on the information, it may be determined whether a non-square data base unit or a non-rectangular data base unit is allowed.

[0142] Alternatively, at least one of the size or shape of the data base unit in the encoder and decoder may be predefined.

[0143] The permissibility of the coding block division form may be determined by considering the size of the data base unit. For example, if the coding block generated as a result of dividing the coding block is larger than the data base unit, the division may not be permitted. Alternatively, if the non-square coding block generated as a result of dividing the coding block is larger than the data base unit, the division may not be permitted. For example, if the width or height of the coding block is greater than a threshold value, or if the number of samples included in the coding block is greater than a threshold value, binary tree or triple tree division may not be permitted. Accordingly, encoding of information related to binary tree or triple tree division may be omitted.

[0144] Alternatively, a coding block larger than a data base unit may be configured to be split. Alternatively, a coding block larger than a data base unit may be configured to be split into binary tree splits or triple tree splits. Accordingly, for a coding block larger than a data base unit, even if the flag split_flag indicating whether the coding block is split is not encoded, the value of the flag may be derived to 1.

[0145] As another example, a coding block larger than a data base unit can be divided into multiple sub-blocks. Here, the sub-blocks can be set as prediction units, which are base units for prediction, or transformation units, which are base units for transformation and / or quantization. In this case, dividing the coding block into multiple prediction units is defined as VPDU prediction unit partitioning, and dividing the coding block into multiple transformation units can be defined as VPDU transformation unit partitioning.

[0146] At least one of VPDU prediction unit partitioning or VPDU conversion unit partitioning may be applied to the coding block. The partitioning form of the coding block according to the application of VPDU prediction unit partitioning may be set to be the same as the partitioning form of the coding block according to the application of VPDU conversion unit partitioning.

[0147] When only VPDU prediction unit partitioning is applied to a coding block, prediction is performed on a sub-block basis, but transformation and / or quantization can be performed on the coding block. In this case, a prediction mode, such as a prediction encoding mode, an intra prediction mode, or an inter prediction mode, can be determined on the coding block.

[0148] When only VPDU conversion unit partitioning is applied to the coding block, prediction is performed on the sub-block, but conversion and / or quantization can be performed on a sub-block basis.

[0149] FIGS. 7 and FIGS. 8 are diagrams illustrating examples in which a coding block is divided into multiple sub-blocks.

[0150] Figure 7 is a diagram showing the partitioning pattern when only square data basic units are allowed, and Figure 8 is a diagram showing the partitioning pattern when square data basic units and non-square data basic units are allowed.

[0151] Assuming that only square data base units are allowed, in FIG. 7 (a) and (b), CU0 and CU3 are defined as two different VPDUs, and CU1 is defined as four different VPDUs. Accordingly, CU0 and CU3 can be divided into two sub-blocks, and CU1 can be divided into four sub-blocks.

[0152] Assuming that square data base units and non-square data base units are allowed, in FIG. 8 (a) and (b), CU0 and CU3 can be defined as one VPDU, whereas CU1 can be defined using two different VPDUs. Accordingly, CU0 and CU3 are not divided into sub-blocks, whereas CU1 can be divided into two sub-blocks.

[0153] At this time, CU1 can be divided into square sub-blocks or non-square sub-blocks. For example, CU1 can be divided into two square sub-blocks based on a horizontal line dividing CU1 vertically. Or, CU1 can be divided into two non-square sub-blocks based on a vertical line dividing CU1 horizontally.

[0154] If there are multiple partition type candidates that can be applied to a coding block, information indicating one of the multiple partition type candidates can be signaled via a bitstream. For example, the information may indicate whether the coding block is partitioned into square sub-blocks or whether the coding block is partitioned into non-square sub-blocks.

[0155] Alternatively, dividing a coding block into square sub-blocks can be configured to have a higher priority than dividing a coding block into non-square sub-blocks. For example, dividing a coding block into non-square sub-blocks may be permitted when the coding block cannot be divided into square sub-blocks.

[0156] Alternatively, the partitioning form of the coding block can be determined based on the partitioning form of the parent node, the coding block. For example, if the parent node, the coding block, is partitioned based on a triple tree, the coding block can be configured to be partitioned into square sub-blocks. On the other hand, if the parent node, the coding block, is partitioned based on a binary tree or a triple tree, the coding block can be configured to be partitioned into non-square sub-blocks.

[0157]

[0158] Inter-prediction is a prediction coding mode that predicts the current block using information from the previous picture. For example, a block at the same location as the current block within the previous picture (hereinafter referred to as the "collocated block") can be set as the prediction block for the current block. Hereinafter, a prediction block generated based on the block at the same location as the current block will be referred to as the "Collocated Prediction Block."

[0159] On the other hand, if an object present in the previous picture has moved to a different location in the current picture, the current block can be effectively predicted using the object's movement. For example, if the direction and size of the object's movement can be determined by comparing the previous picture and the current picture, a predicted block (or predicted image) of the current block can be generated by considering the object's movement information. Hereinafter, the predicted block generated using movement information may be referred to as a motion prediction block.

[0160] A residual block can be generated by differencing the prediction block from the current block. In this case, if object movement exists, using a motion prediction block instead of a collocated prediction block can reduce the energy of the residual block and, consequently, improve the compression performance of the residual block.

[0161] As described above, generating prediction blocks using motion information can be referred to as motion-compensated prediction. In most inter-predictions, prediction blocks can be generated based on motion-compensated prediction.

[0162] Motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index. The motion vector indicates the direction and magnitude of the object's movement. The reference picture index specifies the reference picture of the current block among the reference pictures included in the reference picture list. The prediction direction indicates any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). Depending on the prediction direction of the current block, at least one of the movement information in the L0 direction or the movement information in the L1 direction may be used. The bidirectional weight index specifies the weight applied to the L0 prediction block and the weight applied to the L1 prediction block.

[0163] FIG. 9 is a flowchart of an inter prediction method according to one embodiment of the present invention.

[0164] Referring to FIG. 9, the inter prediction method includes the step of determining an inter prediction mode of the current block (S901), the step of obtaining movement information of the current block according to the determined inter prediction mode (S902), and the step of performing a movement compensation prediction for the current block based on the obtained movement information (S903).

[0165] Here, the inter prediction mode represents various techniques for determining motion information of the current block, and may include an inter prediction mode using translational motion information and an inter prediction mode using affine motion information. For example, an inter prediction mode using translational motion information may include a merge mode and a motion vector prediction mode, and an inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. Depending on the inter prediction mode, the motion information of the current block may be determined based on information parsed from neighboring blocks or bitstreams adjacent to the current block.

[0166] Below, we will explain in detail the inter prediction method using affine motion information.

[0167] Figure 10 is a diagram illustrating the non-linear movement of an object.

[0168] Non-linear movement of an object within a video may occur. For example, as shown in the example illustrated in FIG. 10, non-linear movement of an object may occur, such as camera zoom-in, zoom-out, rotation, or affine transformation. When non-linear movement of an object occurs, translational motion vectors cannot effectively represent the object's movement. Accordingly, in parts where non-linear movement of the object occurs, affine motion can be used instead of translational motion to improve encoding efficiency.

[0169] FIG. 11 is a flowchart of an inter prediction method based on affine motion according to an embodiment of the present invention.

[0170] Whether an inter prediction technique based on affine motion is applied to the current block can be determined based on information parsed from the bitstream. Specifically, whether an inter prediction technique based on affine motion is applied to the current block can be determined based on at least one of a flag indicating whether an affine merge mode is applied to the current block or a flag indicating whether an affine motion vector prediction mode is applied to the current block.

[0171] When an inter prediction technique based on affine motion is applied to the current block, the affine motion model of the current block can be determined (S1101). The affine motion model can be determined as at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. A 6-parameter affine motion model is one that expresses affine motion using 6 parameters, and a 4-parameter affine motion model is one that expresses affine motion using 4 parameters.

[0172] Equation 1 represents affine motion using 6 parameters. Affine motion represents translational movement over a specific region determined by affine seed vectors.

[0173]

[0174] When expressing affine motion using 6 parameters, complex movements can be expressed, but the number of bits required to encode each parameter increases, which may reduce encoding efficiency. Accordingly, affine motion can also be expressed using 4 parameters. Equation 2 expresses affine motion using 4 parameters.

[0175]

[0176] Information for determining the affine motion model of the current block may be encoded and signaled via a bitstream. For example, the information may be a 1-bit flag 'affine_type_flag'. A value of 0 for the flag may indicate that a 4-parameter affine motion model is applied, and a value of 1 for the flag may indicate that a 6-parameter affine motion model is applied. The flag may be encoded at the slice, tile, or block level (e.g., coding block or coding tree unit). When the flag is signaled at the slice level, the affine motion model determined at the slice level may be applied to all blocks belonging to the slice.

[0177] Alternatively, the affine motion model of the current block can be determined based on the affine inter prediction mode of the current block. For example, if the affine merge mode is applied, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, if the affine motion vector prediction mode is applied, information for determining the affine motion model of the current block can be encoded and signaled through a bitstream. For example, if the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block can be determined based on a 1-bit flag 'affine_type_flag'.

[0178] Next, an affine seed vector of the current block can be derived (S1102). If a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be derived. On the other hand, if a 6-parameter affine motion model is selected, motion vectors at three control points of the current block can be derived. Motion vectors at control points can be referred to as affine seed vectors. A control point may include at least one of the top-left corner, the top-right corner, or the bottom-left corner of the current block.

[0179] Figure 12 is a diagram illustrating affine seed vectors for each affine motion model.

[0180] In a 4-parameter affine motion model, affine seed vectors can be derived for two of the top-left corner, top-right corner, or bottom-left corner. For example, when a 4-parameter affine motion model is selected, as in the example illustrated in FIG. 12 (a), affine vectors can be derived using affine seed vector sv0 for the top-left corner of the current block (e.g., top-left sample (x1, y1)) and affine seed vector sv1 for the top-right corner of the current block (e.g., top-right sample (x1, y1)). It is also possible to use an affine seed vector for the bottom-left corner instead of an affine seed vector for the top-left corner, or to use an affine seed vector for the bottom-left corner instead of an affine seed vector for the top-right corner.

[0181] In a 6-parameter affine motion model, affine seed vectors for the top-left corner, top-right corner, and bottom-left corner can be derived. For example, as shown in the example illustrated in FIG. 12 (b), when a 6-parameter affine motion model is selected, affine vectors can be derived using affine seed vector sv0 for the top-left corner of the current block (e.g., top-left sample (x1, y1)), affine seed vector sv1 for the top-right corner of the current block (e.g., top-right sample (x1, y1)), and affine seed vector sv2 for the top-left corner of the current block (e.g., top-left sample (x2, y2)).

[0182] In the embodiments described below, under a 4-parameter affine motion model, the affine seed vectors of the upper-left control point and the upper-right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments using the first affine seed vector and the second affine seed vector described below, at least one of the first affine seed vector and the second affine seed vector may be replaced with the affine seed vector of the lower-left control point (third affine seed vector) or the affine seed vector of the lower-right control point (fourth affine seed vector).

[0183] Additionally, under the 6-parameter affine motion model, the affine seed vectors of the upper-left control point, the upper-right control point, and the lower-left control point are referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector, respectively. In the embodiments using the first affine seed vector, the second affine seed vector, and the third affine seed vector described below, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector may be replaced with the affine seed vector of the lower-right control point (the fourth affine seed vector).

[0184] Affine vectors can be derived for each sub-block using affine seed vectors (S1103). Here, the affine vector represents a translational motion vector derived based on the affine seed vectors. The affine vector of a sub-block can be referred to as an affine sub-block motion vector or a sub-block motion vector.

[0185] FIG. 13 is a diagram illustrating the affine vectors of sub-blocks under a 4-parameter motion model.

[0186] The affine vector of a sub-block can be derived based on the location of the control point, the location of the sub-block, and the affine seed vector. As an example, Equation 3 shows an example of deriving an affine sub-block vector.

[0187]

[0188] In the above mathematical formula 3, (x, y) represents the position of the sub-block. Here, the position of the sub-block represents the position of the reference sample included in the sub-block. The reference sample may be a sample located at the top-left corner of the sub-block, or a sample where at least one of the x-axis or y-axis coordinates is at the center position. (x0, y0) represents the position of the first control point, and (sv 0x , sv 0y ) represents the first affine seed vector. Also, (x1, y1) represents the location of the second control point, and (sv 1x , sv 1y ) represents the second affine seed vector.

[0189] If the first control point and the second control point correspond to the top-left corner and the top-right corner of the current block, respectively, x1-x0 can be set to a value equal to the width of the current block.

[0190] Subsequently, motion compensation prediction for each sub-block can be performed using the affine vector of each sub-block (S1104). As a result of performing motion compensation prediction, a prediction block for each sub-block can be generated. The prediction blocks of the sub-blocks can be set as the prediction block of the current block.

[0191] Next, we will explain in detail the inter prediction method that utilizes translational motion information.

[0192] The motion information of the current block can be derived from the motion information of other blocks. Here, other blocks may be blocks that were encoded / decoded via inter-prediction prior to the current block. Setting the motion information of the current block to be identical to the motion information of other blocks can be defined as merge mode. Additionally, setting the motion vector of other blocks as the predicted value of the motion vector of the current block can be defined as motion vector prediction mode.

[0193] Figure 14 is a flowchart of the process of inducing movement information of the current block under merge mode.

[0194] A merge candidate for the current block can be derived (S1401). The merge candidate for the current block can be derived from a block that was encoded / decoded by inter-prediction prior to the current block.

[0195] FIG. 15 is a diagram illustrating candidate blocks used to derive merge candidates.

[0196] Candidate blocks may include at least one of neighbor blocks containing samples adjacent to the current block or non-neighbor blocks containing samples not adjacent to the current block. Hereinafter, the samples determining the candidate blocks are defined as reference samples. Additionally, reference samples adjacent to the current block are referred to as neighbor reference samples, and reference samples not adjacent to the current block are referred to as non-neighbor reference samples.

[0197] Neighbor reference samples may be included in the neighbor column of the leftmost column of the current block or in the neighbor row of the top row of the current block. For example, when the coordinates of the top-left sample of the current block are (0, 0), at least one of the following may be used as a candidate block: a block containing a reference sample at position (-1, H-1), a block containing a reference sample at position (W-1, -1), a block containing a reference sample at position (W, -1), a block containing a reference sample at position (-1, H), or a block containing a reference sample at position (-1, -1). Referring to the drawings, neighbor blocks of indices 0 to 4 may be used as candidate blocks.

[0198] A non-neighbor reference sample represents a sample in which at least one of the x-axis distance or y-axis distance to a reference sample adjacent to the current block has a predefined value. For example, at least one of a block containing a reference sample in which the x-axis distance to the left reference sample is a predefined value, a block containing a non-neighbor sample in which the y-axis distance to the top reference sample is a predefined value, or a block containing a non-neighbor sample in which both the x-axis distance and y-axis distance to the top-left reference sample are predefined values ​​may be used as a candidate block. The predefined value may be a natural number such as 4, 8, 12, 16, etc. Referring to the drawing, at least one of the blocks of index 5 to 26 may be used as a candidate block.

[0199] Samples that are not located on the same vertical, horizontal, or diagonal line as neighboring reference samples can also be set as non-neighboring reference samples.

[0200] Figure 16 is a diagram showing the locations of reference samples.

[0201] As shown in the example illustrated in FIG. 16, the x-coordinates of the upper non-neighbor reference samples can be set differently from the x-coordinates of the upper neighbor reference samples. For example, if the position of the upper neighbor reference sample is (W-1, -1), the position of the upper non-neighbor reference sample located N units away from the upper neighbor reference sample along the y-axis can be set to ((W / 2)-1, -1-N), and the position of the upper non-neighbor reference sample located 2N units away from the upper neighbor reference sample along the y-axis can be set to (0, -1-2N). That is, the position of the non-neighbor reference sample can be determined based on the position of the adjacent reference sample and the distance between the adjacent reference sample and the adjacent reference sample.

[0202] Hereinafter, among the candidate blocks, a candidate block containing a neighbor criterion sample will be referred to as a neighbor block, and a block containing a non-neighbor criterion sample will be referred to as a non-neighbor block.

[0203] If the distance between the current block and the candidate block is greater than or equal to a threshold value, the candidate block may be set as unavailable as a merge candidate. The threshold value may be determined based on the size of the coding tree unit. For example, the threshold value may be set to the height of the coding tree unit (ctu_height) or a value obtained by adding or subtracting an offset from the height of the coding tree unit (e.g., ctu_height ± N). The offset N is a value predefined in the encoder and decoder and may be set to 4, 8, 16, 32, or ctu_height.

[0204] If the difference between the y-axis coordinate of the current block and the y-axis coordinate of the sample included in the candidate block is greater than the threshold value, the candidate block may be determined not to be available as a merge candidate.

[0205] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block may be set as unavailable as merge candidates. For example, if a reference sample extends beyond the upper boundary of the coding tree unit to which the current block belongs, the candidate block containing the reference sample may be set as unavailable as a merge candidate.

[0206] If the top boundary of the current block is adjacent to the top boundary of the coding tree unit, multiple candidate blocks are determined not to be available as merge candidates, which may reduce the encoding / decoding efficiency of the current block. To resolve this problem, candidate blocks can be configured such that the number of candidate blocks located to the left of the current block is greater than the number of candidate blocks located to the top of the current block.

[0207] Figure 17 is a diagram illustrating candidate blocks used to derive merge candidates.

[0208] As shown in the example illustrated in FIG. 17, the top blocks belonging to the top N block columns of the current block and the left blocks belonging to the left M block columns of the current block can be set as candidate blocks. In this case, M can be set larger than N so that the number of left candidate blocks is greater than the number of top candidate blocks.

[0209] For example, the difference between the y-axis coordinate of a reference sample within the current block and the y-axis coordinate of the top block that can be used as a candidate block can be set so that it does not exceed N times the height of the current block. Additionally, the difference between the x-axis coordinate of a reference sample within the current block and the x-axis coordinate of the left block that can be used as a candidate block can be set so that it does not exceed M times the width of the current block.

[0210] For example, in the example illustrated in FIG. 17, the blocks belonging to the top two block columns of the current block and the blocks belonging to the left five block columns of the current block are shown as being set as candidate blocks.

[0211] As another example, if a candidate block does not belong to the same coding tree unit as the current block, a merge candidate can be derived by using a block belonging to the same coding tree unit as the current block or a block containing a reference sample adjacent to the boundary of the coding tree unit instead of the candidate block.

[0212] Figure 18 is a diagram illustrating an example in which the position of a reference sample is changed.

[0213] If a reference sample is included in a coding tree unit different from the current block and the reference sample is not adjacent to the boundary of the coding tree unit, a candidate block can be determined using a reference sample adjacent to the boundary of the coding tree unit instead of the reference sample.

[0214] For example, in the example illustrated in FIG. 18 (a) and (b), when the top boundary of the current block and the top boundary of the coding tree unit are in contact, the reference samples at the top of the current block belong to a coding tree unit different from the current block. Among the reference samples belonging to a coding tree unit different from the current block, reference samples that are not adjacent to the top boundary of the coding tree unit can be replaced with samples that are adjacent to the top boundary of the coding tree unit.

[0215] For example, as shown in the example illustrated in FIG. 18 (a), the reference sample at position 6 can be replaced with a sample at position 6' located at the top boundary of the coding tree unit, and as shown in the example illustrated in FIG. 18 (b), the reference sample at position 15 can be replaced with a sample at position 15' located at the top boundary of the coding tree unit. In this case, the y-coordinate of the replacement sample can be changed to an adjacent position of the coding tree unit, and the x-coordinate of the replacement sample can be set to be the same as that of the reference sample. For example, the sample at position 6' can have the same x-coordinate as the sample at position 6, and the sample at position 15' can have the same x-coordinate as the sample at position 15.

[0216] Alternatively, the x-coordinate of the replacement sample may be set by adding or subtracting an offset from the x-coordinate of the reference sample. For example, if the x-coordinates of a neighboring reference sample and a non-neighboring reference sample located at the top of the current block are identical, the x-coordinate of the replacement sample may be set by adding or subtracting an offset from the x-coordinate of the reference sample. This is to prevent the replacement sample replacing the non-neighboring reference sample from being located at the same position as another non-neighboring reference sample or a neighboring reference sample.

[0217] Figure 19 is a diagram illustrating an example in which the position of a reference sample is changed.

[0218] When replacing a reference sample that is included in a coding tree unit different from the current block and is not adjacent to the boundary of the coding tree unit with a sample located at the boundary of the coding tree unit, the value obtained by adding or subtracting an offset from the x-coordinate of the reference sample can be set as the x-coordinate of the replacement sample.

[0219] For example, in the example illustrated in FIG. 19, the reference sample at position 6 and the reference sample at position 15 can be replaced with the sample at position 6' and the sample at position 15', respectively, whose y-coordinates are the same as the row adjacent to the top boundary of the coding tree unit. In this case, the x-coordinate of the sample at position 6' can be set to a value obtained by differentiating W / 2 from the x-coordinate of the reference sample at position 6, and the x-coordinate of the sample at position 15' can be set to a value obtained by differentiating W-1 from the x-coordinate of the reference sample at position 15.

[0220] Unlike the example shown in FIGS. 18 and 19, the y-coordinate of the row located at the top of the top row of the current block, or the y-coordinate of the top boundary of the coding tree unit, may be set as the y-coordinate of the alternative sample.

[0221] Although not shown, a sample that replaces the reference sample can be determined based on the left boundary of the coding tree unit. For example, if the reference sample is not included in the same coding tree unit as the current block and is not adjacent to the left boundary of the coding tree unit, the reference sample can be replaced with a sample adjacent to the left boundary of the coding tree unit. In this case, the replacement sample may have the same y-coordinate as the reference sample, or may have a y-coordinate obtained by adding or subtracting an offset from the y-coordinate of the reference sample.

[0222] Subsequently, a block containing an alternative sample is set as a candidate block, and a merge candidate for the current block can be derived based on the said candidate block.

[0223] Merge candidates can also be derived from temporal neighbor blocks included in a picture different from the current block. For example, merge candidates can be derived from collaborating blocks included in a collaborating picture.

[0224] The motion information of the merge candidate can be set to be identical to the motion information of the candidate block. For example, at least one of the motion vector, reference picture index, prediction direction, or bidirectional weight index of the candidate block can be set as the motion information of the merge candidate.

[0225] A list of merge candidates including merge candidates can be generated (S1402). The merge candidates can be classified into adjacent merge candidates derived from neighboring blocks adjacent to the current block and non-adjacent merge candidates derived from non-neighboring blocks.

[0226] The indices of merge candidates within the merge candidate list may be assigned according to a predetermined order. For example, the index assigned to an adjacent merge candidate may have a smaller value than the index assigned to a non-adjacent merge candidate. Alternatively, an index may be assigned to each of the merge candidates based on the index of each block shown in FIG. 15 or FIG. 17.

[0227] If multiple merge candidates are included in the merge candidate, at least one of the multiple merge candidates can be selected (S1403). At this time, information indicating whether the movement information of the current block is derived from an adjacent merge candidate can be signaled through a bitstream. The information may be a 1-bit flag. For example, a syntax element isAdjancentMergeFlag indicating whether the movement information of the current block is derived from an adjacent merge candidate can be signaled through a bitstream. If the value of the syntax element isAdjancentMergeFlag is 1, the movement information of the current block can be derived based on an adjacent merge candidate. On the other hand, if the value of the syntax element isAdjancentMergeFlag is 0, the movement information of the current block can be derived based on a non-adjacent merge candidate.

[0228] Table 1 shows a syntax table containing the syntax element isAdjancentMergeFlag.

[0229] coding_unit( x0, y0, cbWidth, cbHeight, treeType ) {Descriptorif( slice_type != I ) {pred_mode_flagae(v)}if( CuPredMode[ x0 ][ y0 ] = = MODE_INTRA ) {if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) {intra_luma_mpm_flag[ x0 ][ y0 ]if( intra_luma_mpm_flag[ x0 ][ y0 ] )intra_luma_mpm_idx[ x0 ][ y0 ]ae(v)elseintra_luma_mpm_remainder[ x0 ][ y0 ]ae(v)}if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA )intra_chroma_pred_mode[ x0 ][ y0 ]ae(v)} else {if (cu_skip_falg[x0][y0]) {if (MaxNumMergeCand > 1){isAdjacentMergeflagae(v)if (isAdjcanetMergeflag){merge_idx[x0][y0]ae(v)} else{NA_merge_idx[x0][y0]ae(v)}}} else { / * MODE_INTER* / merge_flag[x0][y0]ae(v)if (merge_flag[x0][y0]){if (MaxNumMergeCand > 1){isAdjacentMergeflagae(v)if (isAdjcanetMergeflag){merge_idx[x0][y0]ae(v)} else{NA_merge_idx[x0][y0]ae(v)}}}if( CuPredMode[ x0 ][ y0 ] != MODE_INTRA )cu_cbfae(v)if( cu_cbf ) {transform_tree( x0, y0, cbWidth, cbHeight, treeType )}

[0230] Information for specifying one of multiple merge candidates can be signaled via a bitstream. For example, information indicating the index of one of the merge candidates included in the merge candidate list can be signaled via a bitstream.

[0231] If isAdjacentMergeflag is 1, the syntax element merge_idx, which specifies one of the adjacent merge candidates, may be signaled. The maximum value of the syntax element merge_idx can be set to the number of adjacent merge candidates differed by 1.

[0232] When isAdjacentMergeflag is 0, a syntax element NA_merge_idx specifying one of the non-adjacent merge candidates may be signaled. The syntax element NA_merge_idx represents the value obtained by differentiating the number of adjacent merge candidates from the index of the non-adjacent merge candidate. The decoder can select a non-adjacent merge candidate by adding the number of adjacent merge candidates to the index specified by NA_merge_idx.

[0233]

[0234] If the number of merge candidates included in the merge candidate list is less than the threshold value, the merge candidate included in the inter-region motion information table may be added to the merge candidate list. Here, the threshold value may be the maximum number of merge candidates that the merge candidate list can include, or the value obtained by subtracting an offset from the maximum number of merge candidates. The offset may be a natural number such as 1 or 2. The inter-region motion information table may include merge candidates derived based on blocks encoded / decoded prior to the current block.

[0235] The inter-domain motion information table includes merge candidates derived from blocks encoded / decoded based on inter predictions within the current picture. For example, the motion information of the merge candidates included in the inter-domain motion information table may be set to be identical to the motion information of the blocks encoded / decoded based on inter predictions. Here, the motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index.

[0236] For the sake of convenience of explanation, merge candidates included in the inter-region motion information table will be referred to as inter-region merge candidates.

[0237] The maximum number of merge candidates that an inter-domain motion information table can include may be predefined in the encoder and decoder. For example, the maximum number of merge candidates that an inter-domain motion information table can include may be 1, 2, 3, 4, 5, 6, 7, 8 or more (e.g., 16).

[0238] Alternatively, information indicating the maximum number of merge candidates in the inter-regional motion information table may be signaled via a bitstream. This information may be signaled at the sequence, picture, or slice level.

[0239] Alternatively, the maximum number of merge candidates for the inter-region motion information table can be determined based on the size of the picture, the size of the slice, or the size of the coding tree unit.

[0240] The inter-area motion information table can be initialized at the level of a picture, slice, tile, brick, coding tree unit, or coding tree unit line (row or column). For example, if a slice is initialized, the inter-area motion information table is also initialized, so that the inter-area motion information table may not contain any merge candidates.

[0241] Alternatively, information indicating whether to initialize the inter-area motion information table may be signaled via a bitstream. This information may be signaled at the slice, tile, brick, or block level. Until this information instructs to initialize the inter-area motion information table, the pre-configured inter-area motion information table may be used.

[0242] Alternatively, information regarding initial inter-region merge candidates can be signaled via a picture parameter set or a slice header. Even if the slice is initialized, the inter-region motion information table may contain initial inter-region merge candidates. Accordingly, inter-region merge candidates can also be used for the first encoding / decoding target block within the slice.

[0243] Blocks are encoded / decoded according to the encoding / decoding order, and the encoded / decoded blocks based on inter prediction can be sequentially set as inter-region merge candidates according to the encoding / decoding order.

[0244] FIG. 20 is a diagram illustrating the update pattern of an inter-region motion information table.

[0245] For the current block, if an inter prediction is performed (S2001), an inter-region merge candidate can be derived based on the current block (S2002). The movement information of the inter-region merge candidate can be set to be the same as the movement information of the current block.

[0246] If the inter-region motion information table is empty (S2003), an inter-region merge candidate derived based on the current block can be added to the inter-region motion information table (S2004).

[0247] If the inter-region motion information table already contains inter-region merge candidates (S2003), a redundancy check may be performed on the motion information of the current block (or, inter-region merge candidates derived based thereon) (S2005). The redundancy check is intended to determine whether the motion information of the inter-region merge candidates stored in the inter-region motion information table is identical to the motion information of the current block. The redundancy check may be performed on all inter-region merge candidates stored in the inter-region motion information table. Alternatively, among the inter-region merge candidates stored in the inter-region motion information table, the redundancy check may be performed on inter-region merge candidates whose index is above or below a threshold value.

[0248] If there are no inter-predicted merge candidates that have the same motion information as the current block, inter-region merge candidates derived based on the current block can be added to the inter-region motion information table (S2008). Whether the inter-predicted merge candidates are identical can be determined based on whether the motion information of the inter-predicted merge candidates (e.g., motion vector and / or reference picture index, etc.) is identical.

[0249] At this time, if the maximum number of inter-region merge candidates is already stored in the inter-region motion information table (S2006), the oldest inter-region merge candidate can be deleted (S2007), and an inter-region merge candidate derived based on the current block can be added to the inter-region motion information table (S2008).

[0250] Inter-region merge candidates can each be identified by an index. When an inter-region merge candidate derived from the current block is added to the inter-region motion information table, the lowest index (e.g., 0) is assigned to the said inter-region merge candidate, and the indices of the previously stored inter-region merge candidates are increased by 1. At this time, if the maximum number of inter-predicted merge candidates has already been stored in the inter-region motion information table, the inter-region merge candidate with the largest index is removed.

[0251] Alternatively, when an inter-region merge candidate derived from the current block is added to the inter-region motion information table, the largest index may be assigned to the inter-region merge candidate. For example, if the number of inter-predicted merge candidates already stored in the inter-region motion information table is less than the maximum value, an index with a value equal to the number of previously stored inter-predicted merge candidates may be assigned to the inter-region merge candidate. Alternatively, if the number of inter-predicted merge candidates already stored in the inter-region motion information table is equal to the maximum value, an index obtained by subtracting 1 from the maximum value may be assigned to the inter-region merge candidate. Additionally, the inter-region merge candidate with the smallest index is removed, and the indices of the remaining previously stored inter-region merge candidates are decreased by 1.

[0252] Figure 21 is a diagram showing the update pattern of the inter-region merge candidate table.

[0253] It is assumed that as an inter-region merge candidate derived from the current block is added to the inter-region merge candidate table, the largest index is assigned to said inter-region merge candidate. Additionally, it is assumed that the maximum number of inter-region merge candidates is already stored in the inter-region merge candidate table.

[0254] When adding an inter-region merge candidate HmvpCand[n+1] derived from the current block to the inter-region merge candidate table HmvpCandList, the inter-region merge candidate HmvpCand[0] with the smallest index among the previously stored inter-region merge candidates may be deleted, and the indices of the remaining inter-region merge candidates may be decreased by 1. Additionally, the index of the inter-region merge candidate HmvpCand[n+1] derived from the current block may be set to a maximum value (n in the example illustrated in FIG. 21).

[0255] If an inter-region merge candidate identical to the inter-region merge candidate derived based on the current block is already stored (S2005), the inter-region merge candidate derived based on the current block may not be added to the inter-region motion information table (S2009).

[0256] Alternatively, while adding an inter-region merge candidate derived based on the current block to the inter-region motion information table, a previously stored inter-region merge candidate identical to the said inter-region merge candidate may be removed. In this case, the same effect as updating the index of the previously stored inter-region merge candidate occurs.

[0257] Figure 22 is a diagram showing an example in which the index of a previously stored inter-region merge candidate is updated.

[0258] If the index of a previously stored inter-region merge candidate identical to mvCand derived based on the current block is hIdx, the previously stored inter-region merge candidate may be deleted, and the index of inter-region merge candidates with an index greater than hIdx may be decreased by 1. For example, in the example illustrated in FIG. 22, HmvpCand[2] identical to mvCand is deleted from the inter-region motion information table HvmpCandList, and the indices from HmvpCand[3] to HmvpCand[n] are shown to be decreased by 1.

[0259] And, the inter-region merge candidate mvCand derived based on the current block can be added to the end of the inter-region motion information table.

[0260] Alternatively, the index assigned to a previously stored inter-region merge candidate identical to the inter-region merge candidate derived based on the current block can be updated. For example, the index of a previously stored inter-region merge candidate can be changed to a minimum or maximum value.

[0261] Motion information of blocks included in a designated area may be configured not to be added to the inter-area motion information table. For example, inter-area merge candidates derived based on motion information of blocks included in the merge processing area may not be added to the inter-area motion information table. Since the encoding / decoding order is not defined for the blocks included in the merge processing area, it is inappropriate to use the motion information of any one of them when predicting the inter of another block. Accordingly, inter-area merge candidates derived based on the blocks included in the merge processing area may not be added to the inter-area motion information table.

[0262] When motion compensation prediction is performed at the sub-block level, inter-region merge candidates can be derived based on the motion information of a representative sub-block among multiple sub-blocks included in the current block. For example, when a sub-block merge candidate is used for the current block, inter-region merge candidates can be derived based on the motion information of a representative sub-block among the sub-blocks.

[0263] The motion vectors of the sub-blocks can be derived in the following order. First, one of the merge candidates included in the merge candidate list of the current block is selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. Then, by adding the initial shift vector to the position (xSb, ySb) of the reference sample (e.g., top-left sample or middle position sample) of each sub-block within the coding block, a shift sub-block with a reference sample position of (xColSb, yColSb) can be derived. Equation 4 below represents the formula for deriving the shift sub-block.

[0264]

[0265] Then, the motion vector of the collocated block corresponding to the center position of the sub-block containing (xColSb, yColSb) can be set to the motion vector of the sub-block containing (xSb, ySb).

[0266] The representative sub-block may refer to a sub-block containing the top-left or center sample of the current block.

[0267] Figure 23 is a diagram showing the location of the representative sub-block.

[0268] FIG. 23 (a) shows an example where a sub-block located at the top left of the current block is set as the representative sub-block, and FIG. 23 (b) shows an example where a sub-block located at the center of the current block is set as the representative sub-block. When motion compensation prediction is performed on a sub-block basis, an inter-region merge candidate of the current block can be derived based on the motion vector of a sub-block containing a sample at the top left of the current block or a sub-block containing a sample at the center of the current block.

[0269] Based on the inter prediction mode of the current block, it may be determined whether to use the current block as an inter-region merge candidate. For example, a block encoded / decoded based on an affine motion model may be set as unavailable for use as an inter-region merge candidate. Accordingly, even if the current block is encoded / decoded by inter prediction, if the inter prediction mode of the current block is an affine prediction mode, the inter prediction motion information table may not be updated based on the current block.

[0270] Alternatively, an inter-region merge candidate may be derived based on at least one sub-block vector among the sub-blocks included in the block encoded / decoded based on the affine motion model. For example, an inter-region merge candidate may be derived using the sub-block located at the top-left, the sub-block located at the center, or the sub-block located at the top-right of the current block. Alternatively, the average value of the sub-block vectors of multiple sub-blocks may be set as the motion vector of the inter-region merge candidate.

[0271] Alternatively, an inter-region merge candidate may be derived based on the average value of the affine seed vectors of the blocks encoded / decoded based on the affine motion model. For example, the average of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block may be set as the motion vector of the inter-region merge candidate.

[0272] Alternatively, an inter-region motion information table may be configured for each inter-prediction mode. For example, at least one of an inter-region motion information table for a block encoded / decoded by an intra-block copy, an inter-region motion information table for a block encoded / decoded based on a translational motion model, or an inter-region motion information table for a block encoded / decoded based on an affine motion model may be defined. Depending on the inter-prediction mode of the current block, any one of the multiple inter-region motion information tables may be selected.

[0273] Figure 24 shows an example of how an inter-region motion information table is generated for each inter-prediction mode.

[0274] If a block is encoded / decoded based on a non-affine motion model, the inter-region merge candidate mvCand derived based on said block can be added to the inter-region non-affine motion information table HmvpCandList. On the other hand, if a block is encoded / decoded based on an affine motion model, the inter-region merge candidate mvAfCand derived based on said block can be added to the inter-region affine motion information table HmvpAfCandList.

[0275] The affine seed vectors of the block may be stored in the inter-region merge candidates derived from the blocks encoded / decoded based on the affine motion model. Accordingly, the inter-region merge candidates can be used as merge candidates to derive the affine seed vector of the current block.

[0276]

[0277] In addition to the inter-region motion information table described above, additional inter-region motion information tables may be defined. In addition to the inter-region motion information table described above (hereinafter referred to as the first inter-region motion information table), a long-term motion information table (hereinafter referred to as the second inter-region motion information table) may be defined. Here, the long-term motion information table includes long-term merge candidates.

[0278] If both the first inter-region motion information table and the second inter-region motion information table are empty, inter-region merge candidates can first be added to the second inter-region motion information table. Only after the number of available inter-region merge candidates in the second inter-region motion information table reaches the maximum number can inter-region merge candidates be added to the first inter-region motion information table.

[0279] Alternatively, a single inter-predicted merge candidate may be added to both the second inter-region motion information table and the first inter-region motion information table.

[0280] At this time, the second inter-region motion information table, once configured, may no longer be updated. Alternatively, if the decoded area is greater than a predetermined ratio of the slice, the second inter-region motion information table may be updated. Alternatively, the second inter-region motion information table may be updated for every N coding tree unit lines.

[0281] On the other hand, the first inter-region motion information table may be updated whenever a block encoded / decoded by inter prediction occurs. However, the inter-region merge candidate added to the second inter-region motion information table may be configured not to be used to update the first inter-region motion information table.

[0282] Information for selecting either the first inter-region motion information table or the second inter-region motion information table may be signaled via a bitstream. If the number of merge candidates included in the merge candidate list is less than a threshold value, merge candidates included in the inter-region motion information table indicated by the information may be added to the merge candidate list.

[0283] Alternatively, an inter-region motion information table may be selected based on the current block's size, shape, inter-prediction mode, whether bidirectional prediction is performed, whether motion vector refinement is performed, or whether triangular partitioning is performed.

[0284] Alternatively, even if an inter-region merge candidate included in the first inter-region motion information table is added, if the number of merge candidates included in the merge candidate list is less than the maximum number of merges, an inter-region merge candidate included in the second inter-region motion information table can be added to the merge candidate list.

[0285] Figure 25 is a diagram showing an example in which inter-region merge candidates included in a long-term motion information table are added to a merge candidate list.

[0286] If the number of merge candidates included in the merge candidate list is less than the maximum number, the inter-region merge candidates included in the first inter-region motion information table HmvpCandList can be added to the merge candidate list. Even after adding the inter-region merge candidates included in the first inter-region motion information table to the merge candidate list, if the number of merge candidates included in the merge candidate list is less than the maximum number, the inter-region merge candidates included in the long-term motion information table HmvpLTCandList can be added to the merge candidate list.

[0287] Table 2 shows the process of adding inter-region merge candidates included in the long-term motion information table to the merge candidate list.

[0288] For each candidate in HMVPCandList with index HMVPLTIdx = 1..numHMVPLTCand, the following ordered steps are repeated until combStop is equal to true- sameMotion is set to FALSE- If hmvpStop is equal to FALSE and numCurrMergecand is less than (MaxNumMergeCand-1), hmvpLT is set to TRUE- If HMVPLTCandList[NumLTHmvp-HMVPLTIdx] have the same motion vectors and the same reference indices with any mergeCandList[i] with I being 0.. numOrigMergeCand-1 and HasBeenPruned[i] equal to false, sameMotion is set to true- If sameMotion is equal to false, mergeCandList[numCurrMergeCand++] is set to HMVPLTCandList[NumLTHmvp-HMVPLTIdx]- If numCurrMergeCand is equal to (MaxNumMergeCand-1), hmvpLTStop is set to TRUE

[0289] Inter-region merge candidates can be configured to include additional information in addition to movement information. For example, block size, shape, or block partition information can be additionally stored for inter-region merge candidates. When configuring the merge candidate list for the current block, only inter-predicted merge candidates that have the same or similar size, shape, or partition information as the current block can be used, or inter-predicted merge candidates that have the same or similar size, shape, or partition information as the current block can be added to the merge candidate list first.

[0290] Alternatively, an inter-region motion information table can be generated based on block size, shape, or partition information. Among the multiple inter-region motion information tables, an inter-region motion information table that corresponds to the shape, size, or partition information of the current block can be used to generate a merge candidate list for the current block.

[0291] If the number of merge candidates included in the current block's merge candidate list is less than the threshold value, inter-region merge candidates included in the inter-region motion information table can be added to the merge candidate list. The above addition process is performed in ascending or descending order based on index. For example, the inter-region merge candidate with the largest index can be added to the merge candidate list first.

[0292] If you intend to add an inter-region merge candidate included in the inter-region motion information table to the merge candidate list, a duplicate check may be performed between the inter-region merge candidate and the merge candidates already stored in the merge candidate list.

[0293] For example, Table 3 shows the process of adding inter-region merge candidates to the merge candidate list.

[0294] For each candidate in HMVPCandList with index HMVPIdx = 1.. numCheckedHMVPCand, the following ordered steps are repeated until combStop is equal to true- sameMotion is set to false- If HMVPCandList[NumHmvp-HMVPIdx] have the same motion vectors and the same reference indices with any mergeCandList[i] with I being 0.. numOrigMergeCand-1 and HasBeenPruned[i] equal to false, sameMotion is set to true- If sameMotion is equal to false, mergeCandList[numCurrMergeCand++] is set to HMVPCandList[NumHmvp-HMVPIdx]- If numCurrMergeCand is equal to (MaxNumMergeCand-1), hmvpStop is set to TRUE

[0295] Redundancy checks may be performed on only some of the inter-region merge candidates included in the inter-region motion information table. For example, redundancy checks may be performed only on inter-region merge candidates whose index is above or below a threshold value. Alternatively, redundancy checks may be performed only on the N merge candidates with the largest indexes or the N merge candidates with the smallest indexes.

[0296] Alternatively, a redundancy check may be performed only on some of the merge candidates already stored in the merge candidate list. For example, a redundancy check may be performed only on merge candidates whose index is above or below a threshold value, or on merge candidates derived from a block at a specific location. Here, the specific location may include at least one of the left neighbor block, the top neighbor block, the right top neighbor block, or the left bottom neighbor block of the current block.

[0297] Figure 26 illustrates an example where redundancy checks are performed on only some of the merge candidates.

[0298] If you want to add an inter-regional merge candidate HmvpCand[j] to the merge candidate list, you can perform a duplicate check with the two merge candidates with the largest indices, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], for the inter-regional merge candidate. Here, NumMerge may represent the number of available spatial merge candidates and temporal merge candidates.

[0299] Unlike the illustrated example, if you intend to add an inter-region merge candidate HmvpCand[j] to the merge candidate list, you may perform a duplicate check on the inter-region merge candidate with up to two merge candidates with the smallest indices. For example, you may check for the identity of mergeCandList[0] and mergeCandList[1] with HmvpCand[j]. Alternatively, you may perform a duplicate check only on merge candidates derived from a specific location. For example, you may perform a duplicate check on at least one of a merge candidate derived from a neighboring block located to the left of the current block or a merge candidate derived from a neighboring block located above the current block. If there is no merge candidate derived from a specific location in the merge candidate list, you may add the inter-region merge candidate to the merge candidate list without a duplicate check.

[0300]

[0301] If a merge candidate identical to the first inter-region merge candidate is found, the redundancy check with the merge candidate identical to the first inter-region merge candidate may be omitted during the redundancy check for the second inter-region merge candidate.

[0302] Figure 27 is a diagram showing an example where redundancy checks with specific merge candidates are omitted.

[0303] When adding the inter-region merge candidate HmvpCand[i] with index i to the merge candidate list, a duplicate check is performed between the inter-region merge candidate and the merge candidates already stored in the merge candidate list. At this time, if a merge candidate mergeCandList[j] identical to the inter-region merge candidate HmvpCand[i] is found, the inter-region merge candidate HmvpCand[i] is not added to the merge candidate list, and a duplicate check between the inter-region merge candidate HmvpCand[i-1] with index i-1 and the merge candidates may be performed. In this case, the duplicate check between the inter-region merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] may be omitted.

[0304] For example, in the example illustrated in FIG. 27, it was determined that HmvpCand[i] and mergeCandList[2] are identical. Accordingly, HmvpCand[i] is not added to the merge candidate list, and a duplicate check for HmvpCand[i-1] can be performed. At this time, the duplicate check between HvmpCand[i-1] and mergeCandList[2] can be omitted.

[0305] If the number of merge candidates included in the current block's merge candidate list is less than the threshold value, in addition to the inter-regional merge candidate, at least one of the pairwise merge candidate or the zero merge candidate may be included. A pairwise merge candidate refers to a merge candidate whose motion vector is the average of the motion vectors of two or more merge candidates, and a zero merge candidate refers to a merge candidate whose motion vector is 0.

[0306] In the current block's merge candidate list, merge candidates can be added in the following order.

[0307] Spatial Merge Candidate - Temporal Merge Candidate - Inter-region Merge Candidate - (Inter-region Affine Merge Candidate) - Pairwise Merge Candidate - Zero Merge Candidate

[0308] Spatial merge candidates refer to merge candidates derived from at least one of neighboring blocks or non-neighboring blocks, and temporal merge candidates refer to merge candidates derived from a previous reference picture. Inter-region affine merge candidates represent inter-region merge candidates derived from blocks encoded / decoded by an affine motion model.

[0309] The inter-region motion information table can also be used in motion vector prediction mode. For example, if the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold value, the inter-region merge candidates included in the inter-region motion information table can be set as motion vector prediction candidates for the current block. Specifically, the motion vectors of the inter-region merge candidates can be set as motion vector prediction candidates.

[0310] If any one of the motion vector prediction candidates included in the list of motion vector prediction candidates for the current block is selected, the selected candidate can be set as the motion vector predictor for the current block. Subsequently, after decoding the motion vector residual value of the current block, the motion vector predictor and the motion vector residual value can be combined to obtain the motion vector of the current block.

[0311] The list of candidate motion vector predictions for the current block can be constructed in the following order.

[0312] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Inter-decoding region merge candidate - (Inter-decoding region affine merge candidate) - Zero motion vector prediction candidate

[0313] Spatial motion vector prediction candidates refer to motion vector prediction candidates derived from at least one of neighboring blocks or non-neighboring blocks, and temporal motion vector prediction candidates refer to motion vector prediction candidates derived from a previous reference picture. Inter-region affine merge candidates represent inter-region motion vector prediction candidates derived from blocks encoded / decoded by an affine motion model. Zero motion vector prediction candidates represent candidates where the value of the motion vector is 0.

[0314]

[0315] When a merge candidate for the current block is selected, the motion vector of the selected merge candidate is set as the initial motion vector, and motion compensation prediction for the current block can be performed using the motion vector derived by adding or subtracting an offset vector to the initial motion vector. Deriving a new motion vector by adding or subtracting an offset vector to the motion vector of the merge candidate can be defined as a merge offset vector encoding method.

[0316] Information indicating whether to use the merge offset encoding method may be signaled via a bitstream. This information may be a 1-bit flag, merge_offset_vector_flag. For example, a value of 1 for merge_offset_vector_flag indicates that the merge offset vector encoding method is applied to the current block. If the merge offset vector encoding method is applied to the current block, the motion vector of the current block can be derived by adding or subtracting the offset vector to the motion vector of the merge candidate. A value of 0 for merge_offset_vector_flag indicates that the merge offset vector encoding method is not applied to the current block. If the merge offset encoding method is not applied, the motion vector of the merge candidate can be set as the motion vector of the current block.

[0317] The above flags may be signaled only when the value of the skip flag, which indicates whether skip mode is applied, is true, or the value of the merge flag, which indicates whether merge mode is applied, is true. For example, when the value of skip_flag, which indicates that skip mode is applied to the current block, is 1, or when the value of merge_flag, which indicates that merge mode is applied to the current block, is 1, the merge_offset_vector_flag may be encoded and signaled.

[0318] If it is determined that a merge offset encoding method is applied to the current block, at least one of information specifying one of the merge candidates included in the merge candidate list, information indicating the magnitude of the offset vector, or information indicating the direction of the offset vector may be additionally signaled.

[0319] Information for determining the maximum number of merge candidates that a merge candidate list can include can be signaled via a bitstream. For example, the maximum number of merge candidates that a merge candidate list can include can be set to a natural number less than or equal to 6.

[0320] If it is determined that the merge offset encoding method is applied to the current block, only a pre-configured maximum number of merge candidates can be set as the initial motion vector of the current block. In other words, the number of merge candidates available to the current block can be adaptively determined depending on whether the merge offset encoding method is applied. For example, if the value of merge_offset_vector_flag is set to 0, the maximum number of merge candidates available to the current block is set to M, whereas if the value of merge_offset_vector_flag is set to 1, the maximum number of merge candidates available to the current block can be set to N. Here, M represents the maximum number of merge candidates that the merge candidate list can contain, and N represents a natural number equal to or smaller than M.

[0321] For example, if M is 6 and N is 2, the two merge candidates with the smallest indices among the merge candidates included in the merge candidate list may be set as available for the current block. Accordingly, the motion vector of the merge candidate with index value 0 or the motion vector of the merge candidate with index value 1 may be set as the initial motion vector of the current block. If M and N are the same (e.g., M and N are 2), all merge candidates included in the merge candidate list may be set as available for the current block.

[0322] Alternatively, whether neighboring blocks are available as merge candidates may be determined based on whether the merge offset vector encoding method is applied to the current block. For example, if the value of merge_offset_vector_flag is 1, at least one of the neighboring blocks adjacent to the top-right corner, the neighboring blocks adjacent to the bottom-left corner, or the neighboring blocks adjacent to the bottom-left corner of the current block may be set as unavailable as merge candidates. Accordingly, if the merge offset vector encoding method is applied to the current block, the motion vector of at least one of the neighboring blocks adjacent to the top-right corner, the neighboring blocks adjacent to the bottom-left corner, or the neighboring blocks adjacent to the bottom-left corner of the current block cannot be set as the initial motion vector. Alternatively, if the value of merge_offset_vector_flag is 1, the temporal neighboring blocks of the current block may be set as unavailable as merge candidates.

[0323] If the merge offset vector encoding method is applied to the current block, it can be configured not to use at least one of the pairwise merge candidates or zero merge candidates. Accordingly, when the value of merge_offset_vector_flag is 1, at least one of the pairwise merge candidate or zero merge candidate may not be added to the merge candidate list even if the number of merge candidates included in the merge candidate list is less than the maximum number.

[0324] The motion vector of a merge candidate can be set as the initial motion vector of the current block. In this case, if there are multiple merge candidates available to the current block, information specifying one of the multiple merge candidates can be signaled via a bitstream. For example, if the maximum number of merge candidates that can be included in the merge candidate list is greater than 1, information `merge_idx` pointing to one of the multiple merge candidates can be signaled via a bitstream. That is, under the merge offset encoding method, a merge candidate can be specified by the information `merge_idx` for specifying one of the multiple merge candidates. The initial motion vector of the current block can be set as the motion vector of the merge candidate pointed to by `merge_idx`.

[0325] On the other hand, if the number of merge candidates available to the current block is 1, the signaling of information for specifying the merge candidate can be omitted. For example, if the maximum number of merge candidates that can be included in the merge candidate list is not greater than 1, the signaling of information merge_idx for specifying the merge candidate can be omitted. That is, under the merge offset encoding method, if there is 1 merge candidate in the merge candidate list, the encoding of information merge_idx for specifying the merge candidate can be omitted, and an initial motion vector can be determined based on the merge candidate included in the merge candidate list. The motion vector of the above merge candidate can be set as the initial motion vector of the current block.

[0326] As another example, after determining the merge candidate for the current block, it may be determined whether to apply the merge offset vector encoding method to the current block. For example, if the maximum number of merge candidates that can be included is greater than 1, information merge_idx to specify one of the merge candidates may be signaled. After selecting a merge candidate based on merge_idx, a merge_offset_vector_flag indicating whether the merge offset vector encoding method is applied to the current block may be decoded. Table 4 is a diagram showing a syntax table according to the above-described embodiment.

[0327] coding_unit( x0, y0, cbWidth, cbHeight, treeType ) {Descriptorif( slice_type != I ) {cu_skip_flag[ x0 ][ y0 ]ae(v)if( cu_skip_flag[ x0 ][ y0 ] = = 0 )pred_mode_flagae(v)}if( CuPredMode[ x0 ][ y0 ] = = MODE_INTRA ) {if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) {intra_luma_mpm_flag[ x0 ][ y0 ]if( intra_luma_mpm_flag[ x0 ][ y0 ] )intra_luma_mpm_idx[ x0 ][ y0 ]ae(v)elseintra_luma_mpm_remainder[ x0 ][ y0 ]ae(v)}if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA )intra_chroma_pred_mode[ x0 ][ y0 ]ae(v)} else { / * MODE_INTER * / if( cu_skip_flag[ x0 ][ y0 ] ) {if( merge_affine_flag[ x0 ][ y0 ] = = 0 && MaxNumMergeCand > 1 ) {merge_idx[ x0 ][ y0 ]ae(v)merge_offset_vector_flagae(v)if (merge_idx < 2 && merge_offset_vector_flag) {distance_idx[ x0 ][ y0 ]ae(v)direction_idx[ x0 ][ y0 ]ae(v)}}} else {merge_flag[ x0 ][ y0 ]ae(v)if( merge_flag[ x0 ][ y0 ] ) {if( merge_affine_flag[ x0 ][ y0 ] = = 0 && MaxNumMergeCand > 1){merge_idx[ x0 ][ y0 ]ae(v)merge_offset_vector_flagae(v)if (merge_idx < 2 &&merge_offset_vector_flag){distance_idx[ x0 ][ y0 ]ae(v)direction_idx[ x0 ][ y0 ]ae(v)}}} else {if( slice_type = = B )inter_pred_idc[ ][ y0 ]ae(v)if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16 ) {inter_affine_flag[ x0 ][ y0 ]ae(v)if( sps_affine_type_flag && inter_affine_flag[ x0 ][ y0 ] )cu_affine_type_flag[ x0 ][ y0 ]ae(v)}}

[0328] As another example, after determining the merge candidate for the current block, it may be decided whether to apply the merge offset vector encoding method to the current block only if the index of the determined merge candidate is less than the maximum number of merge candidates available when applying the merge offset vector encoding method. For example, if the value of the index information merge_idx is less than N, the merge_offset_vector_flag indicating whether to apply the merge offset vector encoding method to the current block may be encoded and signaled. If the value of the index information merge_idx is equal to or greater than N, the encoding of the merge_offset_vector_flag may be omitted. If the encoding of the merge_offset_vector_flag is omitted, it may be determined that the merge offset vector encoding method is not applied to the current block.

[0329] Alternatively, after determining the merge candidate for the current block, whether to apply the merge offset vector encoding method to the current block can be determined by considering whether the determined merge candidate has bidirectional motion information or unidirectional motion information. For example, if the value of index information merge_idx is less than N and the merge candidate selected by said index information has bidirectional motion information, a merge_offset_vector_flag indicating whether to apply the merge offset vector encoding method to the current block can be encoded and signaled. Alternatively, if the value of index information merge_idx is less than N and the merge candidate selected by said index information has unidirectional motion information, a merge_offset_vector_flag indicating whether to apply the merge offset vector encoding method to the current block can be encoded and signaled.

[0330] Alternatively, whether to apply the merge offset vector encoding method can be determined based on at least one of the size, shape, or whether the current block touches the boundary of a coding tree unit. If at least one of the size, shape, or whether the current block touches the boundary of a coding tree unit does not satisfy a preset condition, the encoding of the merge_offset_vector_flag, which indicates whether to apply the merge offset vector encoding method to the current block, can be omitted.

[0331] When a merge candidate is selected, the motion vector of the merge candidate can be set as the initial motion vector of the current block. Then, the offset vector can be determined by decoding information indicating the magnitude and direction of the offset vector. The offset vector may have a horizontal or a vertical component.

[0332] Information indicating the magnitude of the offset vector may be index information indicating any one of the vector magnitude candidates. For example, index information distance_idx indicating any one of the vector magnitude candidates may be signaled through a bitstream. Table 5 shows the binarization of index information distance_idx and the value of the variable DistFromMergeMV for determining the magnitude of the offset vector according to distance_idx.

[0333] distance_idx[x][y]binarizationDistFromMergeMV[x0][y0]00111022110431110841111016511111032611111106471111111128

[0334] The size of the offset vector can be derived by dividing the variable DistFromMergeMV by a preset value. Equation 5 shows an example of determining the size of the offset vector.

[0335]

[0336] According to the above mathematical formula 5, the value obtained by dividing the variable DistFromMegeMV by 4 or the value obtained by bit-shifting the variable DistFromMergeMV to the left by 2 can be set as the size of the offset vector.

[0337] It is also possible to use a larger or smaller number of vector size candidates than the examples shown in Table 5, or to set the range of motion vector offset size candidates differently from the examples shown in Table 5. For example, the magnitude of the horizontal or vertical component of the offset vector can be set so that it is not greater than 2 sample distances. Table 6 shows the values ​​of the variable DistFromMergeMV for determining the binarization of index information distance_idx and the magnitude of the offset vector according to distance_idx.

[0338] distance_idx[x][y]binarizationDistFromMergeMV[x0][y0]00111022110431118

[0339] Alternatively, the range of motion vector offset size candidates can be set differently based on the motion vector precision. For example, if the motion vector precision for the current block is fractional-pel, the values ​​of the variable DistFromMergeMV corresponding to the values ​​of the index information distance_idx can be set to 1, 2, 4, 8, 16, etc. Here, fractional-pel includes at least one of 1 / 16-pel, octo-pel, quarter-pel, or half-pel. On the other hand, if the motion vector precision for the current block is integer-pel, the values ​​of the variable DistFromMergeMV corresponding to the values ​​of the index information distance_idx can be set to 4, 8, 16, 32, 64, etc. That is, depending on the motion vector precision for the current block, the table referenced to determine the variable DistFromMergeMV can be set differently.

[0340] For example, if the motion vector precision of the current block or merge candidate is quarter-pel, the variable DistFromMergeMV pointed to by distance_idx can be derived using Table 5. On the other hand, if the motion vector precision of the current block or merge candidate is integer-pel, the value of the variable DistFromMergeMV pointed to by distance_idx in Table 5 can be multiplied by N (e.g., 4) to derive the value of the variable DistFromMergeMV.

[0341] Information for determining motion vector precision can be signaled via a bitstream. For example, said information can be signaled at the sequence, picture, slice, or block level. Accordingly, the range of vector size candidates can be set differently by the motion vector precision information signaled via the bitstream. Alternatively, the motion vector precision can be determined based on the merge candidate of the current block. For example, the motion vector precision of the current block can be set to be the same as the motion vector precision of the merge candidate.

[0342] Alternatively, information for determining the search range of the offset vector can be signaled via a bitstream. Based on the search range, at least one of the number of vector size candidates, the minimum value, or the maximum value among the vector size candidates may be determined. For example, a flag `merge_offset_vector_flag` for determining the search range of the offset vector may be signaled via a bitstream. The information may be signaled via a sequence header, a picture header, or a slice header.

[0343] For example, if the value of merge_offset_extend_range_flag is 0, the size of the offset vector can be set so that it does not exceed 2. Accordingly, the maximum value of DistFromMergeMV can be set to 8. On the other hand, if the value of merge_offset_extend_range_flag is 1, the size of the offset vector can be set so that it does not exceed a distance of 32 samples. Accordingly, the maximum value of DistFromMergeMV can be set to 128.

[0344] The size of an offset vector can be determined using a flag that indicates whether the size of the offset vector is greater than a threshold value. For example, a flag `distance_flag` indicating whether the size of the offset vector is greater than a threshold value can be signaled through a bitstream. The threshold value can be 1, 2, 4, 8, or 16. For example, `distance_flag` being 1 indicates that the size of the offset vector is greater than 4. Conversely, `distance_flag` being 0 indicates that the size of the offset vector is 4 or less.

[0345] If the magnitude of the offset vector is greater than the threshold value, the difference between the magnitude of the offset vector and the threshold value can be derived using the index information distance_idx. Alternatively, if the magnitude of the offset vector is less than or equal to the threshold value, the magnitude of the offset vector can be determined using the index information distance_idx. Table 7 is a syntax table showing the encoding patterns of distance_flag and distance_idx.

[0346] coding_unit( x0, y0, cbWidth, cbHeight, treeType ) {Descriptorif( slice_type != I ) {cu_skip_flag[ x0 ][ y0 ]ae(v)if( cu_skip_flag[ x0 ][ y0 ] = = 0 )pred_mode_flagae(v)}if( CuPredMode[ x0 ][ y0 ] = = MODE_INTRA ) {if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) {intra_luma_mpm_flag[ x0 ][ y0 ]if( intra_luma_mpm_flag[ x0 ][ y0 ] )intra_luma_mpm_idx[ x0 ][ y0 ]ae(v)elseintra_luma_mpm_remainder[ x0 ][ y0 ]ae(v)}if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA )intra_chroma_pred_mode[ x0 ][ y0 ]ae(v)} else { / * MODE_INTER * / if( cu_skip_flag[ x0 ][ y0 ] ) {if( merge_affine_flag[ x0 ][ y0 ] = = 0 && MaxNumMergeCand > 1 ) {merge_idx[ x0 ][ y0 ]ae(v)merge_offset_vector_flagae(v)if (merge_idx < 2 && merge_offset_vector_flag) {distance_flag[ x0 ][ y0 ]ae(v)distance_idx[ x0 ][ y0 ]direction_idx[ x0 ][ y0 ]ae(v)}}} else {merge_flag[ x0 ][ y0 ]ae(v)if( merge_flag[ x0 ][ y0 ] ) {if( merge_affine_flag[ x0 ][ y0 ] = = 0 && MaxNumMergeCand > 1){merge_idx[ x0 ][ y0 ]ae(v)merge_offset_vector_flagae(v)if (merge_idx < 2 && merge_offset_vector_flag){distance_flag[ x0 ][ y0 ]ae(v)distance_idx[ x0 ][ y0 ]ae(v)direction_idx[ slice_type = = B )inter_pred_idc[ x0 ][ y0 ]ae(v)if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16 ) {inter_affine_flag[ y0 ] )cu_affine_type_flag[ x0 ][ y0 ]ae(v)}}

[0347] Mathematical Equation 6 shows an example of deriving the variable DistFromMergeMV to determine the magnitude of the offset vector using distance_flag and distance_idx.

[0348]

[0349] In Equation 6, the value of distance_flag can be set to 1 or 0. The value of distance_idx can be set to 1, 2, 4, 8, 16, 32, 64, 128, etc. N represents a coefficient determined by the threshold value. For example, if the threshold value is 4, N can be set to 16.

[0350] Information indicating the direction of the offset vector may be index information indicating any one of the vector direction candidates. For example, index information direction_idx indicating any one of the vector direction candidates may be signaled through a bitstream. Table 8 shows the binarization of the index information direction_idx and the direction of the offset vector according to direction_idx.

[0351] direction_idx[x][y]binarizationsign[x][y][0]sign[x][y][1]000+10101-102100+13110-1

[0352] In Table 8, sign[0] indicates the horizontal direction and sign[1] indicates the vertical direction. +1 indicates that the value of the x or y component of the offset vector is positive, and -1 indicates that the value of the x or y component of the offset vector is negative. Equation 7 shows an example of determining an offset vector based on the magnitude and direction of the offset vector.

[0353]

[0354] In mathematical formula 7, offsetMV[0] represents the vertical component of the offset vector, and offsetMV[1] represents the horizontal component of the offset vector.

[0355] Figure 28 is a diagram showing an offset vector according to the values ​​of distance_idx, which represents the magnitude of the offset vector, and direction_idx, which represents the direction of the offset vector.

[0356] As shown in the example illustrated in FIG. 28, the magnitude and direction of the offset vector can be determined based on the values ​​of distance_idx and direction_idx. The maximum magnitude of the offset vector can be set so as not to exceed a threshold value. Here, the threshold value may have a value predefined in the encoder and decoder. For example, the threshold value may be a distance of 32 samples. Alternatively, the threshold value may be determined based on the magnitude of the initial motion vector. For example, the threshold value for the horizontal direction may be set based on the magnitude of the horizontal component of the initial motion vector, and the threshold value for the vertical direction may be set based on the magnitude of the vertical component of the initial motion vector.

[0357] When a merge candidate has bidirectional motion information, the L0 motion vector of the merge candidate can be set as the L0 initial motion vector of the current block, and the L1 motion vector of the merge candidate can be set as the L1 initial motion vector of the current block. At this time, the L0 offset vector and the L1 offset vector can be determined by considering the output order difference value between the L0 reference picture of the merge candidate and the current picture (hereinafter referred to as the L0 difference value) and the output order difference value between the L1 reference picture of the merge candidate and the current picture (hereinafter referred to as the L1 difference value).

[0358] First, if the signs of the L0 difference value and the L1 difference value are the same, the L0 offset vector and the L1 offset vector can be set identically. On the other hand, if the signs of the L0 difference value and the L1 difference value are different, the L1 offset vector can be set in the opposite direction to the L0 offset vector.

[0359] The magnitude of the L0 offset vector and the magnitude of the L1 offset vector can be set to be the same. Alternatively, the magnitude of the L1 offset vector can be determined by scaling the L0 offset vector based on the L0 difference value and the L1 difference value.

[0360] For example, Equation 8 represents the L0 offset vector and the L1 offset vector when the signs of the L0 difference value and the L1 difference value are the same.

[0361]

[0362] In mathematical equation 8, offsetMVL0[0] represents the horizontal component of the L0 offset vector, and offsetMVL0[1] represents the vertical component of the L0 offset vector. offsetMVL1[0] represents the horizontal component of the L1 offset vector, and offsetMVL1[1] represents the vertical component of the L1 offset vector.

[0363] Equation 9 represents the L0 offset vector and the L1 offset vector when the signs of the L0 difference value and the L1 difference value are different.

[0364]

[0365] You can define more than four candidate vector directions. Tables 9 and 10 show examples where eight candidate vector directions are defined.

[0366] direction_idx[x][y]binarizationsign[x][y][0]sign[x][y][1]0000+101001-1020100+130110-14100+1+15101+1-16110-1+17111-1-1

[0367] direction_idx[x][y]binarizationsign[x][y][0]sign[x][y][1]0000+101001-1020100+130110-14100+1 / 2+1 / 25101+1 / 2-1 / 26110-1 / 2+1 / 27111-1 / 2-1 / 2

[0368] In Tables 9 and 10, the absolute values ​​of sign[0] and sign[1] being greater than 0 indicate that the offset vector is diagonal. When Table 8 is used, the magnitudes of the x-axis and y-axis components of the diagonal offset vector are set to abs(offsetMV), whereas when Table 9 is used, the magnitudes of the x-axis and y-axis components of the diagonal offset vector can be set to abs(offsetMV / 2).

[0369] Figure 29 is a diagram showing an offset vector according to the values ​​of distance_idx, which represents the magnitude of the offset vector, and direction_idx, which represents the direction of the offset vector.

[0370] Figure 29 (a) is an example of the case where Table 8 is applied, and Figure 29 (b) is an example of the case where Table 9 is applied.

[0371] Information for determining at least one of the number or size of vector direction candidates may be signaled through a bitstream. For example, a flag merge_offset_direction_range_flag for determining vector direction candidates may be signaled through a bitstream. The flag may be signaled at the sequence, picture, or slice level. For example, if the value of the flag is 0, the four vector direction candidates exemplified in Table 8 may be used. On the other hand, if the value of the flag is 1, the eight vector direction candidates exemplified in Table 9 or Table 10 may be used.

[0372] Alternatively, based on the magnitude of the offset vector, at least one of the number or magnitude of vector direction candidates can be determined. For example, if the value of the variable DistFromMergeMV for determining the magnitude of the offset vector is equal to or less than the threshold value, the eight vector direction candidates exemplified in Table 9 or Table 10 can be used. On the other hand, if the value of the variable DistFromMergeMV is greater than the threshold value, the four vector direction candidates exemplified in Table 8 can be used.

[0373] Alternatively, at least one of the number or magnitude of vector direction candidates can be determined based on the value of the x component MVx and the value of the y component MVy of the initial motion vector. For example, if the difference between MVx and MVy or the absolute value of the difference is less than or equal to a threshold value, eight vector direction candidates exemplified in Table 9 or Table 10 may be used. On the other hand, if the difference between MVx and MVy or the absolute value of the difference is greater than the threshold value, four vector direction candidates exemplified in Table 8 may be used.

[0374] The motion vector of the current block can be derived by adding an offset vector to the initial motion vector. Equation 10 shows an example of determining the motion vector of the current block.

[0375]

[0376] In Equation 10, mvL0 represents the L0 motion vector of the current block, and mvL1 represents the L1 motion vector of the current block. mergeMVL0 represents the L0 initial motion vector of the current block (i.e., the L0 motion vector of the merge candidate), and mergeMVL1 represents the L1 initial motion vector of the current block. [0] represents the horizontal component of the motion vector, and [1] represents the vertical component of the motion vector.

[0377] Even when a coding block is divided into multiple sub-units and inter-prediction is performed for each of the sub-units, the merge offset vector coding method can be applied. Here, performing inter-prediction at the sub-unit level may include at least one of ATMVP (Advanced Temporal Motion Vector Prediction), STMVP (Spatial Temporal Motion Vector Prediction), or a triangular partitioning technique.

[0378] For example, under the ATMVP method, the initial motion vector can be derived as follows.

[0379] First, an initial shift vector can be derived using the motion vector of a merge candidate derived from a neighbor block adjacent to the coding block. Then, a shift block of a sub-block included in the coding block can be derived using the initial shift vector. Equation 11 is a diagram showing the position of the shift block.

[0380]

[0381] In the above mathematical formula 11, (xColSb, yColSb) represents the position of the top-left sample of the shift block, and (xSb, ySb) represents the position of the top-left sample of the sub-block. shVector represents the shift vector.

[0382] When a shift block is determined, the motion vector of a collocated block that is at the same location as the shift block within the collocated picture can be set as the motion vector of a sub-block. That is, the motion vector of a collocated block containing a sample at location (xColSb, yColSb) within the collocated block can be set as the motion vector of a sub-block containing a sample at location (xSb, ySb).

[0383] When the triangular partitioning technique is applied, the coding block can be divided into triangular sub-units. For example, the coding block can be divided into two sub-units by a diagonal line connecting the top-left and bottom-right corners of the coding block or a diagonal line connecting the top-right and bottom-left corners of the coding block.

[0384] Figure 30 is a diagram showing the division pattern of a coding block when a triangular partitioning technique is applied.

[0385] Movement information for each of the triangular sub-units can be specified by a merge candidate. To this end, index information representing any one of the merge candidates can be signaled for each sub-unit. For example, the index information merge_1st_idx of the first sub-unit can specify the merge candidate of the first sub-unit, and the index information merge_2nd_idx of the second sub-unit can specify the merge candidate of the second sub-unit.

[0386] The initial motion vectors of each sub-unit can be determined individually. For example, if an affine motion model is applied to a coding block, the affine vectors of the sub-blocks derived from the affine seed vectors of the coding block can be set as the initial motion vectors of the sub-blocks. The motion vector of each sub-block can be derived by adding or subtracting an offset vector to the initial motion vector.

[0387] When a merge offset vector encoding method is applied to a coding block divided into multiple sub-units, the multiple sub-units can be configured to use the same offset vector. That is, the initial motion vector of each of the multiple sub-units can be modified using the same offset vector.

[0388] Alternatively, the coding block can be divided into multiple sub-units, and the offset vector of each sub-unit can be determined individually. Accordingly, at least one offset vector of the sub-units can be set differently from the offset vector of the other sub-units.

[0389] FIG. 31 is a diagram showing an example in which the offset vectors of each sub-unit are set differently.

[0390] As shown in the example illustrated in FIG. 31, information distance_idx representing the magnitude of the offset vector and information direction_idx representing the direction of the offset vector can be encoded and signaled for each sub-unit.

[0391] Alternatively, the magnitude of the offset vector for all sub-units can be set equally, while the direction of the offset vector for each sub-unit can be set individually. For example, sub-units can be configured to share the value of distance_idx signaled at the coding level, and direction_idx can be encoded and signaled for each sub-unit.

[0392] Alternatively, the direction of the offset vector for all sub-units can be set identically, while the magnitude of the offset vector for each sub-unit can be set individually. For example, sub-units can be configured to share the value of direction_idx signaled at the coding level, and distance_idx can be encoded and signaled for each sub-unit.

[0393] The merge offset vector encoding method may be applied to only some of the multiple sub-units generated by dividing a coding block. For example, if the current block is divided into a first sub-unit and a second sub-unit, the motion vector of the first sub-unit can be set to be the same as the motion vector of the merge candidate, and the motion vector of the second sub-unit can be derived by adding an offset vector to the motion vector of the merge candidate.

[0394] Instead of signaling information to determine the offset vector, the offset vector may be derived in the decoder. Specifically, the offset vector may be derived using the average value of the horizontal gradient and the average value of the vertical gradient of the prediction samples included in the sub-block.

[0395] Here, the gradient can be derived based on the difference between the reconstructed sample corresponding to the predicted sample in the reference picture and the neighboring samples adjacent to the reconstructed sample. For example, the horizontal gradient may represent the difference between the reconstructed sample and the reconstructed samples adjacent to the left and / or right, and the vertical gradient may represent the difference between the reconstructed sample and the reconstructed samples adjacent to the top and / or bottom.

[0396] Among the merge candidates included in the merge candidate list, a merge candidate having a motion vector derived by adding or subtracting an offset vector to the motion vector of a reference merge candidate can be added to the merge candidate list. A merge candidate having a motion vector derived by adding or subtracting an offset vector to the motion vector of a reference merge candidate can be referred to as a refined merge candidate.

[0397] The residual motion information, excluding the motion vector of the refine merge candidate, can be set to be identical to that of the reference merge candidate.

[0398] Figure 32 is a diagram showing motion vector candidates that a refine merge candidate can take.

[0399] If the motion vector of the reference merge candidate is (MvLX[0], MvLX[1]), the motion vector of the refine merge candidate can be derived by adding or subtracting an offset to at least one of the x-component or y-component of the motion vector of the reference merge candidate. For example, the motion vector of the refine merge candidate can be set to (MvLX[0]+M, MvLX[1]), (MvLX[0]-M, MvLX[1]), (MvLX[0], MvLX[1]+M), or (MvLX[0], MvLX[1]-M). M represents the magnitude of the offset vector.

[0400] The reference merge candidate may be a merge candidate having a predefined index value within the merge candidate list. For example, among the merge candidates included in the merge candidate list, the merge candidate with the smallest index value (i.e., the merge candidate with an index value of 0) or the merge candidate with the largest index value may be set as the reference merge candidate. Alternatively, the inter-region merge candidate with the smallest index value or the inter-region merge candidate with the largest index value within the inter-region motion information table may be set as the reference merge candidate.

[0401] Alternatively, among the merge candidates with bidirectional movement information, the merge candidate with the smallest index value can be set as the reference merge candidate. That is, when candidate blocks are searched sequentially, the first bidirectional merge candidate found can be set as the reference merge candidate.

[0402] A base merge candidate may also be selected based on the size, shape, or whether the current block touches the boundary of a coding tree unit. For example, if the current block is square or non-square where the height is greater than the width, a merge candidate with index 0 or a merge candidate derived from a neighbor block located at the top of the current block may be set as the base merge candidate. If the current block is non-square where the width is greater than the height, a merge candidate with index 1 or a merge candidate derived from a neighbor block located to the left of the current block may be set as the base merge candidate.

[0403] Alternatively, information specifying a reference merge candidate may be signaled via a bitstream. The information may be index information specifying one of the merge candidates included in the merge candidate list.

[0404] Information indicating whether to use a refine merge candidate may be signaled via a bitstream. The information may be a 1-bit flag. If the value of the flag is 1, a refine merge candidate generated based on a reference merge candidate may be added to the merge candidate list. If the value of the flag is 0, the merge candidate list may not include a refine merge candidate.

[0405] Alternatively, if the number of merge candidates already added to the merge candidate list is less than the maximum number of merge candidates the merge candidate list can contain, a refined merge candidate may be added to the merge candidate list. Here, the merge candidates already added may include at least one of a spatial merge candidate, a temporal merge candidate, an inter-region merge candidate, or a pairwise merge candidate. For example, if the number of at least one of the spatial merge candidate, temporal merge candidate, or inter-region merge candidate included in the merge candidate list is less than or equal to a threshold value, a refined merge candidate may be added to the merge candidate list.

[0406] Alternatively, if the number of merge candidates already added to the merge candidate list is greater than or equal to a threshold value, a refined merge candidate may be used.

[0407] Depending on whether Refine Merge Candidates are used, the maximum number of merge candidates that can be included in the merge candidate list may be set differently. For example, if Refine Merge Candidates are not used, the maximum number of merge candidates that can be included is set to N, whereas if Refine Merge Candidates are used, the maximum number of merge candidates that can be included is set to N+n.

[0408] Refine merge candidates can have a larger index than the merge candidates already added to the merge candidate list. For example, Table 11 shows an example of configuring the merge candidate list.

[0409] mergeCand[0]mergeCand[1]mergeCand[2]mergeCand[3]mergeCand[4]mergeCand[5]mergeCand[6] : Refine merge candidate with motion vector (mvLx[0]+M , mvLx[1]) mergeCand[7] : Refine merge candidate with motion vector (mvLx[0]-M , mvLx[1]) mergeCand[8] : Refine merge candidate with motion vector (mvLx[0], mvLx[1]+M) mergeCand[9] : Refine merge candidate with motion vector (mvLx[0], mvLx[1]-M)

[0410] In Table 11, mergeCand[X] represents a merge candidate with index X. mvLx[0] represents the x-component motion vector of the reference merge candidate, and mvLx[1] represents the y-component motion vector of the reference merge candidate. For example, if the reference merge candidate is mergeCand[0], mvLx[0] and mvLx[1] can represent the motion vectors of mergeCand[0].

[0411] The size M of the offset vector can be defined in the encoder and decoder. For example, the size M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0412] Alternatively, information for determining the offset vector may be signaled via a bitstream. The information may be signaled at the sequence, picture, slice, or block level. For example, the offset vector may be determined using at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector.

[0413] As shown in the example in Table 11, at least one refined merge candidate derived based on a reference merge candidate can be added to the merge candidate list. If there is a merge candidate among the added merge candidates that has the same movement information as the refined merge candidate, the refined merge candidate may not be added to the merge candidate list. For example, if a refined merge candidate derived based on the reference merge candidate mergeCand[0] is identical to any one of mergeCand[1] to mergeCand[5], the refined merge candidate may not be added to the merge candidate list.

[0414] Alternatively, if there exists a merge candidate with motion information identical to the refine merge candidate, the offset vector may be changed, or the merge candidate with motion information identical to the refine merge candidate may be re-derived by resetting it as the refine merge candidate. For example, if the motion information of the refine merge candidate mergeCand[6] derived based on the reference merge candidate mergeCand[0] is identical to that of the merge candidate mergeCand[2], the motion vector of the refine merge candidate mergeCand[6] may be changed to a value obtained by adding or subtracting the offset vector from the motion vector of the merge candidate mergeCand[2]. For example, the motion vector of mergeCand[6] may be changed from (mergeCand[0]_mxLx[0]+M, mergeCand[0]_mvLx[1]) to (mergeCand[2]_mxLx[0]+M, mergeCand[2]_mvLx[1]). Here, mergeCand[X]_mvLx represents the motion vector of the merge candidate with index X.

[0415]

[0416] As another example, an offset vector can be determined using a merge refinement offset list containing at least one merge offset candidate. If a merge candidate identified by the index information of the current block is a reference merge candidate, an offset vector can be determined using the merge refinement offset list. Then, the motion vector of the current block can be derived by adding or subtracting the offset vector to the motion vector of the merge candidate. The reference merge candidate may be a merge candidate having a predefined index value within the merge candidate list. For example, among the merge candidates included in the merge candidate list, the merge candidate with the smallest index value (i.e., a merge candidate with an index value of 0) or the merge candidate with the largest index value may be set as the reference merge candidate. Alternatively, the inter-region merge candidate with the smallest index value or the inter-region merge candidate with the largest index value within the inter-region motion information table may be set as the reference merge candidate.

[0417] FIG. 33 is a diagram illustrating the configuration of a merge refinement offset list.

[0418] In Fig. 33, the reference merge candidate is assumed to be a merge candidate with an index of 6.

[0419] If the index of a merge candidate specified by the index information merge_idx representing one of the merge candidates is not 6, the movement vector of the merge candidate can be set to the movement vector of the current block.

[0420] On the other hand, if the index of the merge candidate specified by the index information merge_idx is 6, the offset vector can be derived using the merge refinement offset list. The index information MrgOffset_idx, which specifies one of the merge offset candidates included in the merge refinement offset list, can be signaled through the bitstream.

[0421] Once the offset vector is specified, the motion vector of the current block can be derived by adding or subtracting the offset vector from the motion vector of the reference merge candidate.

[0422] The merge refinement offset list may include at least one merge offset candidate. For example, the number of merge offset candidates included in the merge refinement offset list may be 4, 8, or 16.

[0423] Figures 34 and 35 are diagrams showing offset vectors specified by merge offset candidates.

[0424] Figure 34 shows an example where the number of merge offset candidates is 8, and Figure 35 shows an example where the number of merge offset candidates is 16.

[0425] As in the example illustrated in FIG. 34 (a), the offset vectors represented by the merge offset candidates may be set so that the absolute value of the horizontal motion vector and / or the absolute value of the vertical motion vector has a fixed value. Alternatively, as in the example illustrated in FIG. 35, the offset vectors represented by the merge offset candidates whose index is smaller than the threshold value may be set so that the absolute value of the horizontal motion vector and / or the absolute value of the vertical motion vector has a first value, and the offset vectors represented by the other merge offset candidates may be set so that the absolute value of the horizontal motion vector and / or the absolute value of the vertical motion vector has a second value.

[0426] Alternatively, as shown in (b) of FIG. 34, the offset vectors represented by the merge offset candidates can be set so that the sum of the absolute value of the horizontal motion vector and the absolute value of the vertical motion vector is a fixed value.

[0427] Multiple reference merge candidates may be set. For example, the two merge candidates with the smallest indices among the merge candidates included in the merge candidate list may be set as reference merge candidates. Accordingly, if the index of a merge candidate specified by the index information merge_idx is 0 or 1, an offset vector can be derived using the merge refinement offset list. Alternatively, the merge candidate with the smallest index among the merge candidates included in the merge candidate list and the merge candidate with the largest index among the merge candidates included in the inter-region merge candidate list may be set as reference merge candidates.

[0428]

[0429] In motion vector prediction mode, the motion vector of the current block can be derived by adding a motion difference vector to the motion prediction vector. The motion prediction vector of the current block can be determined based on a motion vector prediction candidate list containing at least one motion prediction vector candidate. For example, any one of the motion prediction vector candidates can be set as the motion prediction vector of the current block.

[0430] Motion vector prediction candidates can be derived based on at least one of the spatial neighbor blocks of the current block or the temporal neighbor blocks of the current block.

[0431] Figure 36 is a diagram showing candidate blocks used to derive motion vector prediction candidates.

[0432] Spatial neighbor blocks may include top neighbor blocks located above the current block and left neighbor blocks located to the left of the current block. The top neighbor blocks may include at least one of block B0 containing a sample at position (xCb+CbW, yCb-1), block B1 containing a sample at position (xCb+CbW-1, yCb-1), block B2 containing a sample at position (xCb-1, yCb-1), or block B3 containing a sample at position (xCb, yCb-1). Here, (xCb, yCb) represents the position of the top-left sample of the current block, and CbW represents the width of the current block. The left neighbor blocks may include at least one of block A0 containing a sample at position (xCb-1, yCb+CbH), block A1 containing a sample at position (xCb-1, yCb+CbH-1), or block A2 containing a sample at position (xCb-1, yCb). Here, CbH represents the height of the current block.

[0433] The temporal neighbor block may include at least one of block C0, which contains a central sample of a block having the same location and size as the current block within the collocated block, or block C1, which contains a sample adjacent to the upper right corner of said block.

[0434] The maximum number of motion vector prediction candidates that can be included in the motion vector prediction candidate list is 2. The derivation order of the motion vector prediction candidates is as follows.

[0435] 1. If at least one of left neighbor block A0 or left neighbor block A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0436] 2. If at least one of the top neighbor block B0, top neighbor block B1, or top neighbor block B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0437] 3. If temporal neighbor blocks are available, set the temporal motion vector as a motion vector prediction candidate.

[0438] 4. Set the zero motion vector as the motion vector prediction candidate.

[0439] Alternatively, if the number of motion vector prediction candidates derived according to the order of 1-3 is less than 2, motion vectors included in the inter-domain motion information table may be set as motion vector prediction candidates. If the inter-domain motion information table is available, motion vector prediction candidates may be derived according to the following order.

[0440] 1. If at least one of left neighbor block A0 or left neighbor block A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0441] 2. If at least one of the top neighbor block B0, top neighbor block B1, or top neighbor block B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0442] 3. If temporal neighbor blocks are available, set the temporal motion vector as a motion vector prediction candidate.

[0443] 4. Set the motion vectors included in the inter-domain motion information table as motion vector prediction candidates.

[0444] 5. Set the zero motion vector as the motion vector prediction candidate.

[0445] Motion vector prediction candidates having motion vectors derived by adding or subtracting an offset vector to the motion vector of a reference motion vector prediction candidate may also be added to the motion vector prediction candidate list. Motion vector prediction candidates having motion vectors derived by adding or subtracting an offset vector to the motion vector of a reference motion vector prediction candidate may be referred to as refined motion vector prediction candidates.

[0446] Figure 37 is a diagram showing motion vector candidates that can be set as refined motion vector prediction candidates.

[0447] If the motion vector of the reference motion vector prediction candidate is (MvpLX[0], MvpLX[1]), the motion vector of the refined motion vector prediction candidate can be derived by adding or subtracting an offset to at least one of the x-component or y-component of the motion vector of the reference motion vector prediction candidate. For example, the motion vector of the refined motion vector prediction candidate can be set to (MvpLX[0]+M, MvpLX[1]), (MvpLX[0]-M, MvpLX[1]), (MvpLX[0], MvpLX[1]+M), or (MvpLX[0], MvpLX[1]-M). M represents the magnitude of the offset vector.

[0448] The size M of the offset vector can be defined in the encoder and decoder. For example, the size M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0449] Alternatively, information for determining the offset vector may be signaled via a bitstream. The information may be signaled at the sequence, picture, slice, or block level. For example, the offset vector may be determined using at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector.

[0450] A reference motion vector prediction candidate may be a motion vector prediction candidate having a predefined index value within the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, the motion vector prediction candidate with the lowest index value of 0 or the motion vector prediction candidate with the lowest index value of 1 may be set as the reference motion vector prediction candidate.

[0451] As another example, an offset vector can be determined using a merge refinement offset list containing at least one prediction vector offset candidate. If the motion vector prediction candidate specified by the index information of the current block is the reference motion vector prediction candidate, the offset vector can be determined using the prediction vector refinement offset list. Then, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector to the motion vector of the motion vector prediction candidate. The reference motion vector prediction candidate may be a motion vector prediction candidate having a predefined index value within the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, the motion vector prediction candidate with the smallest index value or the motion vector prediction candidate with the largest index value may be set as the reference motion vector prediction candidate.

[0452] When calculating an offset vector using a prediction vector offset refinement list, the maximum number of prediction vector candidates that can be included in the prediction vector candidate list may be set to a value greater than 2.

[0453] Figure 38 is a diagram illustrating the configuration of a prediction vector refinement offset list.

[0454] In Fig. 38, the reference prediction vector candidate is assumed to be a prediction vector candidate with an index of 2.

[0455] If the index of a prediction vector candidate specified by AMVPcand_idx, which represents one of the prediction vector candidates, is not 2, the motion vector of the prediction vector candidate can be set as the motion prediction vector of the current block.

[0456] On the other hand, if the index of the prediction vector candidate specified by the index information AMVPcand_idx is 2, the offset vector can be derived using the prediction vector refinement offset list. The index information AMVPOffset_idx, which specifies one of the prediction vector offset candidates included in the prediction vector refinement offset list, can be signaled through the bitstream.

[0457] Once the offset vector is specified, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector to the motion vector of the reference prediction vector candidate.

[0458] Even when a coding block is encoded based on an affine motion model, a motion vector refinement technique may be used. For example, when an affine motion vector prediction mode is applied, the affine seed vectors of the coding block can be derived by adding an affine seed difference vector to the affine seed prediction vector. Here, the affine seed prediction vector can be derived based on the affine seed vectors of the spatial neighbor blocks or temporal neighbor blocks of the coding block. The affine seed difference vector can be determined based on information signaled from a bitstream. In this case, the same affine seed difference vector can be applied to all control points. Alternatively, information for determining the affine seed vector can be signaled per control point.

[0459] Based on the affine seed vectors of the coding blocks, when an affine vector for a sub-block is derived, the affine vector can be set as an initial motion vector, and then an offset vector can be derived. The motion vector of each sub-block can be derived by adding or subtracting the offset vector to the initial motion vector.

[0460] Instead of signaling information to determine the offset vector, the offset vector may be derived in the decoder. Specifically, the offset vector may be derived using the average value of the horizontal gradient and the average value of the vertical gradient of the prediction samples included in the sub-block.

[0461]

[0462] Intra prediction is the prediction of the current block using reconstructed samples that have completed encoding / decoding around the current block. In this case, for the intra prediction of the current block, reconstructed samples prior to the application of the in-loop filter may be used.

[0463] Intra prediction techniques include matrix-based intra prediction and general intra prediction considering the directionality with surrounding reconstructed samples. Information indicating the intra prediction technique of the current block may be signaled via a bitstream. The information may be a 1-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, shape of the current block or the intra prediction technique of neighboring blocks. For example, if the current block exists across a picture boundary, matrix-based intra prediction may not be applied to the current block.

[0464] Matrix-based intra prediction is a method for obtaining a predicted block of the current block based on matrix multiplication between a matrix previously stored in an encoder and a decoder and a reconstructed sample surrounding the current block. Information for specifying one of a plurality of previously stored matrices may be signaled via a bitstream. The decoder may determine a matrix for intra prediction of the current block based on the said information and the size of the current block.

[0465] General intra prediction is a method of obtaining a predicted block for the current block based on a non-directional intra prediction mode or a directional intra prediction mode.

[0466]

[0467] A residual image can be derived by differencing the predicted image from the original image. In this case, when the residual image is converted to the frequency domain, the subjective image quality does not significantly degrade even if high-frequency components among the frequency components are removed. Accordingly, if the values ​​of high-frequency components are transformed to be small or set to zero, it is possible to increase compression efficiency without significant visual distortion. Reflecting these characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. This transformation can be performed using transformation techniques such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST).

[0468] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the transformation based on DCT or DST can be defined as the first transformation, and the transformation of the block to which the first transformation has been applied can be defined as the second transformation.

[0469] The first transformation can be performed using any one of a plurality of transformation core candidates. For example, the first transformation can be performed using any one of DCT2, DCT8, or DCT7.

[0470] Different conversion cores may be used for the horizontal and vertical directions. Information indicating the combination of the horizontal and vertical conversion cores may also be signaled via a bitstream.

[0471] The units of execution for the first transformation and the second transformation may be different. For example, the first transformation may be performed on an 8x8 block, and the second transformation may be performed on a 4x4 sub-block among the transformed 8x8 blocks. In this case, the transformation coefficients of the remaining regions where the second transformation is not performed may be set to 0.

[0472] Alternatively, a first transformation may be performed on a 4x4 block, and a second transformation may be performed on an 8x8 area containing the transformed 4x4 block.

[0473] Information indicating whether the second conversion is performed can be signaled through a bitstream.

[0474] In the decoder, the inverse of the second transformation (second inverse transformation) is performed, and the inverse of the first transformation (first inverse transformation) can be performed on the result thereof. As a result of performing the second inverse transformation and the first inverse transformation, residual signals for the current block can be obtained.

[0475] Quantization is intended to reduce the energy of a block, and the quantization process includes the process of dividing a transformation coefficient by a specific constant value. The constant value can be derived by a quantization parameter, and the quantization parameter can be defined as a value between 1 and 63.

[0476] When the encoder performs transform and quantization, the decoder can obtain a residual block through inverse quantization and inverse transform. The decoder can obtain a reconstructed block for the current block by adding the prediction block and the residual block.

[0477]

[0478] When the restored block of the current block is obtained, the loss of information occurring during the quantization and encoding process can be reduced through in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

[0479]

[0480] Applying the embodiments described with reference to the decoding process or the encoding process to the encoding process or the decoding process is included within the scope of the present invention. Changing the embodiments described in a predetermined order to a different order from that described is also included within the scope of the present invention.

[0481] The above-described embodiments are described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention and may be performed simultaneously or in a different order as necessary. Additionally, each component constituting the block diagram in the above-described embodiments (e.g., units, modules, etc.) may be implemented as a hardware device or software, or a plurality of components may be combined to be implemented as a single hardware device or software. The above-described embodiments may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. The above hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0482] The present invention can be applied to an electronic device that encodes / decodes images.

Claims

1. A step of determining whether a merge offset vector encoding method is applied to the current block; A step of generating a merge candidate list for the current block; A step of specifying a merge candidate for the current block based on the merge candidate list; and A step of deriving a motion vector for the current block based on the above merge candidate, A video decoding method, characterized in that when the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate, and when the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bitstream indicating one of the merge candidates, and when the maximum number is 1, the merge candidate is determined without decoding the index information.

2. In paragraph 1, An image decoding method, characterized in that the size of the above offset vector is determined based on first index information that specifies one of the vector size candidates.

3. In paragraph 2, An image decoding method, characterized in that at least one of the maximum or minimum values ​​of the above vector size candidates is set differently depending on the value of a flag indicating the range of the above vector size candidates.

4. In paragraph 3, A video decoding method, characterized in that the above flag is signaled at the picture level.

5. In paragraph 2, An image decoding method, characterized in that at least one of the maximum or minimum values ​​of the above vector size candidates is set differently according to the motion vector precision for the current block.

6. In paragraph 2, An image decoding method, characterized in that the size of the offset vector is obtained by applying a bit shifting operation to a value indicated by a vector size candidate specified by the first index information.

7. In paragraph 2, An image decoding method, characterized in that the direction of the above offset vector is determined based on second index information that specifies one of the vector direction candidates.

8. A step of determining whether the merge offset vector encoding method is applied to the current block; A step of generating a merge candidate list for the current block; A step of specifying a merge candidate for the current block based on the merge candidate list; and A step of deriving a motion vector for the current block based on the above merge candidate, A video encoding method characterized in that, when the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate, and when the maximum number of merge candidates that the merge candidate list can include is plural, index information indicating the merge candidate of the current block among the merge candidates is encoded, and when the maximum number is 1, encoding of the index information is omitted.

9. In paragraph 8, An image encoding method further comprising a step of encoding first index information for specifying a vector size candidate representing the size of the offset vector among a plurality of vector size candidates.

10. In paragraph 9, A video encoding method, further comprising a step of encoding a flag indicating a range of the vector size candidates, characterized in that at least one of the maximum value or minimum value of the vector size candidates is set differently depending on the value of the flag.

11. In paragraph 10, A video encoding method, characterized in that the above flag is encoded at the picture level.

12. In paragraph 9, A video encoding method, characterized in that at least one of the maximum or minimum values ​​of the above vector size candidates is set differently according to the motion vector precision for the current block.

13. In paragraph 9, An image encoding method, characterized in that the above vector size candidate has a value derived by applying a bit shifting operation to the size of the above offset vector.

14. In paragraph 9, An image encoding method further comprising a step of encoding second index information for specifying a vector direction candidate indicating the direction of the offset vector among a plurality of vector direction candidates.

15. A step of determining whether the merge offset vector encoding method is applied to the current block; An inter prediction unit is included that generates a merge candidate list for the current block, specifies a merge candidate for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidate. An image decoding device characterized in that, when the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate, and when the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bitstream indicating one of the merge candidates, and when the maximum number is 1, the merge candidate is determined without decoding the index information.

Images