Image encoding and decoding method and digital storage medium

Abstract

The invention provides an image encoding and decoding method and a digital storage medium. The image decoding method includes: constructing a merge candidate list of a current block; deriving motion information of the current block including at least one of a motion vector or a reference picture index based on the merge index of the current block and the merge candidate list; and performing inter prediction of the current block based on the motion information of the current block. Wherein a current picture including a current block is divided into a plurality of coding tree units, the current block is obtained by hierarchically dividing the coding tree units, and a motion vector of the current block is adaptively modified using a motion vector difference based on flag information signaled from a bitstream, the flag information is equal to 0 (equal to 1) indicating that the motion vector of the current block is not modified by using the motion vector difference value, the merge candidate list of the current block comprises M merge candidates, and based on the flag information is equal to 0 (equal to 1), motion information, Nlt of the current block is obtained from one of the M (N) merge candidates; m.

Description

[0001] This application is a divisional application of invention patent application No. 201980086601.X, with the application date of December 27, 2019, international application number PCT / KR2019 / 018644, and entered the Chinese national phase on June 25, 2021, with the invention name “Image encoding / decoding method and device”. Technical Field

[0002] The present disclosure relates to methods and apparatus for processing video signals. Background Art

[0003] As the market demand for high-resolution video increases, technology that can effectively compress high-resolution images is needed. In response to this market demand, the ISO / IEC Moving Picture Expert Group (MPEG) and the ITU-T Video Coding Expert Group (VCEG) jointly formed the Joint Collaborative Team on Video Coding (JCT-VC), developed the High Efficiency Video Coding (HEVC) video compression standard in January 2013, and actively conducts research and development on the next generation compression standard.

[0004] Video compression mainly consists of intra-frame prediction, inter-frame prediction, transform, quantization, entropy coding, and loop filtering. On the other hand, as the demand for high-resolution images increases, the demand for stereoscopic image content as a new image service also increases. Video compression technology for efficiently providing high-resolution and ultra-high-resolution stereoscopic image content has been discussed. Summary of the invention

[0005] Technical issues

[0006] The purpose of the present disclosure is to provide a method and apparatus for adaptively partitioning a picture.

[0007] The purpose of the present disclosure is to provide an intra-frame prediction method and apparatus.

[0008] The purpose of the present disclosure is to provide an inter-frame prediction method and apparatus.

[0009] An object of the present disclosure is to provide an inter-frame prediction method and apparatus using a motion vector difference (MVD) based merge mode.

[0010] Technical Solutions

[0011] The image encoding / decoding method and apparatus according to the present disclosure may: construct a merge candidate list of a current block; derive motion information of the current block based on a merge index of the current block and the merge candidate list; derive a motion vector difference of the current block; modify the motion vector of the current block by using the motion vector difference; and perform motion compensation on the current block by using the modified motion vector.

[0012] In the image encoding / decoding method and apparatus according to the present disclosure, the motion information may include at least one of a motion vector or a reference picture index.

[0013] In the image encoding / decoding method and apparatus according to the present disclosure, the merge candidate list may include at least one of a spatial merge candidate or a temporal merge candidate of the current block.

[0014] In the image encoding / decoding method and apparatus according to the present disclosure, motion information of a previous block belonging to a motion information list of a current block may be additionally added to a merge candidate list, and the previous block may be one or more blocks decoded before the current block.

[0015] In the image encoding / decoding method and apparatus according to the present disclosure, deriving a motion vector difference may be selectively performed based on a predetermined flag, and the flag may indicate whether the motion vector difference is used in a merge mode.

[0016] In the image encoding / decoding method and apparatus according to the present disclosure, the maximum number of merging candidates available for the current block may be adaptively determined according to a flag.

[0017] In the image encoding / decoding method and apparatus according to the present disclosure, a motion vector difference may be derived based on a merged offset vector, and the merged offset vector may be determined based on the length and direction of the merged offset vector.

[0018] In the image encoding / decoding method and apparatus according to the present disclosure, the length of the merged offset vector may be determined by considering at least one of the accuracy of the motion vector of the current block or the encoding distance index.

[0019] In the image encoding / decoding method and apparatus according to the present disclosure, the direction of the merged offset vector may be determined based on the encoding direction index, and the direction may be determined as any one of the left, right, up, or down directions.

[0020] In the image encoding / decoding method and device according to the contents of the present disclosure, the motion vector difference can be obtained by applying a predetermined scaling factor to the merged offset vector, and the scaling factor can be determined based on the POC difference between the current picture to which the current block belongs and the reference picture of the current block.

[0021] A digital storage medium for storing a video stream according to the present disclosure may record / store a video decoding program, which: constructs a merge candidate list of a current block; derives motion information of the current block based on a merge index of the current block and the merge candidate list; derives a motion vector difference of the current block; modifies the motion vector of the current block by using the motion vector difference; and performs motion compensation on the current block by using the modified motion vector.

[0022] An aspect of the present disclosure provides an image decoding method, comprising: constructing a merge candidate list of a current block; deriving motion information of the current block based on a merge index of the current block and the merge candidate list, the motion information comprising at least one of a motion vector or a reference picture index; and performing inter-frame prediction of the current block based on the motion information of the current block, wherein a current picture including the current block is divided into a plurality of coding tree units, wherein the current block is obtained by hierarchically dividing the coding tree units, wherein the motion vector of the current block is adaptively modified using a motion vector difference based on flag information signaled from a bitstream, wherein, The flag information being equal to 0 indicates that the motion vector difference is not used to modify the motion vector of the current block, and the flag information being equal to 1 indicates that the motion vector difference is used to modify the motion vector of the current block, wherein the merge candidate list of the current block includes M merge candidates, wherein, based on the flag information being equal to 0, the motion information of the current block is derived from one of the M merge candidates in the merge candidate list, wherein, based on the flag information being equal to 1, the motion information of the current block is derived from N merge candidates in the merge candidate list instead of one of the M merge candidates, and wherein the value of N is less than the value of M.

[0023] Another aspect of the present disclosure provides an image encoding method, comprising: constructing a merge candidate list of a current block; deriving motion information of the current block based on the merge candidate list, the motion information comprising at least one of a motion vector or a reference picture index; and performing inter-frame prediction of the current block based on the motion information of the current block, wherein a current picture including the current block is divided into a plurality of coding tree units, wherein the current block is obtained by hierarchically dividing the coding tree units, wherein flag information indicating whether the motion vector of the current block is modified using a motion vector difference is encoded in a bitstream, wherein the flag information is equal to 0 indicates that the motion vector of the current block is not modified by using the motion vector difference, and the flag information is equal to 1, indicating that the motion vector of the current block is modified by using the motion vector difference, wherein the merge candidate list of the current block includes M merge candidates, wherein, based on the flag information being equal to 0, the motion information of the current block is derived from one of the M merge candidates in the merge candidate list, wherein, based on the flag information being equal to 1, the motion information of the current block is derived from N merge candidates in the merge candidate list instead of one of the M merge candidates, and wherein the value of N is less than the value of M.

[0024] Another aspect of the present disclosure provides a digital storage medium for storing a bit stream generated by an image encoding method, the image encoding method comprising: constructing a merge candidate list of a current block; deriving motion information of the current block based on the merge candidate list, wherein the motion information comprises at least one of a motion vector or a reference picture index; and performing inter-frame prediction of the current block based on the motion information of the current block, wherein a current picture including the current block is divided into a plurality of coding tree units, wherein the current block is obtained by hierarchically dividing the coding tree units, wherein flag information indicating whether the motion vector of the current block is modified using a motion vector difference is encoded in a bit stream. In a special stream, wherein the flag information is equal to 0 to indicate that the motion vector difference is not used to modify the motion vector of the current block, and the flag information is equal to 1 to indicate that the motion vector difference is used to modify the motion vector of the current block, wherein the merge candidate list of the current block includes M merge candidates, wherein based on the flag information being equal to 0, the motion information of the current block is derived from one of the M merge candidates in the merge candidate list, wherein based on the flag information being equal to 1, the motion information of the current block is derived from N merge candidates in the merge candidate list instead of one of the M merge candidates, and wherein the value of N is less than the value of M.

[0025] Technical Effects

[0026] The present disclosure may improve encoding / decoding efficiency of a video signal by dividing a picture in predetermined units and performing encoding / decoding.

[0027] The present disclosure may improve the coding efficiency of intra prediction by using subdivided directional modes and / or selectable pixel rows.

[0028] The present disclosure may improve the encoding efficiency of inter-frame prediction by using an affine mode or inter-frame region motion information.

[0029] The present disclosure may improve the accuracy and coding efficiency of motion vectors by using a merge mode based on motion vector difference (MVD). BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a block diagram illustrating an image encoding apparatus according to the present disclosure.

[0031] Figure 2 is a block diagram illustrating an image decoding apparatus according to the present disclosure.

[0032] Figures 3 to 5 A method of dividing a picture into a plurality of blocks is shown as an embodiment to which the present disclosure is applied.

[0033] Figures 6 to 8 An affine inter prediction method is shown as an embodiment to which the present disclosure is applied.

[0034] Fig. 9 An image encoding / decoding method using a motion vector difference (MVD)-based merge mode is shown as an embodiment to which the present disclosure is applied.

[0035] Figures 10 to 14 A previous block that may be used as a merge candidate as an embodiment to which the present disclosure is applied is shown.

[0036] Figures 15 to 19 A method of deriving a merging offset vector for a merging mode is shown as an embodiment of applying the present disclosure.

[0037] Figure 20 to Figure 24 An intra prediction method is shown as an embodiment to which the present disclosure is applied.

[0038] Figure 25 to Figure 28 A wide-angle based intra prediction method is shown as an embodiment to which the present disclosure is applied.

[0039] Fig.29 and Fig.30 A method of applying a loop filter to a reconstructed block is shown as an embodiment to which the present disclosure is applied.

[0040] Best Mode for Carrying Out the Invention

[0041] The image encoding / decoding method and apparatus according to the present disclosure may: construct a merge candidate list of a current block; derive motion information of the current block based on a merge index of the current block and the merge candidate list; derive a motion vector difference of the current block; modify the motion vector of the current block by using the motion vector difference; and perform motion compensation on the current block by using the modified motion vector.

[0042] In the image encoding / decoding method and apparatus according to the present disclosure, the motion information may include at least one of a motion vector or a reference picture index.

[0043] In the image encoding / decoding method and apparatus according to the present disclosure, the merge candidate list may include at least one of a spatial merge candidate or a temporal merge candidate of the current block.

[0044] In the image encoding / decoding method and apparatus according to the present disclosure, motion information of a previous block belonging to a motion information list of a current block may be additionally added to a merge candidate list, and the previous block may be one or more blocks decoded before the current block.

[0045] In the image encoding / decoding method and apparatus according to the present disclosure, deriving a motion vector difference may be selectively performed based on a predetermined flag, and the flag may indicate whether the motion vector difference is used in a merge mode.

[0046] In the image encoding / decoding method and apparatus according to the present disclosure, the maximum number of merging candidates available for the current block may be adaptively determined according to a flag.

[0047] In the image encoding / decoding method and apparatus according to the present disclosure, a motion vector difference may be derived based on a merged offset vector, and the merged offset vector may be determined based on the length and direction of the merged offset vector.

[0048] In the image encoding / decoding method and apparatus according to the present disclosure, the length of the merged offset vector may be determined by considering at least one of the accuracy of the motion vector of the current block or the encoding distance index.

[0049] In the image encoding / decoding method and apparatus according to the present disclosure, the direction of the merged offset vector may be determined based on the encoding direction index, and the direction may be determined as any one of the left, right, up, or down directions.

[0050] In the image encoding / decoding method and device according to the contents of the present disclosure, the motion vector difference can be obtained by applying a predetermined scaling factor to the merged offset vector, and the scaling factor can be determined based on the POC difference between the current picture to which the current block belongs and the reference picture of the current block.

[0051] A digital storage medium for storing a video stream according to the present disclosure may record / store a video decoding program, which: constructs a merge candidate list of a current block; derives motion information of the current block based on a merge index of the current block and the merge candidate list; derives a motion vector difference of the current block; modifies the motion vector of the current block by using the motion vector difference; and performs motion compensation on the current block by using the modified motion vector.

[0052] Embodiments of the present invention

[0053] With reference to the drawings in this specification, embodiments of the present disclosure are described in detail so that a person skilled in the art can easily implement the present disclosure. However, the present disclosure can be implemented in various forms and is not limited to the embodiments described herein. Also, portions not relevant to the description are omitted, and similar reference numerals are assigned to similar portions throughout the specification to clearly describe the present disclosure in the drawings.

[0054] In this specification, when a part is referred to as being “connected to” another part, it includes the case of being electrically connected with another element interposed therebetween as well as the case of being directly connected.

[0055] In addition, in the present specification, unless otherwise stated, when a part is referred to as “comprising” components, it means that other components may be additionally included without excluding the other components.

[0056] In addition, terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from other components.

[0057] In addition, in the embodiments of the apparatus and method described in this specification, some configurations of the apparatus or some steps of the method may be omitted. In addition, the order of some configurations of the apparatus or some steps of the method may be changed. In addition, another configuration may be inserted into some configurations of the apparatus, or another step may be inserted into some steps of the method.

[0058] In addition, some configurations or some steps in the first embodiment of the present disclosure may be added to the second embodiment of the present disclosure, or may be replaced with some configurations or some steps in the second embodiment.

[0059] In addition, when the construction units shown in the embodiments of the present disclosure are independently displayed to represent different characteristic functions, this does not mean that each construction unit is configured in a separate hardware or software construction unit. In other words, for ease of description, each construction unit can be described by being listed as a respective construction unit, at least two construction units in each construction unit can be combined to configure a construction unit, or a construction unit can be divided into multiple construction units to perform functions. Without departing from the essence of the present disclosure, such integrated implementation and separate implementation in each construction unit are also included in the scope of the present disclosure.

[0060] In this specification, a block may be variously expressed as a unit, a region, a unit, a partition, etc., and a sample may be variously expressed as a pixel, a picture element, a pixel, etc.

[0061] Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in more detail. In describing the present disclosure, repeated descriptions of the same components are omitted.

[0062] Figure 1 is a block diagram illustrating an image encoding apparatus according to the present disclosure.

[0063] Reference Figure 1 , a conventional image encoding device 100 may include: a picture division unit 110, prediction units 120, 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150 and a memory 155.

[0064] The picture division unit 110 may divide the input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). Hereinafter, in an embodiment of the present disclosure, a coding unit may be used as a unit for performing encoding, and may be used as a unit for performing decoding.

[0065] The prediction unit may be divided into at least one square shape or rectangular shape, etc. having the same size within one coding unit, and the prediction unit may be divided so that any one of the prediction units divided in one coding unit will have a shape and / or size different from another prediction unit. When a prediction unit for performing intra-frame prediction is generated based on a coding unit, if it is not a minimum coding unit, intra-frame prediction may be performed without dividing the prediction unit into a plurality of prediction units N×N.

[0066] The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. Whether inter prediction or intra prediction is performed on the prediction unit may be determined, and specific information according to each prediction method (e.g., intra prediction mode, motion vector, reference picture, etc.) may be determined. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit 165 and sent to the decoder.

[0067] The inter-frame prediction unit 120 may predict a prediction unit based on information of at least one picture in a previous picture or a subsequent picture of the current picture, and in some cases may predict a prediction unit based on information of some encoded regions in the current picture. The inter-frame prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0068] In the reference picture interpolation unit, reference picture information may be provided from the memory 155, and pixel information equal to or less than an integer pixel may be generated in the reference picture. For luma pixels, an 8-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 4 pixels. For chrominance signals, a 4-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 8 pixels.

[0069] The motion prediction unit may perform motion prediction based on a reference picture interpolated by a reference picture interpolation unit. As a method for calculating a motion vector, various methods such as a full search based block matching algorithm (FBMA), a three-step search (TSS) algorithm, a new three-step search (NTS) algorithm, etc. may be used. Based on the interpolated pixel, the motion vector may have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel. In the motion prediction unit, the current prediction unit may be predicted by making the motion prediction method different. For the motion prediction method, various methods such as a skip mode, a merge mode, an advanced motion vector prediction (AMVP) mode, an intra-block copy mode, an affine mode, etc. may be used.

[0070] The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block and pixel information in the current picture. When the reference pixel is a pixel for performing inter-frame prediction because the surrounding blocks in the current prediction unit are blocks for performing inter-frame prediction, the reference pixel included in the block for performing inter-frame prediction may be used by replacing it with reference pixel information of the block for performing surrounding intra-frame prediction. In other words, when the reference pixel is unavailable, the unavailable reference pixel information may be used by replacing it with at least one reference pixel among the available reference pixels.

[0071] In addition, a residual block including residual value information which is a difference value between a prediction unit predicted based on the prediction unit generated in the prediction units 120 and 125 and an original block in the prediction unit may be generated. The generated residual block may be input to the transform unit 130.

[0072] In the transform unit 130, the original block and the residual block including the residual value information in the prediction unit generated in the prediction units 120 and 125 may be transformed by using a transform method such as discrete cosine transform (DCT), discrete sine transform (DCT), KLT. Whether to apply DCT, DST, or KLT to transform the residual block may be determined based on intra prediction mode information in the prediction unit used to generate the residual block.

[0073] The quantization unit 135 may quantize the value transformed into the frequency domain in the transformation unit 130. The quantization coefficient may be changed according to the block or according to the image importance. The value calculated in the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0074] The rearrangement unit 160 may perform rearrangement of coefficient values ​​on the quantized residual value.

[0075] The rearrangement unit 160 can change the coefficients of the two-dimensional block form into a one-dimensional vector form by a coefficient scanning method. For example, in the rearrangement unit 160, the coefficients from the DC coefficient to the high frequency domain can be scanned by a zigzag scanning method, and can be changed into a one-dimensional vector form. According to the size of the transform unit and the intra-frame prediction mode, the zigzag scanning can be replaced by a vertical scanning of the coefficients of the two-dimensional block form by column scanning or a horizontal scanning of the coefficients of the two-dimensional block form by row scanning. In other words, it can be determined which scanning method to use among the zigzag scanning, the vertical scanning and the horizontal scanning according to the size of the transform unit and the intra-frame prediction mode.

[0076] The entropy encoding unit 165 may perform entropy encoding based on the value calculated by the rearrangement unit 160. For example, the entropy encoding may use various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). In this regard, the entropy encoding unit 165 may encode residual value coefficient information in the coding unit from the rearrangement unit 160 and the prediction units 120, 125. In addition, according to the present disclosure, information indicating that motion information is derived and used from the decoder side and information about a method for deriving motion information may be signaled and transmitted.

[0077] In the inverse quantization unit 140 and the inverse transform unit 145, the value quantized in the quantization unit 135 is dequantized, and the value transformed in the transform unit 130 is inversely transformed. The residual value generated in the inverse quantization unit 140 and the inverse transform unit 145 can generate a reconstructed block by combining with the prediction unit predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125.

[0078] The filter unit (150) may include at least one of a deblocking filter, an offset modification unit, and an adaptive loop filter (ALF). The deblocking filter may remove block distortion generated by boundaries between blocks in a reconstructed image. For an image to be deblocked, the offset modification unit may modify the offset from the original image in units of pixels. The following method may be used to perform offset modification on a specific image: a method of dividing pixels included in an image into a certain number of regions, determining regions to be offset, and applying the offset to corresponding regions; or a method of applying the offset by considering edge information of each pixel. The ALF (Adaptive Loop Filter) may be performed based on a comparison value of a filtered reconstructed image with an original image. The pixels included in the image may be divided into predetermined groups, a filter to be applied to the corresponding group may be determined, and filtering may be performed differently by group.

[0079] The memory 155 may store the reconstructed block or picture calculated in the filter unit 150 , and may provide the stored reconstructed block or picture to the prediction units 120 and 125 when performing inter prediction.

[0080] Figure 2 is a block diagram illustrating an image decoding apparatus according to the present disclosure.

[0081] Reference Figure 2 , the image decoder 200 may include: an entropy decoding unit 210 , a rearrangement unit 215 , an inverse quantization unit 220 , an inverse transform unit 225 , prediction units 230 and 235 , a filter unit 240 , and a memory 245 .

[0082] When an image bit stream is input from an image encoder, the input bit stream may be decoded in a process reverse to that of the image encoder.

[0083] The entropy decoding unit 210 may perform entropy decoding in a process opposite to the process of performing entropy encoding in the entropy encoding unit of the image encoder. For example, various methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC) may be applied in response to the method performed in the image encoder.

[0084] In the entropy decoding unit 210, information related to intra prediction and inter prediction performed in the encoder may be decoded.

[0085] The rearrangement unit 215 may perform rearrangement on the bitstream entropy-decoded at the entropy decoding unit 210 based on the rearrangement method of the encoding unit. Coefficients expressed in a one-dimensional vector form may be reconstructed into coefficients in a two-dimensional block form and may be rearranged.

[0086] The inverse quantization unit 220 may perform dequantization based on a quantization parameter provided in the encoder and a coefficient value of the rearranged block.

[0087] The inverse transform unit 225 may perform inverse DCT, inverse DST, and inverse KLT, i.e., inverse transforms for DCT, DST, and KLT (i.e., transforms performed in the transform unit on the quantization result performed in the image encoder). The inverse transform may be performed based on the transmission unit determined in the image encoder. In the inverse transform unit 225 of the image decoder, a transform method (e.g., DCT, DST, KLT) may be selectively performed according to a plurality of information (e.g., a prediction method, a size of a current block, a prediction direction, etc.).

[0088] The prediction units 230 and 235 may generate a prediction block based on the information related to prediction block generation provided in the entropy decoding unit 210 and previously decoded block or picture information provided in the memory 245 .

[0089] As described above, when performing intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit is the same as the size of the transform unit, the intra prediction of the prediction unit can be performed based on the pixel at the left position, the pixel at the upper left position, and the pixel at the upper position of the prediction unit, but when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, the intra prediction can be performed by using reference pixels based on the transform unit. In addition, intra prediction using only N×N partitioning for the minimum coding unit can be used.

[0090] The prediction units 230 and 235 may include: a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information input from the entropy decoding unit 210, such as prediction unit information, prediction mode information of an intra prediction method, information related to motion prediction of an inter prediction method, etc.; classify prediction units in a current coding unit; and determine whether a prediction unit performs inter prediction or intra prediction. On the other hand, if information indicating that motion information is derived and used from the decoder side and information about a method for deriving motion information are sent in the encoding device 100 without sending motion prediction related information for inter prediction, the prediction unit determination unit determines whether the inter prediction unit 230 performs prediction based on the information sent from the encoding device 100.

[0091] The inter prediction unit 230 may perform inter prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction unit by using information required for inter prediction of the current prediction unit provided by the image encoder. In order to perform inter prediction, it may be determined whether the motion prediction method in the prediction unit included in the corresponding coding unit based on the coding unit is a skip mode, a merge mode, an AMVP mode, an intra block copy mode, or an affine mode.

[0092] The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit on which intra prediction is performed, intra prediction may be performed based on intra prediction mode information in the prediction unit provided by the image encoder.

[0093] The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. As part of performing filtering on the reference pixels of the current block, the AIS filter may be applied by determining whether to apply the filter according to the prediction mode in the current prediction unit. AIS filtering may be performed on the reference pixels of the current block by using the prediction mode in the prediction unit and the AIS filter information provided by the image encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0094] When the prediction mode in the prediction unit is a prediction unit that performs intra-frame prediction based on the pixel value of the interpolated reference pixel, the reference pixel interpolation unit may interpolate the reference pixel to generate the reference pixel in units of pixels equal to or less than the integer value. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. When the prediction mode of the current block is the DC mode, the DC filter may generate the prediction block by filtering.

[0095] The reconstructed block or picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset modification unit, and an ALF.

[0096] Information about whether to apply a deblocking filter to a corresponding block or picture and information about whether to apply a strong filter or a weak filter when applying the deblocking filter may be provided by the image encoder. The deblocking filter of the image decoder may receive information related to the deblocking filter provided by the image encoder and perform deblocking filtering on the corresponding block in the image decoder.

[0097] The offset modification unit may perform offset modification on the reconstructed image based on the type of offset modification applied to the image in encoding, offset value information, etc. ALF may be applied to the coding unit based on information provided by the encoder about whether ALF is applied, ALF coefficient information, etc. Such ALF information may be provided by being included in a specific parameter set.

[0098] The memory 245 may store the reconstructed picture or block to be used as a reference picture or a reference block, and may also provide the reconstructed picture to the output unit.

[0099] Figures 3 to 5 A method of dividing a picture into a plurality of blocks is shown as an embodiment to which the present disclosure is applied.

[0100] Reference Figure 3, the picture 300 is divided into multiple basic coding units (Coding Tree Unit, Coding Tree Unit, hereinafter referred to as CTU).

[0101] The size of the CTU can be specified in units of pictures or in units of video sequences, and each CTU is constructed so as not to overlap with other CTUs. For example, the CTU size can be set to 128×128 throughout the sequence, and any one of 128×128 to 256×256 can be selected and used in units of pictures.

[0102] A coding block / coding unit (hereinafter referred to as CU) can be generated by hierarchically dividing CTU. Prediction and transformation can be performed in units of coding units, and the coding unit becomes a basic unit for determining a prediction coding mode. The prediction coding mode may represent a method for generating a predicted image, and consider intra-frame prediction, inter-frame prediction, or combined prediction, etc. as examples. Specifically, for example, a prediction block can be generated by using a prediction coding mode of at least any one of intra-frame prediction, inter-frame prediction, or combined prediction in units of coding units. When the reference picture indicates the current picture in the inter-frame prediction mode, a prediction block can be generated based on an area that has already been encoded in the current picture. This can be included in inter-frame prediction because the prediction block is generated by using a reference picture index and a motion vector. Intra-frame prediction is a method for generating a prediction block by using information of the current picture, inter-frame prediction is a method for generating a prediction block by using information of other pictures that have been decoded, and combined prediction is a method for combining inter-frame prediction and intra-frame prediction. Combined prediction can encode / decode some areas of multiple sub-areas constituting a coding block using inter-frame prediction, and can encode / decode other areas using intra-frame prediction. Alternatively, the combined prediction may first perform inter-frame prediction on a plurality of sub-regions and then perform intra-frame prediction. In this case, the prediction value of the coding block may be obtained by performing a weighted average on the prediction value according to the inter-frame prediction and the prediction value according to the intra-frame prediction. The number of sub-regions constituting one coding block may be 2, 3, 4 or more, and the shape of the sub-region may be a quadrilateral, a triangle or other polygon.

[0103] Reference Figure 4 , CTU can be divided in the form of quadtree, binary tree or ternary tree. The divided blocks can be additionally divided in the form of quadtree, binary tree or ternary tree. The method of dividing the current block into 4 square partitions is called quadtree partition, the method of dividing the current block into 2 square partitions or non-square partitions is called binary tree partition, and the method of dividing the current block into 3 partitions is defined as ternary tree partition.

[0104] Binary tree partitioning in the vertical direction ( Figure 4The SPLIT_BT_VER in the above example is called vertical binary tree partitioning, while the horizontal binary tree partitioning ( Figure 4 SPLIT_BT_HOR) in is defined as a horizontal binary tree partition.

[0105] Ternary tree partitioning in the vertical direction ( Figure 4 The SPLIT_TT_VER in the vertical ternary tree partition is called the vertical ternary tree partition, while the horizontal ternary tree partition ( Figure 4 SPLIT_TT_HOR) in is defined as horizontal ternary tree partitioning.

[0106] The number of partitions may be referred to as a partition depth, and the maximum value of the partition depth may be set differently for each sequence, picture, sub-picture, slice, or tile, and may be set to have different partition depths according to a partition tree form (quadtree / binary tree / ternary tree), and the syntax for representing the partition depth may be signaled.

[0107] As a method such as quadtree partitioning, binary tree partitioning, or other multi-tree partitioning (such as ternary tree partitioning), the coding units of leaf nodes may be configured by additionally partitioning the partitioned coding units or without additional partitioning.

[0108] Reference Figure 5 , a coding unit can be set by hierarchically dividing a CTU, and the coding unit can be divided by using at least any one of binary tree division, quadtree division / ternary tree division. Such a method is called multi-tree division.

[0109] A coding unit generated by splitting an arbitrary coding unit having a split depth of k is referred to as a low coding unit, and the split depth is (k+1). A coding unit having a split depth of k including a low coding unit having a split depth of (k+1) is referred to as a high coding unit.

[0110] The split type of the current coding unit may be limited according to the split type of the upper coding unit and / or the split types of coding units around the current coding unit.

[0111] In this case, the partition type represents an indicator indicating which partition among binary tree partition, quad tree partition, and ternary tree partition is used.

[0112] The predicted image may be generated by various methods of encoding / decoding a video, and the method of generating the predicted image is called a predictive coding mode.

[0113] The prediction coding mode can be configured with an intra-frame prediction coding mode, an inter-frame prediction coding mode, a current picture reference coding mode or a combined coding mode (combined prediction), etc.

[0114] The inter-frame prediction coding mode is referred to as a prediction coding mode that generates a prediction block (prediction image) of the current block by using information of a previous picture, and the intra-frame prediction coding mode is referred to as a prediction coding mode that generates a prediction block by using samples adjacent to the current block. The prediction block can be generated by using a pre-reconstructed image of the current picture, which is referred to as a current picture reference mode or an intra-frame block copy mode.

[0115] A prediction block may be generated by using at least 2 or more prediction coding modes among an inter prediction coding mode, an intra prediction coding mode, or a current picture reference coding mode, which is referred to as a combined coding mode (combined prediction).

[0116] A method of generating a predicted block (predicted image) of a block in a current picture by using information of a previous picture is called an inter-frame prediction encoding mode.

[0117] For example, a predicted image may be generated based on a corresponding block (collocated block) of a previous picture, or a predicted block (predicted image) may be generated based on a specific block of a previous picture.

[0118] A block in which a corresponding block of a previous image is subtracted from a current block is called a corresponding prediction block (co-located prediction block).

[0119] The prediction block may be generated by considering the motion of the object in the inter-prediction encoding mode.

[0120] For example, if it is known in which direction and how much an object in a previous picture has moved, a prediction block (prediction image) can be generated by subtracting the block taking into account the motion from the current block, which is called a motion prediction block.

[0121] The residual block may be generated by subtracting the motion prediction block or the corresponding prediction block from the current block.

[0122] When an object generates motion, if a motion prediction block is used instead of a corresponding prediction block, the energy of the residual block is reduced, thereby improving compression performance.

[0123] Therefore, the method using the motion prediction block is called motion compensated prediction, and motion compensated prediction is used for most inter-frame prediction encoding.

[0124] The value indicating in which direction and how much the object in the previous picture moves is called a motion vector. The motion vector can use motion vectors with different pixel precisions in units of sequences, pictures, sub-pictures, slices, tiles, CTUs, or CUs. For example, the pixel precision of the motion vector in a particular block can be at least any one of 1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, or 8. For each inter-frame prediction coding mode, the type and / or number of available pixel precision candidates may be different, which will be described below. For example, for the affine inter-frame prediction method, k pixel precision is available, and for the inter-frame prediction method using translational motion, i pixel precision is available. For the current picture reference mode, j pixel precision is available. In this case, k, i, and j can be natural numbers such as 1, 2, 3, 4, 5, or more. However, k can be less than i, and i can be less than j. The affine inter prediction method may use at least one pixel precision of 1 / 16, 1 / 4, or 1, while the inter prediction method using translation motion (e.g., merge mode, AMVP mode) may use at least one pixel precision of 1 / 4, 1 / 2, 1, or 4. The current picture reference mode may use at least one pixel precision of 1, 4, or 8.

[0125] For the inter prediction mode, an inter prediction method using translation motion and an affine inter prediction method using affine motion may be selectively used.

[0126] Figures 6 to 8 An affine inter prediction method is shown as an embodiment to which the present disclosure is applied.

[0127] In videos, there are many situations where the motion of a particular object does not appear linear. For example, Figure 6 As shown, when only the translation motion vector is used for the movement of an object in an image using affine motion such as enlargement, reduction, rotation, affine transformation that enables transformation into arbitrary shapes, etc., the movement of the object may not be effectively represented and the encoding performance may be reduced.

[0128] Affine motion can be expressed as the following equation 1.

[0129] [Formula 1]

[0130] v x =ax-by+e

[0131] v y =cx+dy+f

[0132] When affine motion is expressed by using a total of 6 parameters, it is effective for an image with complex motion, but many bits are used to encode the affine motion parameters, thus possibly reducing encoding efficiency.

[0133] Therefore, affine motion can be simplified and represented using four parameters, which is called a four-parameter affine motion model. Formula 2 uses four parameters to represent affine motion.

[0134] [Formula 2]

[0135] v x =ax-by+e

[0136] v y =bx+ay+f

[0137] The 4-parameter affine motion model may include motion vectors at 2 control points of the current block. The control points may include at least one of the upper left corner, upper right corner, or lower left corner of the current block. In an example, Figure 7 As shown in the left image of , the 4-parameter affine motion model can be determined by the motion vector sv0 at the upper left sample (x0, y0) of the coding unit and the motion vector sv1 at the upper right sample (x1, y1) of the coding unit, and sv0 and sv1 are called affine seed vectors. In the following, it is assumed that the affine seed vector sv0 at the upper left position is the first affine seed vector, and the affine seed vector sv1 at the upper right position is the second affine seed vector. The affine seed vector at the lower left position can also be used instead of one of the first seed vector and the second seed vector, and the affine seed vector at the lower left position is used in the 4-parameter affine motion model.

[0138] like Figure 7 As shown in the right image of , the 6-parameter affine motion model is an affine motion model in which the motion vector sv2 of the remaining control point (for example, the sample of (x2, y2) at the lower left position) is added to the 4-parameter affine motion model. In the following, it is assumed that the affine seed vector sv0 at the upper left position is the first affine seed vector, the affine seed vector sv1 at the upper right position is the second affine seed vector, and the affine seed vector sv2 at the lower left position is the third affine seed vector.

[0139] Information on the number of parameters for representing affine motion may be encoded in a bitstream. For example, a flag indicating whether 6 parameters are used and a flag indicating whether 4 parameters are used may be encoded / decoded in units of at least one of a picture, a sub-picture, a slice, a tile group, a tile, a coding unit, or a CTU. Therefore, any one of a 4-parameter affine motion model or a 6-parameter affine motion model may be selectively used in a predetermined unit.

[0140] like Figure 8 As shown, a motion vector can be derived from each sub-block of a coding unit by using an affine seed vector, and the motion vector is called an affine sub-block vector.

[0141] The affine sub-block vector may be derived as shown in the following Equation 3. In this case, the basic sample position (x, y) of the sub-block may be a sample located at a corner of the block (e.g., an upper left sample), or may be a sample at the center of at least one of the x-axis or the y-axis (e.g., a center sample).

[0142] [Formula 3]

[0143]

[0144] Motion compensation may be performed in units of coding units or in units of subblocks in a coding unit by using an affine subblock vector, which is referred to as an affine inter prediction mode. In Equation 3, (x1-x0) may have the same value as the width of the coding unit.

[0145] Fig. 9 An image encoding / decoding method using a motion vector difference (MVD)-based merge mode is shown as an embodiment to which the present disclosure is applied.

[0146] The motion information of the current coding unit can be derived from the motion information of the surrounding blocks without encoding the motion information of the current coding unit (or current block). The motion information of any one of the surrounding blocks can be set as the motion information of the current coding unit, which is called merge mode. In other words, the merge mode uses the motion information of the surrounding blocks as the motion information of the current block in the same way, and unlike the AMVP mode, does not require the encoding of a separate motion vector difference. However, even in merge mode, a predetermined motion vector difference (MVD) can be used to improve the accuracy of the motion vector. In the present disclosure, motion information can be understood to include at least one of a motion vector, a reference picture index, or a prediction direction information.

[0147] MVD may be selectively used based on a predetermined flag (hereinafter, MVD_flag). MVD_flag may indicate whether to use motion vector difference (MVD) in merge mode. For example, when the flag is a first value, motion vector difference is used in merge mode, otherwise motion vector difference is not used in merge mode.

[0148] Reference Fig. 9 , a merge candidate list of the current block can be constructed at S900.

[0149] The merge candidate list may include one or more merge candidates, which may be used to derive motion information of the current block. The size of the merge candidate list may be variably determined based on information indicating the maximum number of merge candidates for constructing the merge candidate list (hereinafter, size information). The size information may be encoded in the encoding device and sent with a signal, or may be a fixed value (e.g., an integer such as 2, 3, 4, 5, 6, or more) pre-agreed in the decoding device.

[0150] The plurality of merge candidates belonging to the merge candidate list may include at least one of a spatial merge candidate or a temporal merge candidate.

[0151] The spatial merge candidate may refer to a surrounding block spatially adjacent to the current block or motion information of the surrounding block. In this case, the surrounding block may include at least one of the lower left block (A0), the left block (A1), the upper right block (B0), the upper block (B1), or the upper left block (B2) of the current block. According to a predetermined priority, the available surrounding blocks in the surrounding blocks may be sequentially added to the merge candidate list. For example, the priority may be defined as B1->A1->B0->A1->B2, A1->B1->A0->B1->B2, A1->B1->B0->A0->B2, etc., but it is not limited thereto.

[0152] A temporal merge candidate may refer to one or more collocated blocks or motion information of a collocated block belonging to a collocated picture. In this case, a collocated picture is any one of a plurality of reference pictures belonging to a reference picture list, which may be a picture different from the picture to which the current block belongs. A collocated picture may be a picture that is first located in a reference picture list, or may be a picture that is last located in a reference picture list. Alternatively, a collocated picture may be specified based on an index that is encoded to indicate a collocated picture. A collocated block may include at least one of a block (C1) or a surrounding block (C0) adjacent to the lower right corner of the current block, wherein the block (C1) includes the center position of the current block. Available blocks in C0 and C1 may be sequentially added to the merge candidate list according to a predetermined priority. For example, C0 may have a higher priority than C1. However, it is not limited thereto, and C1 may have a higher priority than C0.

[0153] The encoding / decoding device may include a buffer that stores motion information of one or more blocks (hereinafter, previous blocks) encoded / decoded before the current block. In other words, the buffer may store a list (hereinafter, motion information list) configured with motion information of the previous block.

[0154] The motion information list can be initialized in units of any one of a picture, a slice, a tile, a CTU row, or a CTU. Initialization may refer to a state in which the motion information list is empty. The motion information of the previous block may be sequentially added to the motion information list according to the encoding / decoding order of the previous block, but the motion information list may be updated in a FIFO (first in, first out) manner by considering the size of the motion information list. For example, when the most recently encoded / decoded motion information (hereinafter, the most recent motion information) is the same as the motion information pre-added to the motion information list, the most recent motion information may not be added to the motion information list. Alternatively, the motion information that is the same as the most recent motion information may be removed from the motion information list, and the most recent motion information may be added to the motion information list. In this case, the most recent motion information may be added to the last position of the motion information list, or the most recent motion information may be added to the position of the removed motion information.

[0155] The previous block may include at least one of one or more surrounding blocks that are spatially adjacent to the current block or one or more surrounding blocks that are spatially non-adjacent to the current block. Figures 10 to 14 The scope of the previous blocks of the present disclosure is described in detail.

[0156] For the merge candidate list, a previous block belonging to the buffer or the motion information list or motion information of the previous block may be further added as a merge candidate.

[0157] Specifically, a redundancy check between the motion information list and the merge candidate list may be performed. A redundancy check may be performed on all or some of the merge candidates belonging to the merge candidate list and all or some of the previous blocks of the motion information list. However, for the sake of convenience of description, it is assumed that the redundancy check of the present disclosure is performed on some of the merge candidates belonging to the merge candidate list and some of the previous blocks of the motion information list. In this case, some of the merge candidates of the merge candidate list may include at least one of the left block or the upper block in the spatial merge candidate. However, it is not limited to this, and it may be limited to a block of any one of the spatial merge candidates, or may additionally include at least one of the lower left block, the upper right block, the upper left block, or the temporal merge candidate. Some of the previous blocks of the motion information list may refer to the K previous blocks most recently added to the motion information list. In this case, K may be 1, 2, 3 or more, and may be a fixed value pre-agreed in the encoding / decoding device.

[0158] For example, assume that 5 previous blocks (or motion information of previous blocks) are stored in the motion information list, and indexes 1 to 5 are assigned to the previous blocks, respectively. When the index is larger, it means that it is the most recently stored previous block. In this case, a redundancy check of motion information between the previous blocks with indexes 5, 4, and 3 and some merge candidates in the merge candidate list may be performed. Alternatively, a redundancy check between the previous blocks with indexes 5 and 4 and some merge candidates in the merge candidate list may be performed. Alternatively, excluding the most recently added previous block with index 5, a redundancy check between the previous blocks with indexes 4 and 3 and some merge candidates in the merge candidate list may be performed.

[0159] As a result of the redundancy check, when there is at least one previous block with the same motion information, the previous block of the motion information list may not be added to the merge candidate list. On the other hand, when there is no previous block with the same motion information, all or some of the previous blocks in the motion information list may be added to the last position of the merge candidate list. In this case, the previous blocks may be added to the merge candidate list in the order of the previous blocks most recently added to the motion information list (i.e., in the order from large index to small index). However, there may be a restriction that the previous block most recently added to the motion information list (i.e., the previous block with the largest index) is not added to the merge candidate list. The previous block may be added by considering the size of the merge candidate list. For example, it is assumed that according to the size information of the above-mentioned merge candidate list, the merge candidate list has a maximum of T merge candidates. In this case, there may be a restriction that the previous block is added until the number of merge candidates belonging to the merge candidate list becomes (Tn). In this case, n may be an integer such as 1, 2 or more. Alternatively, the previous block may be repeatedly added until the number of merge candidates belonging to the merge candidate list becomes T.

[0160] Whether an adjacent block can be used as a merge candidate can be adaptively determined according to the value of MVD_flag. In the example, when MVD_flag is 1, the blocks adjacent to the upper right corner, lower left corner, or upper left corner of the current block can be set to unavailable. Alternatively, when MVD_flag is 1, the temporal merge candidate can be set to unavailable. Alternatively, when MVD_flag is 1, the combined merge candidate or the zero motion vector candidate may not be added to the merge candidate list. The combined merge candidate can be derived by the average value of the merge candidates pre-added to the merge candidate list.

[0161] Reference Fig. 9 , the motion information of the current block can be obtained S910 based on the merge candidate list and the merge index (merge_idx).

[0162] The merge index can specify any one of the multiple merge candidates belonging to the merge candidate list. The motion information of the current block can be set to the motion information of the merge candidate specified by the merge index. According to the value of MVD_flag, the maximum number of merge candidates that can be used by the current block can be adaptively determined. When MVD_flag is 0, up to M merge candidates can be used, and when MVD_flag is 1, N merge candidates less than M can be used.

[0163] Specifically, the merge index may be encoded / decoded based on MVD_flag. The merge index may be encoded / decoded only when MVD_flag is the first value. In this case, the merge index may have a value of 0 or 1. In other words, when a motion vector difference is used in merge mode, the motion information of the current block may be derived by using only any one of the first merge candidate (merge_idx=0) or the second merge candidate (merge_idx=1) in the merge candidate list.

[0164] Therefore, although the maximum number of merge candidates belonging to the merge candidate list is M, when the motion vector difference is used in the merge mode, the maximum number of merge candidates that the current block can use may be 2. In this case, the value of M may be determined based on the size information of the merge candidate list described above, and the value of M may be a value greater than 2.

[0165] However, the present invention is not limited thereto. For example, the first merge candidate may be forced to be used by setting the merge index to 0 without encoding / decoding the merge index. Alternatively, the merge index may have a value between 0 and i, and i may be an integer such as 2, 3 or more, but may be less than M.

[0166] Reference Fig. 9 , the motion vector difference (MVD) of the merge mode for the current block in S920 can be obtained.

[0167] The MVD of the current block may be derived based on the merged offset vector (offsetMV). The MVD may include at least one of: an MVD in the L0 direction (MVD0) or an MVD in the L1 direction (MVD1), and each of MVD0 and MVD1 may be derived by using the merged offset vector.

[0168] The merge offset vector may be determined based on the length (mvdDistance) and direction (mvdDirection) of the merge offset vector. For example, the merge offset vector (offsetMV) may be determined as shown in the following equation 4.

[0169] [Formula 4]

[0170] offsetMV[x0][y0][0]=(mvdDistance[x0][y0]<<2)*mvdDirection[x0][y0][0]

[0171] offsetMV[x0][y0][1]=(mvdDistance[x0][y0]<<2)*mvdDirection[x0][y0][1]

[0172] In this case, mvdDistance may be determined by considering at least one of a distance index (distance_idx) or a predetermined flag (pic_fpel_mmvd_enabled_flag). The distance index (distance_idx) may mean an index encoded to specify a length or distance of a motion vector difference (MVD). pic_fpel_mmvd_enabled_flag may indicate whether the motion vector uses integer pixel precision in a merge mode of the current block. For example, when pic_fpel_mmvd_enabled_flag is a first value, the merge mode of the current block uses integer pixel precision. In other words, this may mean that the motion vector resolution of the current block is integer pixel. On the other hand, when pic_fpel_mmvd_enabled_flag is a second value, the merge mode of the current block may use fractional pixel precision. In other words, when pic_fpel_mmvd_enabled_flag is a second value, the merge mode of the current block may use integer pixel precision or fractional pixel precision. Alternatively, when pic_fpel_mmvd_enabled_flag is the second value, there may be a restriction that the merge mode of the current block uses only fractional pixel precision. As an example of fractional pixel precision, it may be 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, etc.

[0173] For example, mvdDistance may be determined as in Table 1 below.

[0174] [Table 1]

[0175]

[0176] In addition, mvdDirection may represent the direction of the merged offset vector and may be determined based on a direction index (direction_idx). In this case, the direction may include at least one of the following: left, right, up, down, upper left, lower left, upper right, or lower right. For example, mvdDirection may be determined as shown in Table 2 below.

[0177] [Table 2]

[0178]

[0179] In Table 2, mvdDirection[x0][y0][0] may mean the sign of the x component of the motion vector difference, and mvdDirection[x0][y0][1] may mean the sign of the y component of the motion vector difference. Respectively, when direction_idx is 0, the direction of the motion vector difference may be determined to be the right direction, when direction_idx is 1, the direction of the motion vector difference may be determined to be the left direction, when direction_idx is 2, the direction of the motion vector difference may be determined to be the downward direction, and when direction_idx is 3, the direction of the motion vector difference may be determined to be the upward direction.

[0180] The distance index and direction index may be encoded / decoded only when MVD_flag is the first value. Figures 15 to 19 The above method of deriving the merged offset vector is described in detail.

[0181] On the other hand, the motion vector difference (MVD) may be set to be the same as the merged offset vector determined above. Alternatively, the merged offset vector may be modified by considering the POC difference (Pocdiff) between the reference picture of the current block and the current picture to which the current block belongs, and the modified merged offset vector may be set to the motion vector difference (MVD). In this case, the current block may be encoded / decoded by bidirectional prediction, and the reference picture of the current block may include a first reference picture (a reference picture in the L0 direction) and a second reference picture (a reference picture in the L1 direction). For convenience of description, hereinafter, the POC difference between the first reference picture and the current picture is referred to as PocDiff0, and the POC difference between the second reference picture and the current picture is referred to as PocDiff1.

[0182] When PocDiff0 is the same as PocDiff1, MVD0 and MVD1 of the current block may be equally set as merged offset vectors, respectively.

[0183] In the case where PocDiff0 and PocDiff1 are not the same, when the absolute value of PocDiff0 is greater than or equal to the absolute value of PocDiff1, MVD0 may be equivalently set to the merge offset vector. On the other hand, MVD1 may be derived based on the preset MVD0. For example, when the first reference picture and the second reference picture are long-term reference pictures, MVD1 may be derived by applying the first scaling factor to MVD0. The first scaling factor may be determined based on PocDiff0 and PocDiff1. On the other hand, when at least one of the first reference picture or the second reference picture is a short-term reference picture, MVD1 may be derived by applying the second scaling factor to MVD0. The second scaling factor may be a fixed value (e.g., -1 / 2, -1, etc.) pre-agreed in the encoding / decoding device. However, the second scaling factor may be applied only when the sign of PocDiff0 is different from the sign of PocDiff1. If the sign of PocDiff0 is the same as the sign of PocDiff1, MVD1 may be set to be the same as MVD0, and separate scaling may not be performed.

[0184] On the other hand, in the case where PocDiff0 is not the same as PocDiff1, when the absolute value of PocDiff0 is less than the absolute value of PocDiff1, MVD1 may be equivalently set as the merge offset vector. On the other hand, MVD0 may be derived based on the preset MVD1. For example, when the first reference picture and the second reference picture are long-term reference pictures, MVD0 may be derived by applying the first scaling factor to MVD1. The first scaling factor may be determined based on PocDiff0 and PocDiff1. On the other hand, when at least one of the first reference picture or the second reference picture is a short-term reference picture, MVD0 may be derived by applying the second scaling factor to MVD1. The second scaling factor may be a fixed value (e.g., -1 / 2, -1, etc.) pre-agreed in the encoding / decoding device. However, the second scaling factor may be applied only when the sign of PocDiff0 is different from the sign of PocDiff1. If the sign of PocDiff0 is the same as the sign of PocDiff1, MVD0 may be set to be the same as MVD1, and separate scaling may not be performed.

[0185] Reference Fig. 9 , the motion vector of the current block may be modified S930 by using a motion vector difference (MVD), and motion compensation of the current block may be performed S940 based on the modified motion vector.

[0186] Figures 10 to 14 A previous block that may be used as a merge candidate as an embodiment to which the present disclosure is applied is shown.

[0187] The surrounding blocks used in merge mode can be such as Fig.10 The blocks adjacent to the current coding unit (blocks adjacent to the boundary of the current coding unit) with merge indexes 0 to 4 may be non-adjacent blocks such as Fig.10 The merge indexes 5 to 26.

[0188] When the distance from the current block is greater than a predefined threshold, the merge candidate may be set as unavailable.

[0189] For example, the predefined threshold may be set to the height of the CTU (ctu_height) or ctu_height+N (referred to as the merge candidate available threshold). i -y0), which is the y-axis coordinate of the merged candidate (y i ) and the y-axis coordinate (y0) of the upper left sample of the current coding unit (hereinafter, referred to as the basic sample of the current coding unit) is greater than the merge candidate available threshold, the merge candidate can be set to be unavailable. In this case, N is a predefined offset value. Specifically, for example, N can be set to 16 or can be set to ctu_height.

[0190] When there are many merge candidates passing through the CTU boundary, many unavailable merge candidates may be generated, which may reduce the coding efficiency. The merge candidate above the coding unit (hereinafter, referred to as the upper merge candidate) can be set to as few as possible, and the merge candidate on the left and below the coding unit (hereinafter, referred to as the lower left merge candidate) can be set to as many as possible.

[0191] like Fig.11 As shown, the difference between the y-axis coordinate of the basic sample of the current coding unit and the y-axis coordinate of the upper merge candidate can be set to not more than twice the height of the coding unit.

[0192] like Fig.11 As shown, the difference between the x-axis coordinate of the basic sample of the current coding unit and the x-axis coordinate of the bottom-left merge candidate can be set to not more than twice the width of the coding unit.

[0193] Merge candidates adjacent to the current coding unit are referred to as adjacent merge candidates, and merge candidates not adjacent to the current coding unit are referred to as non-adjacent merge candidates.

[0194] The isAdjacentMergeflag (a flag indicating whether the merge candidate of the current coding unit is an adjacent merge candidate) can be sent by signal.

[0195] When the value of isAdjacentMergeflag is 1, it indicates that the motion information of the current coding unit can be derived from the adjacent merge candidate, and when the value of isAdjacentMergeflag is 0, it indicates that the motion information of the current coding unit can be derived from the non-adjacent merge candidate, which can be represented in Table 3 below.

[0196] When a non-adjacent merge candidate is used as a merge candidate, a value of (NA_merge_idx[x0][y0]) may be signaled, that is, a non-adjacent merge index obtained by subtracting the value of the number of adjacent merge candidates (NumAdjMerge) from the merge index in the following equation 5.

[0197] [Formula 5]

[0198] NA_merge_idx[x0][y0]=merge_idx[x0][y0]-NumAdjMerge

[0199] [Table 3]

[0200]

[0201]

[0202]

[0203] The position of the non-neighboring merge candidate whose y coordinate is greater than the upper boundary of the current block may be determined based on the distance between the x coordinate of the neighboring merge candidate and the base sample of the coding unit.

[0204] A non-adjacent merge candidate having the same x-coordinate as the adjacent merge candidate A or a non-adjacent merge candidate whose distance from the x-coordinate of the adjacent merge candidate A is within a threshold is referred to as a merge candidate collinear with the vertical axis of A.

[0205] For example, Fig.12 As shown, the x-coordinate of the merge candidate collinear with the vertical axis of A can be obtained by subtracting an offset (hereinafter, a non-adjacent merge candidate offset) from the x-coordinate of the adjacent merge candidate. The non-adjacent merge candidate offset may have a fixed value, or may be set to have a larger value as the distance between the adjacent merge candidate and the non-adjacent merge candidate is longer. The offset may be limited to a value less than the width of the current block.

[0206] Specifically, for example, Fig.12As shown, when the coordinates of the adjacent merge candidate A are (x0, y0), the coordinates of the merge candidate closest to the adjacent merge candidate on the same vertical axis (hereinafter, the first non-adjacent merge candidate) can be set to (x0-(cu_height / 2-1), y1) or (x0-(cu_width / 2-1), y1).

[0207] The coordinates of the merge candidate (hereinafter, the second non-adjacent merge candidate) closest to the first non-adjacent merge candidate on the same vertical axis may be set to (x0-(cu_height-1), y2) or (x0-(cu_width-1), y2).

[0208] When the non-adjacent merge candidate is outside the LCU boundary, it can be set to derive the merge candidate from the sample closest to the non-adjacent merge candidate and adjacent to the LCU boundary, such as Fig.13 shown.

[0209] For example, when the upper boundary of the current block is adjacent to the LCU boundary, the sample with the same x coordinate as the non-adjacent merge candidate B among the samples adjacent to the LCU boundary can be replaced with a non-adjacent merge candidate. Specifically, the non-adjacent merge candidate B (coordinates (x1, y1), Fig.13 The merge candidate with merge index 6 in the left figure replaces the sample B' (coordinate (x1, y0) adjacent to the current coding unit. Fig.13 The merge candidate with merge index 6' in the left figure).

[0210] In another example, the merge candidate C(coordinates (x2, y2), Fig.13 The merge candidate with the merge index of 15 in the right figure replaces the sample B' (coordinate (x2, y0) with the same x coordinate and adjacent to the current coding unit. Fig.13 In the right figure, there is a merge candidate with a merge index of 15').

[0211] In another example, when the merge candidates on the same vertical axis have the same x-coordinate, the samples adjacent to the LCU boundary may be replaced with non-adjacent merge candidates outside the LCU boundary by subtracting an offset from the x-axis coordinate of the non-adjacent merge candidate. Fig.14 The non-adjacent merge candidates outside the LCU boundary shown replace samples adjacent to the current coding unit, and the x-axis coordinates of the replaced samples can be obtained by subtracting a specific offset from the non-adjacent merge candidates. In the example, the non-adjacent merge candidate offset can be used as the specific offset. The non-adjacent merge offset can be set differently for each non-adjacent merge candidate outside the LCU boundary. In this case, the non-adjacent merge offset can be determined according to the order of the non-adjacent merge candidates outside the LCU boundary.

[0212] When a non-adjacent merge candidate is not available, the non-adjacent merge candidate may be replaced with a sample in a sample adjacent to the current block that applies a non-adjacent merge offset to the x-coordinate of the non-adjacent merge candidate. As the number of non-adjacent merge candidates that are not available in the merge candidates on the same vertical axis increases, the size of the non-adjacent merge offset may increase.

[0213] The above-mentioned description may be applied to merge candidates on the same horizontal axis and merge candidates on the same vertical axis.For merge candidates on the same horizontal axis, a non-adjacent merge offset may be applied to the y-axis coordinates of the non-adjacent merge candidates.

[0214] Figures 15 to 19 A method of deriving a merging offset vector for a merging mode is shown as an embodiment of applying the present disclosure.

[0215] As shown in Table 4 below, when the value of the merge flag (merge_flag) or the skip flag (cu_skip_flag) is 1, at least one of the merge offset vector flag (MVD_flag), the merge index, the distance index, and the direction index can be encoded / decoded.

[0216] [Table 4]

[0217]

[0218]

[0219]

[0220]

[0221] The combined offset vector (offsetMV) can be derived from the distance index and the direction index.

[0222] like Fig.15 As shown, the optimal merge offset vector value can be obtained by searching near the upper and lower sides and the left and right sides of the motion derived from the merge candidate. Each of the x-axis vector and the y-axis vector of the merge offset vector can be set to be less than a maximum of 32 integer samples from the merge motion vector that becomes the basis.

[0223] The merge offset vector flag may be signaled only when the value of the merge index is less than the base value. Alternatively, the merge offset vector flag may be signaled only when the merge candidate indicated by the merge index is bidirectional or unidirectional.

[0224] The value obtained by subtracting the POC of the current picture from the POC of the reference picture L0 (short for picture order count, an indicator of the output order of pictures) is called the reference picture L0 difference (PocDiffO), and the value obtained by subtracting the POC of the current picture from the POC of the reference picture L1 is called the reference picture L1 difference (PocDiff1).

[0225] If the sign of the reference picture L0 difference is different from the sign of the reference picture L1 difference, offsetMV can be generated by multiplying DistFromMergeMV (a variable derived from the distance index) by the value derived from the direction index, as shown in Equation 6 below.

[0226] [Formula 6]

[0227] offsetMVL0[0]=(DistFromMergeMV<<2)*sign[x0][y0][0]

[0228] offsetMVL0[1]=(DistFromMergeMV<<2)*sign[x0][y0][1]

[0229] offsetMVL1[0]=(DistFromMergeMV<<2)*sign[x0][y0][0]

[0230] offsetMVL1[1]=(DistFromMergeMV<<2)*sign[x0][y0][1]

[0231] If the sign of the reference picture L0 difference is the same as the sign of the reference picture L1 difference, offsetMV can be generated by multiplying DistFromMergeMV (a variable derived from the distance index) by the value derived from the direction index, as shown in Equation 7 below.

[0232] [Formula 7]

[0233] offsetMVL0[0]=(DistFromMergeMV<<2)*sign[x0][y0][0]

[0234] offsetMVL0[1]=(DistFromMergeMV<<2)*sign[x0][y0][1]

[0235] offsetMVL1[0]=-1*(DistFromMergeMV<<2)*sign[x0][y0][0]

[0236] offsetMVL1[1]=-1*(DistFromMergeMV<<2)*sign[x0][y0][1]

[0237] DistFromMergeMV can be obtained as shown in Table 5 below.

[0238] [Table 5]

[0239] distance_idx[x][y] Binary DistFromMergeMV[x0][y0] 0 0 1 1 10 2 2 110 4 3 1110 8 4 11110 16 5 111110 32 6 1111110 64 7 1111111 128

[0240] Fewer / more DistFromMergeMV values ​​than shown in Table 5 may be used.

[0241] The range of DistFromMergeMV may be determined differently depending on the motion vector precision of the current block or merge candidate. For example, when the motion vector precision of the current block or merge candidate is quarter pixel, DistFromMergeMV may be selected in the range of {1, 2, 4, 8, 16}, and when the motion vector precision of the current block or merge candidate is integer pixel, DistFromMergeMV may be selected in the range of {4, 8, 16, 32, 64}.

[0242] Table 6 below shows sign[x][y][0] (a variable indicating an x-axis offset) and sign[x][y][1] (a variable indicating a y-axis offset) defined according to a direction index value.

[0243] [Table 6]

[0244] direction_idx[x][y] Binary sign[x][y][0] sign[x][y][1] 0 00 +1 0 1 01 -1 0 2 10 0 +1 3 11 0 -1

[0245] Each of the x-axis vector and the y-axis vector of the merged offset vector may be set to be smaller than a maximum of 2 integer samples from the merged motion vector becoming a basis, and distance_idx may be set to have a total of 4 indexes.

[0246] Alternatively, a merge offset vector range flag (merge_offset_range_flag, syntax indicating a search range of a merge offset vector) may be indicated in a picture header or a slice header, etc. In an example, when the value of merge_offset_extend_range_flag is 1, each of the x-axis vector and the y-axis vector of the merge offset vector may be set to be less than a maximum of 32 integer samples from the merged motion vector that becomes a base, and when the value of merge_offset_extend_range_flag is 0, each of the x-axis vector and the y-axis vector of the merge offset vector may be set to be less than up to 2 integer samples from the merged motion vector that becomes a base.

[0247] [Table 7]

[0248]

[0249] like Fig.16 As shown, the optimal merge offset vector value can be obtained by searching around the upper, lower, left, right and diagonal directions of the motion derived from the merge candidate, and direction_idx can be set to have a total of 8 indexes as shown in Table 8.

[0250] In merging the offset vector search range with Fig.16 Table 8 can be used to represent sign[x][y][0] (variable representing x-axis offset) and sign[x][y][1] (variable representing y-axis offset) when the merged offset vector search range is the same as the left figure. Fig.16 When the right figure is the same, Table 8 can be used to represent sign[x][y][0] (the variable representing the x-axis offset) and sign[x][y][1] (the variable representing the y-axis offset).

[0251] [Table 8]

[0252] direction_idx[x][y] Binary sign[x][y][0] sign[x][y][1] 0 000 +1 0 1 001 -1 0 2 010 0 +1 3 011 0 -1 4 100 +1 +1 5 101 +1 -1 6 110 -1 +1 7 111 -1 -1

[0253] [Table 9]

[0254]

[0255]

[0256] Alternatively, depending on whether the selected merge candidate has a bidirectional motion vector, it may have different search ranges and search modes. For example, when the selected merge candidate has a unidirectional motion vector (either an L0 motion vector or an L1 motion vector), it may be set to have a search range and search mode such as Fig.17 The diamond search pattern in the left figure of FIG. 1 and when the selected merge candidate has a bidirectional motion vector (L0 motion vector and L1 motion vector), it can be set to have a Fig.17 The rectangular search pattern is shown in the right figure.

[0257] Depending on the distance index value, different search ranges and search modes may be used. For example, when the distance index value is less than a threshold, it may be set to have a diamond search mode, and when the distance index value is greater than or equal to the threshold, it may be set to have a rectangular search mode.

[0258] Depending on the motion vector accuracy of the current block or merge candidate, different search ranges and search modes may be used. For example, when the motion vector accuracy is less than or equal to an integer pixel (1 / 4 pixel, 1 / 2 pixel, etc.), it may be set to have a diamond search mode, and when the motion vector accuracy is greater than an integer pixel, it may be set to have a rectangular mode.

[0259] Depending on the value of DistFromMergeMV, the range of the search direction can be configured differently. In the example, when the value of DistFromMergeMV is equal to or less than the threshold, the search can be performed in 8 directions such as the example of Table 8 or Table 9, and when the value of DistFromMergeMV is greater than the threshold, the search can be performed in 4 directions such as the example of Table 6.

[0260] Depending on the difference between MVx and MVy, the range of the search direction can be configured differently. In the example, when the difference between MVx and MVy is equal to or less than the threshold, the search can be performed in 8 directions such as the example in Table 8 or Table 9, and when the difference between MVx and MVy is greater than the threshold, the search can be performed in 4 directions such as the example in Table 6.

[0261] The final motion vector can be obtained by adding offsetMV to mergeMV (the motion vector obtained from the merge index), as shown in Equation 8 below.

[0262] [Formula 8]

[0263] mvL0[0]=mergeMVL0[0]+offsetMVL0[0]

[0264] mvL0[1]=mergeMVL0[0]+offsetMVL0[0]

[0265] mvL10[0]=mergeMVL1[0]+offseetMVL1[0]

[0266] mvL1[1]=mergeMVL1[0]+offsetMVL1[0]

[0267] offsetMV (merge offset vector) can be derived from the first distance flag (hereinafter, distance_1st_flag), the second distance index (hereinafter, distance_2nd_idx) and the direction index. DistFromMergeMV can be derived by using the first distance index and the second distance index, as shown in Formula 9.

[0268] [Formula 9]

[0269] DistFromMergeMV=16*distance_1st_flag+(1<<distance_2nd_idx)

[0270] In Equation 9, distance_1st_flag may have a value of 0 or 1, distance_2nd_idx may have a value of 0 to 3, and DistFromMergeMV may be a value between 1 and 128. Specifically, for example, DistFromMergeMV may be a value of at least any one of {1, 2, 4, 8, 16, 32, 64, 128}.

[0271] The first distance flag (hereinafter, distance_1st_flag), the second distance index (hereinafter, distance_2nd_idx), and the direction index may be expressed by using a syntax table as in Table 10 below.

[0272] [Table 10]

[0273]

[0274]

[0275]

[0276]

[0277] When using sub-block merge candidates, the same merge offset vector may be used for all sub-blocks.

[0278] The merge candidates in sub-block units may be derived as shown in the following process.

[0279] 1. The initial shift vector (shVector) can be derived from the motion vectors of the candidate blocks around the current block.

[0280] 2. As in Equation 10, a shifted subblock in which the position of the upper left sample is (xColSb, yColSb) can be obtained by adding the initial shift vector to (xSb, ySb) (the upper left sample of the subblock in the coding unit).

[0281] [Formula 10]

[0282] (xColSb, yColSb) = (xSb+shVector[0]>>4, ySb+shVector[1]>>4)

[0283] 3. The motion vector of the collocated block corresponding to the center position of the sub-block including (xColSb, yColSb) may be derived as the motion vector of the sub-block including the upper left sample (xSb, ySb).

[0284] Different merged offset vectors can be obtained based on the sub-blocks of the coding unit, which is called a merged offset vector encoding method based on sub-blocks.

[0285] In the example, Fig.18 As shown, the coding unit can be divided into sub-blocks of MxM units, and different merge offset vectors can be used in the units of sub-blocks. Both the distance index and the direction index can be set to be different from the previous sub-block.

[0286] Alternatively, the same DistFromMergeMV may be applied to all sub-blocks, but the directions may be determined individually. Conversely, the same direction may be applied to all sub-blocks, but the DistFromMergeMV may be determined individually.

[0287] Alternatively, the merge offset vector may be applied to only some sub-blocks. In an example, when the current block is divided into 2 sub-blocks, the motion information of the first sub-block may be set to be the same as the motion information of the merge candidate indicated by the merge index, and the motion information of the second sub-block may be derived by applying the merge offset vector to the motion information of the first sub-block. Specifically, for example, the first sub-block may be a left triangle prediction unit, and the second sub-block may be a right triangle prediction unit. The left triangle prediction unit may be as follows.

[0288] Fig.19 The left image is called a left triangle partition, and the right image is called a right triangle partition. The prediction unit to which the upper left or lower left sample of the coding unit belongs is called a left triangle prediction unit, and the prediction unit to which the upper right or lower right sample of the coding unit belongs is called a right triangle prediction unit, and the right triangle prediction unit or the left triangle prediction unit is collectively referred to as a triangle prediction unit.

[0289] Figure 20 to Figure 24 An intra prediction method is shown as an embodiment to which the present disclosure is applied.

[0290] For intra prediction, such as Fig. 20 In , pre-coded boundary samples around the current block (which are called intra reference samples) are used to generate intra prediction.

[0291] Intra-frame prediction can be performed in the following manner: setting the average value of intra-frame reference samples as the value of all samples of the prediction block (DC mode); after generating prediction samples in the horizontal direction by performing reference weighted prediction in the horizontal direction and prediction samples in the vertical direction by performing weighted prediction on reference samples in the vertical direction, generating prediction samples by performing weighted prediction on the horizontal direction and the vertical direction (planar mode); or by using a directional intra-frame prediction mode, etc.

[0292] You can use Fig.21 Intra prediction is performed by using 33 directions (a total of 35 intra prediction modes) as in the left figure of the figure or by using 65 directions (a total of 67 intra prediction modes) as in the right figure. When using directional intra prediction, an intra reference sample (reference reference sample) can be generated by considering the directionality of the intra prediction mode, and thereafter, intra prediction can be performed.

[0293] An intra reference sample at a left position of a coding unit is referred to as a left intra reference sample, and an intra reference sample at an upper position of a coding unit is referred to as an upper intra reference sample.

[0294] When performing directional intra prediction, the intra directional parameter (intraPredAng, a parameter indicating a prediction direction (or prediction angle)) may be set according to the intra prediction mode as shown in Table 11. Table 11 below is only an example of a directional intra prediction mode based on values ​​of 2 to 34 when 35 intra prediction modes are used. Of course, when the prediction direction (or prediction angle) of the directional intra prediction mode is further subdivided, 33 or more directional intra prediction modes may be used.

[0295] [Table 11]

[0296]

[0297] When intraPredAng is negative (for example, the intra prediction mode index is between 11 and 25), the left intra reference sample and the upper intra reference sample of the current block can be reconfigured as a one-dimensional reference sample (Ref_1D), which is based on Fig. 22 The angle of the intra prediction mode in is configured in 1D.

[0298] When the intra prediction mode index is between 11 and 18, such as Fig.23 As shown, one-dimensional reference samples may be generated in a counterclockwise direction from an intra reference sample located at the upper right side of the current block to an intra reference sample located at the lower left side.

[0299] In other modes, one-dimensional reference samples may be generated by using only intra reference samples on the upper side or intra reference samples on the left side.

[0300] When the intra prediction mode index is between 19 and 25, such as Fig.24 As shown, one-dimensional reference samples may be generated in a clockwise direction from an intra reference sample located at a lower left side of the current block to an intra reference sample located at an upper right side.

[0301] i Idx (reference sample to determine index) and i fact (Applied to i Idx The weight-related parameter of the at least one reference sample determined can be obtained as shown in the following formula 11. Idx and i fact It can be variably determined according to the slope of the directional intra prediction mode and is represented by i Idx The specified reference samples may correspond to integer pixels.

[0302] [Formula 11]

[0303] i Idx =(y+1)*P ang / 32

[0304] i fact =[(y+1)*P ang ]&31

[0305] The predicted image can be obtained by specifying at least one or more one-dimensional reference samples for each prediction sample. For example, the position of the one-dimensional reference sample that can be used to generate the prediction sample can be specified by considering the slope value of the directional intra-frame prediction mode. Each prediction sample can have a different directional intra-frame prediction mode. Multiple intra-frame prediction modes can be used for a prediction block. Multiple intra-frame prediction modes can be represented by a combination of multiple non-directional intra-frame prediction modes, can be represented by a combination of a non-directional intra-frame prediction mode and at least one directional intra-frame prediction mode, or can be represented by a combination of multiple directional intra-frame prediction modes. Different intra-frame prediction modes can be applied to each predetermined sample group in a prediction block. The predetermined sample group can be configured with at least one sample. The number of sample groups can be variably determined according to the size / number of samples of the current prediction block, or can be a fixed number preset in the encoder / decoder regardless of the size / number of samples of the prediction block.

[0306] Specifically, for example, by using i Idx (reference sample determination index) to specify the position of the one-dimensional reference sample.

[0307] When the slope of the intra prediction mode cannot be represented by only one one-dimensional reference sample according to the slope of the intra prediction mode, the first prediction image may be generated by interpolating adjacent one-dimensional reference samples as shown in Equation 12. When an angle line of the slope / angle according to the intra prediction mode does not pass through a reference sample located at an integer pixel, the first prediction image may be generated by interpolating reference samples adjacent to the left / right or above / below the corresponding angle line. The filter coefficient of the interpolation filter used in this case may be based on i fact For example, the filter coefficients of the interpolation filter may be derived based on the distance between the fractional pixels located on the corner line and the reference samples located at the integer pixels.

[0308] [Formula 12]

[0309] P(x,y)=((32-i fact ) / 32)*Ref_1D(x+i Idx +1)+(i fact / 32)*Ref_1D(x+i Idx +2)

[0310] When the slope of the intra prediction mode can be represented by only one one-dimensional reference sample (i fact When the value of is 0), the first predicted image can be generated as shown in Formula 13.

[0311] [Formula 13]

[0312] P(x, y) = Ref_1D(x+i ldx +1)

[0313] Figure 25 to Figure 28 A wide-angle based intra prediction method is shown as an embodiment to which the present disclosure is applied.

[0314] like Fig.25 As shown, the prediction angle of the directional intra prediction mode can be set between 45 degrees and -135 degrees.

[0315] When the intra prediction mode is performed in a non-square coding unit, due to the predefined prediction angle, the following disadvantage may occur: the current sample is predicted by using an intra reference sample far from the current sample instead of an intra reference sample close to the current sample.

[0316] For example, Fig.26 In the left figure of , for a coding unit whose width is greater than its height (hereinafter, a coding unit in the horizontal direction), intra prediction may be performed with a far sample L instead of a near sample T. In another example, Fig.26In the right figure of , for a coding unit whose height is greater than the width of the coding unit (hereinafter, a coding unit in the vertical direction), intra prediction may be performed with a far sample T instead of a near sample L.

[0317] In a non-square coding unit, intra prediction may be performed at a prediction angle wider than a predefined prediction angle, which is referred to as a wide-angle intra prediction mode.

[0318] The wide-angle intra prediction mode may have a prediction angle of 45-α to -135-β, and a prediction angle other than an angle used in an existing intra prediction mode is referred to as a wide-angle angle.

[0319] exist Fig.26 In the left figure of , sample A along the horizontal direction in the coding unit can be predicted according to the intra-frame reference sample T by using the wide-angle intra-frame prediction mode.

[0320] exist Fig.26 In the right figure of , sample A in the vertical direction in the coding unit can be predicted according to the intra reference sample L by using the wide-angle intra prediction mode.

[0321] The wide-angle intra prediction mode can use N+20 intra prediction modes by adding 20 wide angles to N existing intra prediction modes. Specifically, for example, when 67 intra modes are used, by adding wide angle as shown in Table 12, a total of 87 intra prediction modes can be used.

[0322] [Table 12]

[0323]

[0324] like Fig. 27 As shown, when the wide-angle intra prediction mode is used, the length of the upper intra reference sample may be set to 2W+1, and the length of the left intra reference sample may be set to 2H+1.

[0325] When encoding the intra prediction mode of the Wide intra prediction mode using the Wide intra prediction, the number of intra prediction modes increases, and thus the encoding efficiency may be reduced. The Wide intra prediction mode may be encoded by replacing it with an existing intra prediction mode not used in the Wide intra, and the replaced prediction mode is referred to as the Wide replacement mode.

[0326] Specifically, for example, Fig.28 As shown, when 35 intra predictions are used, wide-angle intra prediction mode 35 can be encoded into intra prediction mode 2 (wide-angle replacement mode), and wide-angle intra prediction mode 36 can be encoded into intra prediction mode 3 (wide-angle replacement mode).

[0327] The replacement mode and number may be set differently according to the shape of the coding block or the aspect ratio of the coding block. Specifically, for example, as shown in Table 13, the replacement mode and number may be set differently according to the shape of the coding block. Table 13 shows the replacement intra prediction mode used according to the aspect ratio of the coding block.

[0328] [Table 13]

[0329] Aspect Ratio Replace intra prediction mode W / H==16 Mode 12,13,14,15 W / H==8 Mode 12,13 W / H==4 Mode 2,3,4,5,6,7,8,9,10,11 W / H==2 Mode 2,3,4,5,6,7, W / H==1 none W / H==1 / 2 Mode 61,62,63,64,65,66 W / H==1 / 4 Mode 57,58,59,60,61,62,63,64,65,66 W / H==1 / 8 Mode 55,56 W / H==1 / 16 Mode 53,54,55,56

[0330] The predicted image may be generated through the above-described inter-frame prediction or intra-frame prediction, and an image obtained by subtracting the predicted image from the original image, that is, a residual image may be obtained.

[0331] The residual image can be decomposed into two-dimensional frequency components by a two-dimensional transform such as DCT (discrete cosine transform). There is a feature that although high-frequency components are removed from the image, no significant visual distortion is generated. When the value corresponding to the high frequency is set to a small value or 0, the compression efficiency can be improved without significant visual distortion.

[0332] DST (Discrete Sine Transform) can be used according to the size of the prediction block or the prediction mode. Specifically, for example, in the intra prediction mode and when the size of the prediction block / coding block is less than NxN, it can be set to use DST transform, and for other prediction blocks / coding blocks, it can be set to use DCT.

[0333] DCT is a process of decomposing (transforming) an image into two-dimensional frequency components by using cosine transform, and representing the frequency components in this case as a basic image. For example, when DCT transform is performed in NxN blocks, N 2 Basic mode component. Performing DCT transformation means obtaining the size of each basic mode component included in the original pixel block. The size of each basic mode component is called a DCT coefficient.

[0334] Generally, in an image where many non-zero components are distributed at low frequencies, DCT (Discrete Cosine Transform) is mainly used, whereas in an image where many high-frequency components are distributed, DST (Discrete Sine Transform) may be used.

[0335] DST means a process of decomposing (transforming) an image into two-dimensional frequency components by using a sine transform. A two-dimensional image can be decomposed (transformed) into two-dimensional frequency components by using a transform method other than DCT transform or DST transform, which is called two-dimensional image transform.

[0336] The two-dimensional image transform may not be performed in certain blocks in the residual image, which is called transform skip. After transform skip, quantization may be applied.

[0337] DCT or DST or two-dimensional image transform can be applied to any block in a two-dimensional image, and the transform used in this case is called the first transform. After performing the first transform, the transform can be performed again in some areas of the transform block, which is called the second transform.

[0338] The first transform may use one of a plurality of transform kernels. Specifically, for example, any one of DCT2, DCT8, or DST7 may be selected and used in the transform block. Alternatively, different transform kernels may be used in the transform in the horizontal direction and the transform in the vertical direction of the transform block.

[0339] The units of the blocks for performing the first transform and the second transform may be set differently. Specifically, for example, after performing the first transform in the 8x8 block of the residual image, the second transform may be performed on each 4x4 sub-block respectively. In another example, after performing the first transform in each 4x4 block, the second transform may be performed in the blocks of 8x8 size respectively.

[0340] The residual image to which the first transform is applied is referred to as a first transformed residual image.

[0341] DCT or DST or two-dimensional image transform may be applied to the first transform residual image, and the transform used in this case is referred to as a second transform. The two-dimensional image to which the second transform is applied is referred to as a second transform residual image.

[0342] The sample values ​​in the block after performing the first transform and / or the second transform are called transform coefficients. Quantization refers to the process of dividing the transform coefficients by a predefined value to reduce the energy of the block. The value defined to apply quantization to the transform coefficients is called a quantization parameter.

[0343] The predefined quantization parameter may be applied in sequence or block units. Typically, the quantization parameter may be defined as a value from 1 to 51.

[0344] After performing the transform and the quantization, a residual reconstructed image may be generated by performing dequantization and inverse transform. A first reconstructed image may be generated by adding the predicted image to the residual reconstructed image.

[0345] A transform block may be generated based on at least one of n transform types predefined in an encoding / decoding device. n may be, for example, an integer of 1, 2, 3, 4 or more. Transform types such as DCT2, DCT8, DST7, transform skip mode, etc. may be used. Only one same transform type may be applied in the vertical / horizontal direction of a block, or different transform types may be applied in the vertical / horizontal direction, respectively. For this purpose, a flag indicating whether one same transform type is applied may be used. The flag may be signaled from the encoding device.

[0346] In addition, the transform type may be determined based on information signaled from the encoding device, or may be determined based on predetermined encoding parameters. In this case, the encoding parameter may mean at least one of the size or shape, intra-frame prediction mode, or component type (e.g., luminance, chrominance) related to the block. The size of the block may be expressed as width, height, ratio of width and height, product of width and height, sum / difference of width and height, etc. For example, when the size of the current block is greater than a predetermined threshold, the transform type in the horizontal direction may be determined to be a first transform type (e.g., DCT2), and the transform type in the vertical direction may be determined to be a second transform type (e.g., DST7). The threshold may be, for example, an integer of 0, 4, 8, 16, 32, or more.

[0347] On the other hand, the residual coefficient according to the present disclosure may be obtained by performing a second transform after the first transform. The second transform may be performed on the residual coefficients of some regions in the current block. In this case, the decoding device may obtain a transform block of the current block by performing a second inverse transform on some regions and performing a first inverse transform on the current block including the inverse transformed regions.

[0348] Fig.29 and Fig.30 A method of applying a loop filter to a reconstructed block is shown as an embodiment to which the present disclosure is applied.

[0349] A predetermined loop filter may be applied to the reconstructed current block. The loop filter may include at least one of a deblocking filter, a SAO (Sample Adaptive Offset) filter, or an ALF (Adaptive Loop Filter).

[0350] Loop filtering is a technique of adaptively performing filtering on a decoded image to reduce the loss of information generated in quantization and encoding processes. A deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF) are examples of loop filtering.

[0351] The second reconstructed image may be generated by performing at least any one of: a deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF) on the first reconstructed image.

[0352] After applying the deblocking filter to the reconstructed image, SAO and ALF may be applied.

[0353] Transformation and quantization are performed in units of blocks in the video encoding process. Losses are generated during the quantization process, and discontinuities are generated at the image boundaries of the reconstructed image. The discontinuous images generated at the block boundaries are called block artifacts.

[0354] The deblocking filter is a method of alleviating block artifacts generated on block boundaries of a first image and improving encoding performance.

[0355] Block artifacts may be mitigated by performing filtering on block boundaries, and a value of a block strength (hereinafter, referred to as BS) may be determined based on at least any one of the following: whether the block is filtered by Fig.29 The intra prediction mode shown is encoded, or whether the difference in absolute values ​​of motion vectors of adjacent blocks is greater than a predefined predetermined threshold, or whether the reference images of adjacent blocks are the same as each other. When the value of BS is 0, no filtering may be performed, and when the value of BS is 1 or 2, filtering may be performed on the block boundary.

[0356] Because quantization is performed in the frequency domain, ringing artifacts are generated on the edges of objects, or the pixel values ​​become larger or smaller by a certain value compared to the original values. SAO can effectively reduce ringing artifacts by adding or subtracting a specific offset in units of blocks by considering the pattern of the first reconstructed image. According to the characteristics of the reconstructed image, SAO is configured with an edge offset (hereinafter, EO) and a band offset (BO). Edge offset is a method of adding an offset to the current sample differently according to the surrounding pixel sample pattern. Band offset reduces coding errors by adding a specific value to a group of pixels with similar pixel brightness values ​​in an area. Pixel brightness can be divided into 32 uniform bands to set pixels with similar brightness values ​​as a group. For example, 4 adjacent bands can be combined into one category. It can be set to use the same offset value in one category.

[0357] ALF (Adaptive Loop Filter) is a method of generating a second reconstructed image by applying any one of predefined filters to a first reconstructed image or performing deblocking filtering on the first reconstructed image to obtain a reconstructed image, as shown in Equation 14.

[0358] [Formula 14]

[0359]

[0360] In this case, the filter may be selected in units of pictures or in units of CTUs.

[0361] For the brightness component, Fig.30 As shown, any of 5x5, 7x7 or 9x9 diamonds can be selected. For chroma components, there may be a restriction that only 5x5 diamonds can be used.

[0362] The various embodiments of the present disclosure do not list all possible combinations but describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or may be applied by a combination of two or more.

[0363] In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof, etc. For implementation by hardware, the implementation may be performed by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), general purpose processors, controllers, microcontrollers, microprocessors, etc.

[0364] The scope of the present disclosure includes: software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that perform actions of methods according to various embodiments in a device or computer; and non-transitory computer-readable media that stores such software or instructions, etc. and is executable in a device or computer.

[0365] Industrial Applicability

[0366] The present disclosure may be used to encode / decode videos.

[0367] Additionally, the present technology may also be configured as follows.

[0368] (1) An image decoding method, comprising:

[0369] Construct a merge candidate list for the current block;

[0370] deriving motion information of the current block based on a merge index of the current block and the merge candidate list, the motion information comprising at least one of a motion vector or a reference picture index;

[0371] Obtaining a motion vector difference of the current block;

[0372] modifying the motion vector of the current block by using the motion vector difference; and

[0373] Motion compensation is performed on the current block by using the modified motion vector.

[0374] (2) The method according to (1), wherein the merge candidate list includes at least one of a spatial merge candidate or a temporal merge candidate of the current block,

[0375] wherein the motion information of the previous block belonging to the motion information list of the current block is added to the merge candidate list, and

[0376] The previous blocks are one or more blocks decoded before the current block.

[0377] (3) The method according to (1), wherein deriving the motion vector difference is selectively performed based on a predetermined flag, and

[0378] The flag indicates whether the motion vector difference is used in the merge mode.

[0379] (4) The method according to (3), wherein a maximum number of merge candidates that can be used for the current block is adaptively determined according to the flag.

[0380] (5) The method according to (1), wherein the motion vector difference is derived based on a merged offset vector, and

[0381] The merged offset vector is determined based on the length and direction of the merged offset vector.

[0382] (6) The method according to (5), wherein the length of the merged offset vector is determined by considering at least one of an encoded distance index or an accuracy of the motion vector of the current block.

[0383] (7) The method of (5), wherein the direction of the merged offset vector is determined based on an encoded direction index, and

[0384] The direction is determined to be any one of left, right, up or down.

[0385] (8) The method of (5), wherein the motion vector difference is obtained by applying a predetermined scaling factor to the merged offset vector, and

[0386] The scaling factor is determined based on a POC difference between a current picture to which the current block belongs and a reference picture of the current block.

[0387] (9) An image encoding method, comprising:

[0388] Construct a merge candidate list for the current block;

[0389] deriving motion information of the current block based on a merge index of the current block and the merge candidate list, the motion information comprising at least one of a motion vector or a reference picture index;

[0390] Obtaining a motion vector difference of the current block;

[0391] modifying the motion vector of the current block by using the motion vector difference; and

[0392] Motion compensation is performed on the current block by using the modified motion vector.

[0393] (10) A digital storage medium for storing a video stream and recording a video decoding program, wherein the video decoding program:

[0394] Constructing a merge candidate list of a current block; deriving motion information of the current block based on a merge index of the current block and the merge candidate list, wherein the motion information includes at least one of a motion vector or a reference picture index; deriving a motion vector difference of the current block; modifying the motion vector of the current block by using the motion vector difference; and performing motion compensation on the current block by using the modified motion vector.

Claims

1. An image decoding method, comprising: Construct a merge candidate list for the current block; deriving motion information of the current block based on a merge index of the current block and the merge candidate list, the motion information comprising at least one of a motion vector or a reference picture index; as well as Based on the motion information of the current block, performing inter-frame prediction of the current block, wherein the current picture including the current block is divided into a plurality of coding tree units, The current block is obtained by hierarchically dividing the coding tree units. wherein the motion vector of the current block is adaptively modified using a motion vector difference based on flag information signaled from a bitstream, The flag information being equal to 0 indicates that the motion vector difference is not used to modify the motion vector of the current block, and the flag information being equal to 1 indicates that the motion vector difference is used to modify the motion vector of the current block, The merge candidate list of the current block includes M merge candidates. Wherein, based on the flag information being equal to 0, the motion information of the current block is obtained from one of the M merge candidates in the merge candidate list, wherein, based on the flag information being equal to 1, the motion information of the current block is derived from the N merge candidates in the merge candidate list instead of one of the M merge candidates, and Among them, the value of N is smaller than the value of M.

2. The method according to claim 1, wherein: The merge candidate list includes at least one of a spatial merge candidate or a temporal merge candidate of the current block, wherein the motion information of the previous block belonging to the motion information list of the current block is added to the merge candidate list, and The previous blocks are one or more blocks decoded before the current block.

3. The method according to claim 1, wherein: deriving the motion vector difference based on the merged offset vector, and The merged offset vector is determined based on the length and direction of the merged offset vector.

4. The method according to claim 3, wherein: The length of the merge offset vector is determined based on a distance index signaled from the bitstream.

5. The method according to claim 3, wherein: determining a direction of the merge offset vector based on a direction index signaled from the bitstream, and The direction is determined to be any one of left, right, up or down.

6. The method according to claim 3, wherein: deriving the motion vector difference by applying a predetermined scaling factor to the merged offset vector, and The scaling factor is determined based on a POC difference between a current picture to which the current block belongs and a reference picture of the current block.

7. An image encoding method, comprising: Construct a merge candidate list for the current block; deriving motion information of the current block based on the merge candidate list, the motion information comprising at least one of a motion vector or a reference picture index; as well as Based on the motion information of the current block, performing inter-frame prediction of the current block, wherein the current picture including the current block is divided into a plurality of coding tree units, The current block is obtained by hierarchically dividing the coding tree units. wherein flag information indicating whether to modify the motion vector of the current block using the motion vector difference is encoded in the bit stream, The flag information being equal to 0 indicates that the motion vector difference is not used to modify the motion vector of the current block, and the flag information being equal to 1 indicates that the motion vector difference is used to modify the motion vector of the current block, The merge candidate list of the current block includes M merge candidates. Wherein, based on the flag information being equal to 0, the motion information of the current block is obtained from one of the M merge candidates in the merge candidate list, wherein, based on the flag information being equal to 1, the motion information of the current block is derived from the N merge candidates in the merge candidate list instead of one of the M merge candidates, and Among them, the value of N is smaller than the value of M.

8. A digital storage medium for storing a bit stream generated by an image encoding method, the image encoding method comprising: Construct a merge candidate list for the current block; deriving motion information of the current block based on the merge candidate list, wherein the motion information includes at least one of a motion vector or a reference picture index; and Based on the motion information of the current block, performing inter-frame prediction of the current block, wherein the current picture including the current block is divided into a plurality of coding tree units, The current block is obtained by hierarchically dividing the coding tree units. wherein flag information indicating whether to modify the motion vector of the current block using the motion vector difference is encoded in the bit stream, The flag information being equal to 0 indicates that the motion vector difference is not used to modify the motion vector of the current block, and the flag information being equal to 1 indicates that the motion vector difference is used to modify the motion vector of the current block, The merge candidate list of the current block includes M merge candidates. Wherein, based on the flag information being equal to 0, the motion information of the current block is obtained from one of the M merge candidates in the merge candidate list, wherein, based on the flag information being equal to 1, the motion information of the current block is derived from the N merge candidates in the merge candidate list instead of one of the M merge candidates, and Among them, the value of N is smaller than the value of M.

Images