Image signal encoding / decoding method and apparatus therefor

Abstract

The image decoding method according to the present application comprises the steps of: dividing a coding block into a first prediction unit and a second prediction unit; deriving a merge candidate list of the coding block; deriving first motion information of the first prediction unit and second motion information of the second prediction unit using the merge candidate list; and obtaining prediction samples in the coding block based on the first motion information and the second motion information.

Description

[0001] This application is a divisional application of the patent application with application number 201980063338.2 and titled "Image signal encoding / decoding method and apparatus therefor" filed on November 8, 2019. TECHNICAL FIELD

[0002] The present application relates to an image signal encoding / decoding method and apparatus therefor. BACKGROUND

[0003] With the trend of larger display panels, there is a growing need for video services with higher image quality. The biggest problem with high-definition video services is a significant increase in data volume, and to address this problem, research is actively being conducted to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). The JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding) and was approved on January 25, 2013, with a compression performance of about twice that of H.264 / AVC. With the rapid development of high-definition video services, the performance of HEVC has gradually revealed its limitations. SUMMARY

[0004] Technical problems to be solved

[0005] The present application relates to an image signal encoding / decoding method and apparatus therefor.

[0006] The present application relates to an image signal encoding / decoding method and apparatus therefor.

[0007] The present application relates to an image signal encoding / decoding method and apparatus therefor.

[0008] The technical problems to be achieved by the present application are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present application pertains through the following description.

[0009] Technical solution

[0010] The video signal decoding / encoding method according to the present application can include the steps of dividing a coding block into a first prediction unit and a second prediction unit, deriving a merge candidate list of the coding block, deriving first motion information of the first prediction unit and second motion information of the second prediction unit using the merge candidate list, and obtaining a prediction sample in the coding block based on the first motion information and the second motion information. In this case, whether to divide the coding block is determined according to a size of the coding block, and the first motion information of the first prediction unit is derived from a first merge candidate in the merge candidate list, and the second motion information of the second prediction unit is derived from a second merge candidate different from the first merge candidate.

[0011] In the video signal decoding / encoding method according to the present application, the coding block can not be divided when at least one of a width and a height of the coding block is greater than a threshold.

[0012] In the video signal decoding / encoding method according to the present application, the method can further include the steps of decoding first index information for specifying the first merge candidate and second index information for specifying the second merge candidate from a bitstream, and when a value of the second index information is equal to or greater than a value of the first index information, the second merge candidate has a value obtained by adding 1 to the value of the second index information as an index.

[0013] In the video signal decoding / encoding method according to the present application, when the prediction sample is included in a boundary region of the first prediction unit and the second prediction unit, the prediction sample can be derived according to a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information.

[0014] In the video signal decoding / encoding method according to the present application, a first weight value applied to the first prediction sample can be determined based on an x-axis coordinate and a y-axis coordinate of the prediction sample.

[0015] In the video signal decoding / encoding method according to the present application, a second weight value applied to the second prediction sample can be derived by subtracting the first weight value from a constant.

[0016] In the video signal decoding / encoding method according to the present application, the maximum number of merge candidates included in the merge candidate list can be determined based on whether the coding block is divided into the first prediction unit and the second prediction unit.

[0017] The features briefly summarized above for the application are merely exemplary implementations of the detailed description of the application that will be described later and do not limit the scope of the application.

[0018] Advantageous effects

[0019] According to the present application, inter prediction efficiency can be improved by providing a method of dividing a coding block into a plurality of prediction blocks and deriving motion information for each of the plurality of prediction blocks.

[0020] According to the present application, inter prediction efficiency can be improved by providing a method of deriving a merge candidate using an inter motion information list.

[0021] Effects obtainable in the present application are not limited to those described above, and other effects not mentioned will become apparent to those skilled in the art from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] FIG. 1 is a block diagram of a video encoder (encoder) according to an embodiment of the present application.

[0023] FIG. 2 is a block diagram of a video decoder (decoder) according to an embodiment of the present application.

[0024] FIG. 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present application.

[0025] FIGS. 4(a), 4(b), 4(c), 4(d), 4(e) are diagrams illustrating various division types of a coding block.

[0026] FIG. 5 is a diagram illustrating a division mode of a coding tree unit.

[0027] FIG. 6 is a flowchart of an inter prediction method according to an embodiment of the present application.

[0028] FIG. 7 is a diagram illustrating a non-linear motion of an object.

[0029] FIG. 8 is a flowchart illustrating an inter prediction method based on affine motion according to an embodiment of the present application.

[0030] FIG. 9 is a diagram illustrating an example of an affine seed vector for each affine motion model.

[0031] FIG. 10 is a diagram showing an example of an affine vector of a sub-block under a 4-parameter motion model.

[0032] FIG. 11 is a flowchart of a process of deriving motion information of a current block in merge mode.

[0033] FIG. 12 is a diagram showing an example of a candidate block used to derive a merge candidate.

[0034] FIG. 13 is a diagram showing the position of reference samples.

[0035] FIG. 14 is a diagram showing an example of a candidate block used to derive a merge candidate.

[0036] FIG. 15 is a flowchart showing the update status of a list of inter motion information.

[0037] FIG. 16 is a diagram showing an embodiment of updating a list of inter merge candidates.

[0038] FIG. 17 is a diagram showing an example of an index of a stored inter merge candidate being updated.

[0039] FIG. 18 is a diagram showing the position of a representative sub-block.

[0040] FIG. 19 shows an example of generating a list of inter motion information for different inter prediction modes.

[0041] FIG. 20 is a diagram showing an example of adding an inter merge candidate included in a long-term motion information list to a list of merge candidates.

[0042] FIG. 21 is a diagram showing an example of performing a redundancy check only for some merge candidates.

[0043] FIG. 22 is a diagram showing an example of skipping a redundancy check for a particular merge candidate.

[0044] FIG. 23 is a diagram showing an example of dividing a coding block into multiple prediction units using diagonal lines.

[0045] FIG. 24 is a diagram showing an example of dividing a coding block into two prediction units.

[0046] FIG. 25FIG. is an example illustrating a case where a coding block is divided into a plurality of prediction blocks having different sizes.

[0047] FIG. 26 FIG. is a diagram illustrating a neighboring block used for deriving a triangle merge candidate.

[0048] FIG. 27 FIG. is a diagram for describing an example of determining availability of a neighboring block of each triangle prediction unit.

[0049] FIG. 28 and FIG. 29 FIG. is an example illustrating a case where a prediction sample is derived based on a weighted sum operation of a first prediction sample and a second prediction sample. DETAILED DESCRIPTION

[0050] Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings. In the following description, a configuration of the present application will be described in the order of the following items.

[0051] Encoding and decoding of a video are performed in units of blocks. For example, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, or reconstruction can be performed on a coding block, a transform block, or a prediction block.

[0052] Hereinafter, a block to be encoded / decoded will be referred to as a "current block". For example, the current block can represent a coding block, a transform block, or a prediction block according to a current encoding / decoding process step.

[0053] In addition, the term "unit" used in the present specification means a basic unit for performing a certain encoding / decoding process, and "block" can be understood to mean an array of samples of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described later, a coding block and a coding unit can be understood to have the same meaning.

[0054] FIG. 1 FIG. is a block diagram of a video encoder (encoder) according to an embodiment of the present application.

[0055] Referring to FIG. 1 , the video encoding apparatus 100 can include an image division section 110, prediction sections 120, 125, a transform section 130, a quantization section 135, a rearrangement section 160, an entropy encoding section 165, an inverse quantization section 140, an inverse transform section 145, a filter section 150, and a memory 155.

[0056] FIG. 1The illustrated components are shown separately to represent mutually different characteristic functions in the video encoding apparatus, and do not represent that the components are constituted by separate hardware or a single software component. That is, for the components, in order to facilitate explanation, at least two of the components are combined into one component or one component is divided into a plurality of components in a manner of arranging the components, and thereby the functions are performed, and such an embodiment of integrating the components and an embodiment of separating the components also belong to the scope of the present application, as long as the essence of the present application is not deviated.

[0057] Further, some of the structural elements are not essential structural elements to perform the essence of the present application in the present application, but are optional structural elements to improve performance. The present application can be implemented by including only the components necessary to implement the essence of the present application except for the structural elements to improve performance, and a structure including only the essential structural elements except for the optional structural elements to improve performance also belongs to the scope of the present application.

[0058] The image division section 110 can divide an input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image division section 110 divides one image into a combination of a plurality of coding units, prediction units, and transform units, and can select one coding unit, prediction unit, and transform unit combination to encode the image based on a predetermined criterion (e.g., a cost function).

[0059] For example, one image can be divided into a plurality of coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used, and one video or largest coding unit can be divided into other coding units that can have child nodes corresponding to the number of divided coding units, with a coding unit as a root. A coding unit that is no longer divided according to certain restrictions will become a leaf node. That is, when it is assumed that a coding unit can be divided only in a square shape, one coding unit can be divided into at most 4 other coding units.

[0060] Hereinafter, in the embodiments of the present application, a coding unit can mean a unit that performs encoding, and can also mean a unit that performs decoding.

[0061] A prediction unit within one coding unit can be divided into at least one isometric shape such as a square or a rectangle of the same size, or one prediction unit within one coding unit can be divided into a shape and / or size different from another prediction unit.

[0062] When the prediction unit for which intra prediction is performed is not the minimum coding unit, intra prediction can be performed without being divided into a plurality of prediction units N x N.

[0063] The prediction units 120, 125 can include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It can be determined whether to use inter prediction or to perform intra prediction for a prediction unit, and specific information (e.g., an intra prediction mode, a motion vector, a reference picture, etc.) is determined based on each prediction method. In this case, the processing unit that performs prediction can be different from the processing unit that determines the prediction method and the specifics. For example, the prediction method and the prediction mode, etc. can be determined by the prediction unit, and the prediction can be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Also, the prediction mode information, the motion vector information, etc. used for prediction can be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When a specific encoding mode is used, the prediction block can not be generated by the prediction unit 120, 125, and the original block can be directly encoded and transmitted to the decoder.

[0064] The inter prediction unit 120 can predict a prediction unit based on information of at least one of a previous picture or a next picture of a current picture, and in some cases, based on information of a part of an already encoded region within the current picture. The inter prediction unit 120 can include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0065] The reference picture interpolation unit receives reference picture information from the memory 155, and can generate integer pixel or fractional pixel pixel information from the reference picture. For a luminance pixel, to generate fractional pixel pixel information in units of 1 / 4 pixel, a DCT-based 8th order interpolation filter (DCT-based Interpolation Filter) having different filter coefficients can be used. For a chrominance signal, to generate fractional pixel pixel information in units of 1 / 8 pixel, a DCT-based 4th order interpolation filter (DCT-based Interpolation Filter) having different filter coefficients can be used.

[0066] The motion prediction section can perform motion prediction based on a reference picture interpolated by the reference picture interpolation section. Methods for calculating a motion vector can use a full search-based block matching algorithm (FBMA), a three step search method (TSS), a new three-step search algorithm (NTS), and the like. A motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. A current prediction unit can be predicted in the motion prediction section by taking different motion prediction methods. Motion prediction methods can use a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, and the like.

[0067] The intra prediction section 125 can generate a prediction unit based on reference pixel information of a current block periphery as pixel information within a current picture. In a case where a neighboring block of a current prediction unit is a block for which inter prediction has been performed and a reference pixel is a pixel for which inter prediction has been performed, the reference pixel included in the block for which inter prediction has been performed can be used as reference pixel information of a periphery of a block for which intra prediction has been performed. That is, in a case where reference pixels are not available, at least one of the available reference pixels can be used instead of the unavailable reference pixel information.

[0068] In intra prediction, a prediction mode can have an angular prediction mode using reference pixel information in a prediction direction and a non-angular mode not using direction information when performing prediction. A mode for predicting luminance information and a mode for predicting chrominance information can be different, and for predicting chrominance information, intra prediction mode information used for predicting luminance information or predicted luminance signal information can be applied.

[0069] When performing intra prediction, if a size of a prediction unit is the same as a size of a transform unit, the prediction unit can be intra predicted based on pixels present on the left side of the prediction unit, pixels present on the upper left side, and pixels present on the upper side. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using reference pixels based on the transform unit. Also, intra prediction using N×N partitioning only for the smallest coding unit can be applied.

[0070] The intra prediction method can generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of adaptive intra smoothing filter for the reference pixels can be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of the prediction units existing in the periphery of the current prediction unit. In the case of predicting the prediction mode of the current prediction unit using the mode information predicted from the peripheral prediction units, if the intra prediction modes of the current prediction unit and the peripheral prediction units are the same, information indicating that the prediction modes of the current prediction unit and the peripheral prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the peripheral prediction units are different, the prediction mode information of the current block can be encoded by performing entropy encoding.

[0071] Also, a residual block including residual information that is a difference between a prediction unit predicted based on the prediction unit generated in the prediction section 120, 125 and an original block of the prediction unit can be generated. The generated residual block can be input to the transform section 130.

[0072] In the transform section 130, a transform method such as a discrete cosine transform (DCT) or a discrete sine transform (DST) can be used to transform a residual block including residual information between an original block and a prediction unit generated by the prediction section 120, 125. Among them, a DCT transform kernel includes at least one of DCT2 or DCT8, and a DST transform kernel includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block. It is also possible to skip the transformation of the residual block. A flag indicating whether to skip the transformation of the residual block can be encoded. For a residual block whose size is below a threshold, a luminance component, or a chroma component (below 4:4:4 format), transform skipping can be allowed.

[0073] The quantization section 135 can quantize values transformed into a frequency domain in the transform section 130. The quantization coefficients can vary depending on the importance of the block or the video. The values calculated in the quantization section 135 can be provided to the inverse quantization section 140 and the rearrangement section 160.

[0074] The rearrangement section 160 can perform rearrangement of coefficient values on the quantized residual values.

[0075] The rearranging unit 160 can change the 2-dimensional block-shaped coefficients into a 1-dimensional vector form by a coefficient scanning method. For example, the rearranging unit 160 can scan the DC coefficient and the coefficients of a high frequency domain using a zig-zag scan method and change them into a 1-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of the zig-zag scan, a vertical scan that scans the 2-dimensional block-shaped coefficients in a column direction and a horizontal scan that scans the 2-dimensional block-shaped coefficients in a row direction can be used. That is, which scan method to use among the zig-zag scan, the vertical scan, and the horizontal scan can be determined depending on the size of the transform unit and the intra prediction mode.

[0076] The entropy encoding unit 165 can perform entropy encoding based on the values calculated by the rearranging unit 160. For example, the entropy encoding can use various encoding methods such as an exponential Golomb, a context-adaptive variable length coding (CAVLC), a context-adaptive binary arithmetic coding (CABAC), etc.

[0077] The entropy encoding unit 165 can encode various information such as residual value coefficient information and block type information of the encoding unit originating from the rearranging unit 160 and the prediction unit 120, 125, prediction mode information, partition unit information, prediction unit information, and transport unit information, motion vector information, reference frame information, interpolation information of a block, filtering information, etc.

[0078] The coefficient values of the encoding unit input from the rearranging unit 160 can be entropy encoded in the entropy encoding unit 165.

[0079] The inverse quantization unit 140 and the inverse transform unit 145 inverse quantize the plurality of values quantized in the quantization unit 135 and inverse transform the values transformed in the transform unit 130. The residual values generated in the inverse quantization unit 140 and the inverse transform unit 145 can generate a reconstructed block by merging with the prediction unit predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction unit 120, 125.

[0080] The filter unit 150 can include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0081] A deblocking filter can remove blocking distortion generated in a reconstructed image due to boundaries between blocks. In order to determine whether to perform deblocking, it is possible to determine whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. In the case of applying a deblocking filter to a block, a strong filter or a weak filter can be applied according to a required deblocking filter strength. Also, in a process of using a deblocking filter, horizontal direction filtering and vertical direction filtering can be processed in synchronization when performing vertical filtering and horizontal filtering.

[0082] The offset correction section can correct an offset between a video in which deblocking is performed and an original video in a pixel unit. In order to perform offset correction on a designated image, a method of determining a region in which offset is to be performed after dividing pixels included in a video into a predetermined number of regions and applying offset to the corresponding region, or a method of applying offset while considering edge information of each pixel can be used.

[0083] Adaptive loop filtering (ALF) can be performed based on a value obtained by comparing a filtered reconstructed image and an original video. After dividing pixels included in a video into predetermined groups, filtering can be performed differently for each group by determining one filter to be used for the corresponding group. Information related to whether to apply adaptive loop filtering and a luminance signal can be transmitted per coding unit (CU), and a shape of an adaptive loop filter to be applied and a filter coefficient can be different for each block. Also, it is possible to apply an adaptive loop filter of the same type (fixed type) regardless of characteristics of a block to which it is applied.

[0084] The memory 155 can store a reconstructed block or image calculated by the filter section 150, and can provide the stored reconstructed block or image to the prediction sections 120 and 125 when inter prediction is performed.

[0085] FIG. 2 is a block diagram of a video decoder (decoder) according to an embodiment of the present application.

[0086] Referring to FIG. 2 , the video decoder 200 can include an entropy decoding section 210, a rearranging section 215, an inverse quantization section 220, an inverse transform section 225, a prediction section 230, a prediction section 235, a filter section 240, and a memory 245.

[0087] When a video bitstream is input from a video encoder, the input bitstream can be decoded in a step opposite to that of the video encoder.

[0088] The entropy decoding unit 210 can perform entropy decoding in steps opposite to those performed in the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc. can be applied in correspondence to the methods performed in the video encoder.

[0089] The entropy decoding unit 210 can decode information related to intra prediction and inter prediction performed by the encoder.

[0090] The rearranging unit 215 can perform rearranging based on a method of rearranging a bitstream entropy-decoded by the entropy decoding unit 210 in the encoding unit. The rearranging can be performed by reconstructing a plurality of coefficients represented in a 1-dimensional vector form into a 2-dimensional block-shaped coefficient. The rearranging unit 215 receives information related to coefficient scanning performed in the encoding unit, and can perform rearranging by a method of performing inverse scanning based on a scanning order performed in the corresponding encoding unit.

[0091] The inverse quantization unit 220 can perform inverse quantization based on a quantization parameter provided by the encoder and the coefficient values of the rearranged block.

[0092] The inverse transform unit 225 can perform inverse discrete cosine transform, inverse discrete sine transform, which belong to inverse transform of the transform performed in the transform unit, i.e., inverse transform of the discrete cosine transform, the discrete sine transform, on the quantization result performed by the video encoder. Among them, the DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform unit 225 can not perform inverse transform. The inverse transform can be performed based on a transport unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, the transform method (e.g., DCT or DST) can be selectively performed according to the prediction method, the size of the current block, the prediction direction, and the like.

[0093] The prediction units 230, 235 can generate a prediction block based on information related to prediction block generation provided by the entropy decoding unit 210 and previously decoded block or image information provided by the memory 245.

[0094] As described above, when performing intra prediction in the same manner as the operation in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixels present to the left of the prediction unit, the pixels present above and to the left, and the pixels present above. If the size of the prediction unit when performing intra prediction is different from the size of the transform unit, intra prediction can be performed using reference pixels based on the transform unit. Also, intra prediction using N x N partitioning for only the smallest coding unit can be applied.

[0095] The prediction units determination section receives various information such as prediction unit information input from the entropy decoding section 210, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, and determines whether the prediction unit is performing inter prediction or intra prediction according to the current coding unit. The inter prediction section 230 can use information required for inter prediction of the current prediction unit provided by the video encoder, and perform inter prediction on the current prediction unit based on information included in at least one of the previous picture or the next picture of the current picture to which the current prediction unit belongs. Alternatively, inter prediction can also be performed based on information of a part of the region that has been reconstructed within the current picture to which the current prediction unit belongs.

[0096] In order to perform inter prediction, it is possible to determine which of the Skip Mode, the Merge Mode, the AMVP Mode, and the Intra Block Copy Mode is the motion prediction method of the prediction unit included in the corresponding coding unit based on the coding unit.

[0097] The intra prediction section 235 can generate a prediction block based on pixel information within the current picture. When the prediction unit is a prediction unit on which intra prediction has been performed, it is possible to perform intra prediction based on the intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction section 235 can include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation section, and a DC filter. The adaptive intra smoothing filter is a section that performs filtering on the reference pixels of the current block, and it is possible to determine whether to apply the filter according to the prediction mode of the current prediction unit. It is possible to perform adaptive intra smoothing filtering on the reference pixels of the current block using the prediction mode of the prediction unit provided by the video encoder and the adaptive intra smoothing filter information. If the prediction mode of the current block is a mode in which adaptive intra smoothing filtering is not performed, the adaptive intra smoothing filter can not be applied.

[0098] For the reference pixel interpolation section, if the prediction mode of the prediction unit is a prediction mode in which intra prediction is performed based on pixel values obtained by interpolating reference pixels, the reference pixels in integer or fractional pixel units can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode in which the prediction block is generated without interpolating the reference pixels, the reference pixels can not be interpolated. If the prediction mode of the current block is a DC mode, the DC filter can generate the prediction block by filtering.

[0099] The filter section 240 can be provided with the reconstructed blocks or images. The filter section 240 can include a deblocking filter, an offset correction section, an ALF.

[0100] Information related to whether to apply the deblocking filter to the corresponding block or image and information related to whether to apply strong filtering or weak filtering when the deblocking filter is applied can be received from the video encoder. The information related to the deblocking filter provided by the video encoder is received from the deblocking filter of the video decoder, and the deblocking filter can be performed on the corresponding block by the video decoder.

[0101] The offset correction section can perform offset correction on the reconstructed video based on the type of offset correction for the video at the time of encoding and offset amount information, etc.

[0102] The ALF can be applied to the coding unit based on information related to whether to apply the ALF, ALF coefficient information, etc. provided by the encoder. Such ALF information can be provided by being included in a specific parameter set.

[0103] The memory 245 stores the reconstructed images or blocks so that the images or blocks can be used as reference images or blocks, and the reconstructed images can be provided to the output section.

[0104] FIG. 3 FIG. 1 is a diagram illustrating a basic coding tree unit of an embodiment of the present application.

[0105] The coding block having the largest size can be defined as a coding tree block. One image can be divided into a plurality of coding tree units (CTU). The coding tree unit is the largest coding unit, and can also be referred to as a largest coding unit (LCU). FIG. 3 An example of dividing one image into a plurality of coding tree units is illustrated.

[0106] The size of the coding tree unit can be defined at the image level or the sequence level. For this, information indicating the size of the coding tree unit can be signaled through the picture parameter set or the sequence parameter set.

[0107] For example, the size of the coding tree unit of the entire image within a sequence can be set to 128x128. Alternatively, either one of 128x128 or 256x256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in a first image can be set to 128x128, and the size of the coding tree unit in a second image can be set to 256x256.

[0108] A coding block can be generated by partitioning a coding tree unit. The coding block represents a basic unit for encoding / decoding processing. For example, prediction or transform can be performed per different coding block, or a prediction coding mode can be determined per different coding block. The prediction coding mode represents a method of generating a prediction image. For example, the prediction coding mode can include Intra Prediction (Intra Prediction), Inter Prediction (Inter Prediction), Current Picture Referencing (CPR, or Intra Block Copy (IBC)), or Combined Prediction. For a coding block, at least one of Intra Prediction, Inter Prediction, Current Picture Referencing, or Combined Prediction can be used to generate a prediction block related to the coding block.

[0109] Information indicating the prediction coding mode of the current block can be signaled through a bitstream. For example, the information can be a 1-bit flag indicating whether the prediction coding mode is an Intra mode or an Inter mode. Current Picture Referencing or Combined Prediction can be used only in the case where the prediction coding mode of the current block is determined to be the Inter mode.

[0110] Current Picture Referencing is used to set a current picture as a reference picture, and to obtain a prediction block of a current block from a coded / decoded area within the current picture. The current picture means a picture including the current block. Information indicating whether to apply Current Picture Referencing to the current block can be signaled through a bitstream. For example, the information can be a 1-bit flag. When the flag is true, the prediction coding mode of the current block can be determined to be Current Picture Referencing, and when the flag is false, the prediction mode of the current block can be determined to be Inter Prediction.

[0111] Alternatively, the prediction encoding mode of the current block can be determined based on the reference picture index. For example, when the reference picture index points to the current picture, the prediction encoding mode of the current block can be determined as the current picture reference. When the reference picture index points to other pictures other than the current picture, the prediction encoding mode of the current block can be determined as the inter prediction. That is, the current picture reference is a prediction method using information of an already encoded / decoded area within the current picture, and the inter prediction is a prediction method using information of already encoded / decoded other pictures.

[0112] The combined prediction indicates an encoding mode combined by two or more of the intra prediction, the inter prediction, and the current picture reference. For example, in the case where the combined prediction is applied, a first prediction block can be generated based on one of the intra prediction, the inter prediction, or the current picture reference, and a second prediction block can be generated based on another one. If the first prediction block and the second prediction block are generated, a final prediction block can be generated through an average operation or a weighted sum operation of the first prediction block and the second prediction block. Information indicating whether the combined prediction is applied can be signaled through a bitstream. The information can be a 1-bit flag.

[0113] FIGS. 4(a), 4(b), 4(c), 4(d), 4(e) are diagrams illustrating various partition types of an encoding block.

[0114] The encoding block can be partitioned into a plurality of encoding blocks based on a quad tree partition, a binary tree partition, or a ternary tree partition. The partitioned encoding block can also be partitioned into a plurality of encoding blocks again based on the quad tree partition, the binary tree partition, or the ternary tree partition.

[0115] The quad tree partition refers to a partitioning technique of partitioning a current block into 4 blocks. As a result of the quad tree partition, the current block can be partitioned into 4 square sub-partitions (refer to "SPLIT_QT" in FIG. 4(a) part).

[0116] The binary tree partition refers to a partitioning technique of partitioning a current block into 2 blocks. A process of partitioning the current block into two blocks along a vertical direction (i.e., using a vertical line crossing the current block) can be referred to as a vertical direction binary tree partition, and a process of partitioning the current block into two blocks along a horizontal direction (i.e., using a horizontal line crossing the current block) can be referred to as a horizontal direction binary tree partition. The current block can be partitioned into 2 non-square sub-partitions after the binary tree partition. "SPLIT_BT_VER" in FIG. 4(b) part indicates a vertical direction binary tree partition result, and "SPLIT_BT_HOR" in FIG. 4(c) part indicates a horizontal direction binary tree partition result.

[0117] Tri-tree partitioning refers to a partitioning technique that partitions a current block into 3 blocks. A process of partitioning a current block into three blocks along a vertical direction (i.e., using two vertical lines that cross the current block) can be referred to as vertical direction tri-tree partitioning, and a process of partitioning a current block into three blocks along a horizontal direction (i.e., using two horizontal lines that cross the current block) can be referred to as horizontal direction tri-tree partitioning. A current block can be partitioned into 3 non-square partitions after tri-tree partitioning. In this case, a partition located at the center of the current block can have a width / height that is 2 times the width / height of the other partitions. "SPLIT_TT_VER" in FIG. 4(d) denotes a vertical direction tri-tree partitioning result, and "SPLIT_TT_HOR" in FIG. 4(e) denotes a horizontal direction tri-tree partitioning result.

[0118] The number of times a coding tree unit is partitioned can be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at a sequence or picture level. As such, the maximum partitioning depth of a coding tree unit can become different according to different sequences or pictures.

[0119] Alternatively, the maximum partitioning depth for each of the plurality of partitioning techniques can be determined separately. For example, the maximum partitioning depth that allows quad-tree partitioning can be different from the maximum partitioning depth that allows binary-tree partitioning and / or tri-tree partitioning.

[0120] An encoder can signal information representing at least one of a partitioning shape or a partitioning depth of a current block through a bitstream. A decoder can determine a partitioning shape and a partitioning depth of a coding tree unit based on the information parsed from the bitstream.

[0121] FIG. 5 is a diagram illustrating a partitioning mode of a coding tree unit.

[0122] A process of partitioning a coding block using a partitioning technique such as quad-tree partitioning, binary-tree partitioning, and / or tri-tree partitioning can be referred to as multi-tree partitioning.

[0123] A coding block generated by applying multi-tree partitioning to a coding block can be referred to as a plurality of downstream coding blocks. When a partitioning depth of a coding block is k, the partitioning depth of the plurality of downstream coding blocks is set to k+1.

[0124] On the other hand, with respect to a plurality of coding blocks having a partitioning depth of k+1, a coding block having a partitioning depth of k can be referred to as an upstream coding block.

[0125] The partition type of the current coding block can be determined based on at least one of the partition shape of the upstream coding block or the partition type of the adjacent coding blocks. The adjacent coding blocks are adjacent to the current coding block and can include at least one of the current coding block's upper adjacent block, left adjacent block, or adjacent block to its upper left corner. The partition type can include at least one of whether to partition into a quadtree, whether to partition into a binary tree, the binary tree partition direction, whether to partition into a ternary tree, or the ternary tree partition direction.

[0126] To determine the shape of the coded block partition, information indicating whether the coded block has been partitioned can be sent via a bitstream signal. This information is a 1-bit flag "split_cu_flag," and when the flag is true, it indicates that the coded block has been partitioned using a multi-way tree partitioning technique.

[0127] When "split_cu_flag" is true, information indicating whether the coded block has been partitioned by a quadtree can be sent via a bitstream signal. This information is a 1-bit flag "split_qt_flag". When this flag is true, the coded block can be divided into 4 blocks.

[0128] For example, in FIG. 5 The example shown illustrates how the coding tree unit is partitioned by a quadtree to generate four coding blocks with a partition depth of 1. Furthermore, the example illustrates applying quadtree partitioning again to the first and fourth coding blocks generated as a result of the quadtree partitioning. Ultimately, four coding blocks with a partition depth of 2 can be generated.

[0129] Furthermore, a coded block with a partition depth of 3 can be generated by applying a quadtree partition to the coded block with a partition depth of 2.

[0130] When a quadtree partition is not applied to the coded block, it can be determined whether to perform a binary tree partition or a ternary tree partition by considering at least one of the following: the size of the coded block, whether the coded block is located at an image boundary, the maximum partition depth, or the partition shape of adjacent blocks. When it is determined whether to perform a binary tree partition or a ternary tree partition, information indicating the partition direction can be transmitted via a bitstream signal. This information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partition direction (vertical or horizontal) can be determined based on this flag. Alternatively, information indicating whether a binary tree partition or a ternary tree partition is applied to the coded block can be transmitted via a bitstream signal. This information can be a 1-bit flag "mtt_split_cu_binary_flag". The binary tree partition or ternary tree partition can be determined based on this flag.

[0131] For example, in FIG. 5In the illustrated example, vertical direction binary tree partitioning is applied to a coding block having a partition depth of 1, vertical direction ternary tree partitioning is applied to a left coding block generated as a result of the partitioning, and vertical direction binary tree partitioning is applied to a right coding block.

[0132] Inter prediction refers to a prediction encoding mode in which information of a previous image is used to predict a current block. For example, a block having the same position as the current block in the previous image (hereinafter referred to as a collocated block) can be set as a prediction block of the current block. Hereinafter, a prediction block generated based on a block having the same position as the current block will be referred to as a collocated prediction block.

[0133] On the other hand, if an object existing in the previous image has moved to another position in the current image, motion of the object can be used to effectively predict the current block. For example, if a moving direction and a size of the object can be known by comparing the previous image and the current image, a prediction block (or a prediction image) of the current block can be generated considering motion information of the object. Hereinafter, a prediction block generated using motion information can be referred to as a motion prediction block.

[0134] A residual block can be generated by subtracting the prediction block from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using the motion prediction block instead of the collocated prediction block, so that compression performance of the residual block can be improved.

[0135] As described above, a process of generating a prediction block using motion information can be referred to as motion compensated prediction. In most inter prediction, a prediction block can be generated based on motion compensated prediction.

[0136] Motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weighting value index. The motion vector indicates a moving direction and a size of an object. The reference picture index specifies a reference picture of the current block among a plurality of reference pictures included in a reference picture list. The prediction direction refers to any one of a uni-prediction L0, a uni-prediction L1, or a bi-prediction (L0 prediction and L1 prediction). At least one of motion information of an L0 direction or motion information of an L1 direction can be used according to a prediction direction of the current block. The bi-directional weighting value index specifies a weighting value for an L0 prediction block and a weighting value applied to an L1 prediction block.

[0137] FIG. 6 is a flowchart of an inter prediction method according to an embodiment of the present application.

[0138] Referring to FIG. 9The inter prediction method includes: determining an inter prediction mode of the current block (S601); obtaining motion information of the current block according to the determined inter prediction mode (S602); and performing motion compensation prediction of the current block based on the obtained motion information (S603).

[0139] The inter prediction mode indicates a plurality of techniques for determining the motion information of the current block, and can include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information can include a merge mode and an advanced motion vector prediction mode, and the inter prediction mode using affine motion information can include an affine merge mode and an affine motion vector prediction mode. According to the inter prediction mode, the motion information of the current block can be determined based on a neighboring block adjacent to the current block or information parsed from a bitstream.

[0140] Hereinafter, the inter prediction method using affine motion information will be described in detail.

[0141] FIG. 7 is a graph showing non-linear motion of an object.

[0142] Motion of an object within a video can be non-linear motion. For example, as shown in the example of FIG. 7 , zoom-in, zoom-out, rotation, or affine transformation of an object can occur. When non-linear motion of an object occurs, the motion of the object cannot be effectively represented by a translation motion vector. Thus, in a portion where non-linear motion of an object occurs, affine motion can be used instead of translation motion, thereby improving coding efficiency.

[0143] FIG. 8 is a flowchart illustrating an affine motion based inter prediction method according to an embodiment of the present application.

[0144] Whether to apply the affine motion based inter prediction technique to the current block can be determined based on information parsed from a bitstream. Specifically, whether to apply the affine motion based inter prediction technique to the current block can be determined based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0145] When the inter prediction technique based on affine motion is applied to the current block, an affine motion model of the current block can be determined (S1101). The affine motion model can be determined by at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent the affine motion, and the 4-parameter affine motion model uses 4 parameters to represent the affine motion.

[0146] Equation 1 is a case where the affine motion is represented using 6 parameters. The affine motion represents a translational motion of a predetermined region determined by an affine seed vector.

[0147] [Equation 1]

[0148] v x = ax - by + e

[0149] v y = cx + dy + f

[0150] When the affine motion is represented using 6 parameters, a complex motion can be represented, but the number of bits required to encode each parameter increases, which can degrade the coding efficiency. Thus, the affine motion can also be represented using 4 parameters. Equation 2 is a case where the affine motion is represented using 4 parameters.

[0151] [Equation 2]

[0152] v x = ax - by + e

[0153] v y = bx + ay + f

[0154] Information for determining the affine motion model of the current block can be encoded and can be signaled through a bitstream. For example, the information can be a 1-bit flag "affine_type_flag". A value of the flag is 0 to indicate that the 4-parameter affine motion model is applied, and a value of the flag is 1 to indicate that the 6-parameter affine motion model is applied. The flag can be encoded in units of a slice, a tile, or a block (e.g., a coding block or a coding tree unit). When the flag is signaled at a slice level, the affine motion model determined at the slice level can be applied to all blocks belonging to the slice.

[0155] Alternatively, the affine motion model of the current block can be determined based on the affine inter-frame prediction mode of the current block. For example, when applying the affine merging mode, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when applying the affine motion vector prediction mode, the information used to determine the affine motion model of the current block can be encoded and transmitted as a signal via a bitstream. For example, when applying the affine motion vector prediction mode to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0156] Next, the affine seed vector of the current block can be exported (S1102). When a 4-parameter affine motion model is selected, motion vectors at the two control points of the current block can be exported. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at the three control points of the current block can be exported. The motion vectors at the control points can be called affine seed vectors. Control points can include at least one of the top-left, top-right, or bottom-left corners of the current block.

[0157] FIG. 9 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0158] In a 4-parameter affine motion model, two related affine seed vectors can be derived from the top-left, top-right, or bottom-left corners. For example, ... FIG. 9 In the example shown in section (a), when the 4-parameter affine motion model is selected, the affine vectors can be derived by using an affine seed vector sv0 associated with the top-left corner of the current block (e.g., the top-left sample (x0, y0)) and an affine seed vector sv1 associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)). Alternatively, the affine seed vector associated with the bottom-left corner can be used instead of the affine seed vector associated with the top-left corner, or vice versa.

[0159] In a 6-parameter affine motion model, affine seed vectors related to the top-left, top-right, and bottom-left corners can be derived. For example, ... FIG. 9 In the example shown in section (b), when the 6-parameter affine motion model is selected, the affine vectors can be derived by using the affine seed vector sv0 associated with the top-left corner of the current block (e.g., the top-left sample (x0, y0)), the affine seed vector sv1 associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)), and the affine seed vector sv2 associated with the top-left corner of the current block (e.g., the top-left sample (x2, y2)).

[0160] In the embodiments described later, the affine seed vectors of the top-left control point and the top-right control point are referred to as a first affine seed vector and a second affine seed vector, respectively, under the 4-parameter affine motion model. In the embodiments described later using the first affine seed vector and the second affine seed vector, at least one of the first affine seed vector and the second affine seed vector can be replaced with an affine seed vector of the bottom-left control point (a third affine seed vector) or an affine seed vector of the bottom-right control point (a fourth affine seed vector).

[0161] Also, under the 6-parameter affine motion model, the affine seed vectors of the top-left control point, the top-right control point, and the bottom-left control point are referred to as a first affine seed vector, a second affine seed vector, and a third affine seed vector, respectively. In the embodiments described later using the first affine seed vector, the second affine seed vector, and the third affine seed vector, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with an affine seed vector of the bottom-right control point (a fourth affine seed vector).

[0162] The affine vector of each sub-block can be derived by using the affine seed vectors (S1103). Here, the affine vector represents a translational motion vector derived based on the affine seed vectors. The affine vector of the sub-block can be referred to as an affine sub-block motion vector or a sub-block motion vector.

[0163] FIG. 10 is a diagram illustrating an example of the affine vector of the sub-block under the 4-parameter motion model.

[0164] The affine vector of the sub-block can be derived based on the positions of the control points, the position of the sub-block, and the affine seed vectors. For example, Equation 3 represents an example of deriving the affine sub-block vector.

[0165] [Equation 3]

[0166]

[0167] In Equation 3, (x, y) represents the position of the sub-block. Here, the position of the sub-block represents the position of the reference sample included in the sub-block. The reference sample can be a sample located at the top-left corner of the sub-block, or a sample whose at least one of the x-axis or y-axis coordinates is located at the center position. (x0, y0) represents the position of the first control point, and (sv 0x , sv 0y ) represents the first affine seed vector. In addition, (x1, y1) represents the position of the second control point, and (sv 1x , sv 1y ) represents the second affine seed vector.

[0168] When the first control point and the second control point correspond to the upper left corner and the upper right corner of the current block, respectively, x1-x0 can be set to a value identical to the width of the current block.

[0169] Thereafter, motion-compensated prediction of each sub-block can be performed using the affine vectors of each sub-block (S1104). After performing the motion-compensated prediction, a prediction block related to each sub-block can be generated. The prediction block of the sub-block can be set as the prediction block of the current block.

[0170] Next, an inter prediction method using translational motion information will be described in detail.

[0171] Motion information of the current block can be derived from motion information of other blocks of the current block. The other blocks can be blocks that are more preferentially encoded / decoded with inter prediction than the current block. A case where the motion information of the current block is set to be identical to the motion information of the other blocks can be defined as a merge mode. Also, a case where the motion vector of the other blocks is set as a predicted value of the motion vector of the current block can be defined as a motion vector prediction mode.

[0172] FIG. 11 is a flowchart of a process of deriving motion information of a current block in a merge mode.

[0173] A merge candidate of the current block can be derived (S1101). The merge candidate of the current block can be derived from blocks that are encoded / decoded with inter prediction before the current block.

[0174] FIG. 12 is a diagram illustrating an example of a candidate block used to derive a merge candidate.

[0175] The candidate block can include at least one of a neighboring block including samples adjacent to the current block or a non-neighboring block including samples not adjacent to the current block. Hereinafter, a sample used to determine the candidate block will be designated as a reference sample. In addition, a reference sample adjacent to the current block will be referred to as a neighboring reference sample, and a reference sample not adjacent to the current block will be referred to as a non-neighboring reference sample.

[0176] The neighboring reference samples can include samples in an adjacent column of the leftmost column of the current block or samples in an adjacent row of the topmost row of the current block. For example, if the coordinates of the top-left sample of the current block are (0, 0), at least one of a block including a reference sample at position (-1, H-1), a block including a reference sample at position (W-1, -1), a block including a reference sample at position (W, -1), a block including a reference sample at position (-1, H), or a block including a reference sample at position (-1, -1) can be used as a candidate block. Referring to the drawings, the neighboring blocks indexed 0 to 4 can be used as the candidate blocks.

[0177] A non-adjacent reference sample refers to a sample having at least one of an x-axis distance or a y-axis distance from a neighboring reference sample of a current block of a predefined value. For example, at least one of a block including a reference sample having an x-axis distance of a predefined value from a left neighboring reference sample, a block including a non-adjacent sample having a y-axis distance of a predefined value from an above neighboring reference sample, or a block including a non-adjacent sample having x-axis and y-axis distances of predefined values from a top-left neighboring reference sample can be used as a candidate block. The predefined value can be an integer of 4, 8, 12, 16, etc. Referring to the drawings, at least one of blocks having indices of 5 to 26 can be used as a candidate block.

[0178] A sample not on the same vertical line, horizontal line, or diagonal line as a neighboring reference sample can be set as a non-adjacent reference sample.

[0179] FIG. 13 is a diagram illustrating positions of reference samples.

[0180] As FIG. 13 indicated in an example, an x-coordinate of an above non-adjacent reference sample can be set to be different from an x-coordinate of an above neighboring reference sample. For example, when a position of an above neighboring reference sample is (W-1, -1), a position of an above non-adjacent reference sample spaced apart from the above neighboring reference sample by N along a y-axis can be set to be ((W / 2)-1, -1-N), and a position of an above non-adjacent reference sample spaced apart from the above neighboring reference sample by 2N along the y-axis can be set to be (0, -1-2N). That is, a position of a non-adjacent reference sample can be determined based on a position of a neighboring reference sample and a distance between the neighboring reference sample.

[0181] Hereinafter, a candidate block including a neighboring reference sample among candidate blocks is referred to as a neighboring block, and a block including a non-adjacent reference sample is referred to as a non-adjacent block.

[0182] When a distance between a current block and a candidate block is greater than or equal to a threshold value, the candidate block can be set to be unusable as a merge candidate. The threshold value can be determined based on a size of a coding tree unit. For example, the threshold value can be set to a height of the coding tree unit (ctu_height), or a value obtained by adding or subtracting an offset value to or from the height of the coding tree unit (e.g., ctu_height±N). The offset value N is a value predefined in an encoder and a decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0183] When a difference between a y-axis coordinate of a current block and a y-axis coordinate of a sample included in a candidate block is greater than a threshold value, the candidate block can be determined to be unusable as a merge candidate.

[0184] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block can be set as unsuitable for merging. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, candidate blocks that include the reference sample can be set as unsuitable for merging.

[0185] If the upper boundary of the current block is adjacent to the upper boundary of a coding tree unit, multiple candidate blocks will be determined as unsuitable for merging, which will reduce the encoding / decoding efficiency of the current block. To resolve this issue, candidate blocks can be configured such that the number of candidate blocks above the current block is greater than the number of candidate blocks to the left of the current block.

[0186] FIG. 14 This is a diagram showing an example of a candidate block used to derive merge candidates.

[0187] like FIG. 14 The example shown allows setting the top block of the N blocks above the current block and the left block of the M blocks to the left of the current block as candidate blocks. In this case, by setting M to be greater than N, the number of left candidate blocks can be set to be greater than the number of top candidate blocks.

[0188] For example, the difference between the y-axis coordinate of the reference sample within the current block and the y-axis coordinate of the block above which can be used as a candidate block can be set to no more than N times the height of the current block. Additionally, the difference between the x-axis coordinate of the reference sample within the current block and the x-axis coordinate of the block to the left of which can be used as a candidate block can be set to no more than M times the width of the current block.

[0189] For example, such as FIG. 14 The example shown illustrates setting the blocks belonging to the two blocks above the current block and the five blocks belonging to the left of the current block as candidate blocks.

[0190] Merge candidates can also be derived from temporally adjacent blocks included in images different from the current block. For example, merge candidates can be derived from blocks at the same location included in images at the same location.

[0191] The motion information of the merged candidate can be set to be the same as that of the candidate block. For example, at least one of the motion vector, reference image index, prediction direction, or bidirectional weighted index of the candidate block can be set as the motion information of the merged candidate.

[0192] A list of merge candidates, including merge candidates, can be generated (S1102). The merge candidates can be classified into adjacent merge candidates derived from adjacent blocks adjacent to the current block, and non-adjacent merge candidates derived from non-adjacent blocks.

[0193] The indices of the multiple merge candidates in the merge candidate list can be assigned in a predetermined order. For example, the indices assigned to the adjacent merge candidates can have smaller values than the indices assigned to the non-adjacent merge candidates. Alternatively, the indices can be assigned based on the indices of each block as shown in FIG. 12 or FIG. 14 The indices can be assigned to each merge candidate based on the indices of each block as shown in

[0194] When multiple merge candidates are included in the merge candidate list, at least one of the multiple merge candidates can be selected (S1103). At this time, information indicating whether the motion information of the current block is derived from an adjacent merge candidate can be signaled through the bitstream. The information can be a 1-bit flag. For example, a syntax element isAdjancentMergeFlag indicating whether the motion information of the current block is derived from an adjacent merge candidate can be signaled through the bitstream. When the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block can be derived based on the adjacent merge candidate. On the other hand, when the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block can be derived based on a non-adjacent merge candidate.

[0195] Information for specifying any one of the multiple merge candidates can be signaled through the bitstream. For example, information indicating an index of any one of the merge candidates included in the merge candidate list can be signaled through the bitstream.

[0196] When isAdjacentMergeflag is 1, a syntax element merge_idx for determining any one of the adjacent merge candidates can be signaled. The maximum value of the syntax element merge_idx can be set to a value which is 1 less than the number of the adjacent merge candidates.

[0197] When isAdjacentMergeflag is 0, a syntax element NA_merge_idx for determining any one of the non-adjacent merge candidates can be signaled. The syntax element NA_merge_idx indicates a value obtained by subtracting the number of the adjacent merge candidates from the index of the non-adjacent merge candidate. The decoder can select the non-adjacent merge candidate by adding the number of the adjacent merge candidates to the index determined according to NA_merge_idx.

[0198] When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter motion information list can be added to the merge candidate list. The threshold can be the maximum number of merge candidates that the merge candidate list can include or the maximum number of merge candidates minus an offset. The offset can be an integer such as 1 or 2. The inter motion information list can include merge candidates derived based on blocks coded / decoded before the current block.

[0199] The inter motion information list includes merge candidates derived from blocks coded / decoded with inter base prediction within the current picture. For example, the motion information of the merge candidates included in the inter motion information list can be set to be the same as the motion information of the blocks coded / decoded based on inter prediction. The motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weighted value index.

[0200] For ease of illustration, the merge candidates included in the inter motion information list are referred to as inter merge candidates.

[0201] The maximum number of merge candidates that can be included in the inter motion information list can be predefined in the encoder and the decoder. For example, the maximum number of merge candidates that can be included in the inter motion information list can be 1, 2, 3, 4, 5, 6, 7, 8, or larger (e.g., 16).

[0202] Alternatively, information indicating the maximum number of merge candidates of the inter motion information list can be signaled through a bitstream. The information can be signaled at a sequence level, a picture level, or a tile level.

[0203] Alternatively, the maximum number of merge candidates of the inter motion information list can be determined according to a size of a picture, a size of a tile, or a size of a coding tree unit.

[0204] The inter motion information list can be initialized in units of a picture, a slice, a tile, a brick, a coding tree unit, or a coding tree unit line (row or column). For example, when a slice is initialized, the inter motion information list is also initialized, and the inter motion information list can not include any merge candidate.

[0205] Alternatively, information indicating whether to initialize the inter motion information list can also be signaled through a bitstream. The information can be signaled at a slice level, a tile level, a brick level, or a block level. Before the information indicates that the inter motion information list is initialized, a configured inter motion information list can be used.

[0206] Alternatively, information related to inter merge candidates can be signaled through picture parameter set or slice header. Even if a slice is initialized, the inter motion information list can include initial inter merge candidates. Thereby, inter merge candidates can be used for blocks that are first encoded / decoded within a slice.

[0207] According to the encoding / decoding order, blocks are encoded / decoded, and the blocks that are encoded / decoded based on inter prediction can be sequentially set as inter merge candidates according to the encoding / decoding order.

[0208] FIG. 15 is a flowchart illustrating an update state of the inter motion information list.

[0209] When inter prediction is performed on the current block (S1501), an inter merge candidate can be derived based on the current block (S1502). Motion information of the inter merge candidate can be set to be the same as motion information of the current block.

[0210] When the inter motion information list is empty (S1503), the inter merge candidate derived based on the current block can be added to the inter motion information list (S1504).

[0211] When the inter motion information list already includes the inter merge candidate (S1503), a redundancy check can be performed on motion information of the current block (or the inter merge candidate derived based on the current block) (S1505). The redundancy check is used to determine whether motion information of the inter merge candidate already stored in the inter motion information list is the same as motion information of the current block. The redundancy check can be performed on all inter merge candidates already stored in the inter motion information list. Alternatively, the redundancy check can be performed on inter merge candidates having an index above or below a threshold among the inter merge candidates already stored in the inter motion information list.

[0212] When the inter merge candidate having the same motion information as the current block is not included (S1506), the inter merge candidate derived based on the current block can be added to the inter motion information list (S1508). Whether the inter merge candidates are the same can be determined based on whether motion information (e.g., motion vector and / or reference picture index, etc.) of the inter merge candidates is the same.

[0213] In this case, when the maximum number of inter merge candidates has been stored in the inter motion information list (S1506), the oldest inter merge candidate is deleted (S1507), and the inter merge candidate derived based on the current block can be added to the inter motion information list (S1508).

[0214] A plurality of inter merge candidates can be identified according to respective indexes. When an inter merge candidate derived from a current block is added to an inter motion information list, the inter merge candidate is assigned a lowest index (e.g., 0), and indexes of stored inter merge candidates can be increased by 1, respectively. In this case, when a maximum number of inter merge candidates is stored in the inter motion information list, an inter merge candidate having a largest index is removed.

[0215] Alternatively, when an inter merge candidate derived from a current block is added to an inter motion information list, the inter merge candidate can be assigned a largest index. For example, when a number of inter merge candidates stored in the inter motion information list is less than a maximum value, the inter merge candidate can be assigned an index having a value identical to the number of stored inter merge candidates. Alternatively, when the number of inter merge candidates stored in the inter motion information list is equal to the maximum value, the inter merge candidate can be assigned an index of a value obtained by subtracting 1 from the maximum value. Also, an inter merge candidate having a smallest index is removed, and indexes of remaining stored inter merge candidates are decreased by 1, respectively.

[0216] FIG. 16 FIG. 1 illustrates an embodiment of updating an inter merge candidate list.

[0217] It is assumed that an inter merge candidate derived from a current block is added to an inter merge candidate list, and the inter merge candidate is assigned a largest index. Also, it is assumed that a maximum number of inter merge candidates is stored in the inter merge candidate list.

[0218] When an inter merge candidate HmvpCand[n+1] derived from a current block is added to an inter merge candidate list HmvpCandList, an inter merge candidate HmvpCand[0] having a smallest index is deleted from stored inter merge candidates, and indexes of remaining inter merge candidates are decreased by 1, respectively. In addition, an index of the inter merge candidate HmvpCand[n+1] derived from the current block can be set to a maximum value (n in the example shown in FIG. 1). FIG. 16

[0219] When the same inter merge candidate as the inter merge candidate derived based on the current block is stored (S1505), the inter merge candidate derived based on the current block can not be added to an inter motion information list (S1509).

[0220] Alternatively, as the inter merge candidate derived based on the current block is added to the inter motion information list, the same stored inter merge candidate as the inter merge candidate can also be removed. In this case, an effect that indexes of stored inter merge candidates are updated can be caused.

[0221] ​FIG. 17 is a diagram showing an example in which the index of the stored inter merge candidate is updated.

[0222] When the index of the stored inter merge candidate that is the same as the inter merge candidate mvCand derived based on the current block is hIdx, the stored inter merge candidate is deleted, and the indices of the inter merge candidates greater than hIdx can be each decreased by 1. For example, in the example shown in FIG. 17 HmvpCand[2] that is the same as mvCand is deleted from the inter motion information list HmvpCandList, and the indices of HmvpCand[3] to HmvpCand[n] are each decreased by 1.

[0223] And, the inter merge candidate mvCand derived based on the current block can be added to the end of the inter motion information list.

[0224] Alternatively, the index assigned to the stored inter merge candidate that is the same as the inter merge candidate derived based on the current block can be updated. For example, the index of the stored inter merge candidate can be changed to the minimum value or the maximum value.

[0225] The motion information of the blocks included in a predetermined region can be set to be unable to be added to the inter motion information list. For example, the inter merge candidate derived based on the motion information of the blocks included in the merge processing region cannot be added to the inter motion information list. Since the encoding / decoding order of the blocks included in the merge processing region is not defined, it is not appropriate to use the motion information of any of these blocks for the inter prediction of other blocks. Thus, the inter merge candidate derived based on the blocks included in the merge processing region can not be added to the inter motion information list.

[0226] When motion compensation prediction is performed by a sub-block unit, the inter merge candidate can be derived based on the motion information of a representative sub-block among the plurality of sub-blocks included in the current block. For example, when a sub-block merge candidate is used for the current block, the inter merge candidate can be derived based on the motion information of a representative sub-block among the sub-blocks.

[0227] The motion vector of the sub-block can be derived in the following order. First, any one of the merge candidates included in the merge candidate list of the current block can be selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. And, by adding the position (xSb, ySb) of the reference sample (e.g., the top-left sample or the middle position sample) of each sub-block within the coded block to the initial shift vector, a shifted sub-block whose position of the reference sample is (xColSb, yColSb) can be derived. Equation 4 below shows a formula for deriving the shifted sub-block.

[0228] [Equation 4]

[0229] (xColSb, yColSb) = (xSb + shVector[0] » 4, ySb + shVector[1] » 4)

[0230] Next, a motion vector of a collocated block corresponding to a center position of a sub-block including (xColSb, yColSb) is set as a motion vector of a sub-block including (xSb, ySb).

[0231] A representative sub-block can mean a sub-block including a top-left sample or a center sample of a current block.

[0232] FIG. 18 is a diagram showing a position of a representative sub-block.

[0233] FIG. 18 (a) shows an example of setting a sub-block located at a top-left of a current block as a representative sub-block, FIG. 18 (b) shows an example of setting a sub-block located at a center of a current block as a representative sub-block. When performing motion compensation prediction in a sub-block unit, an inter-frame merge candidate of a current block can be derived based on a motion vector of a sub-block including a top-left sample of the current block or a sub-block including a center sample of the current block.

[0234] Based on an inter-frame prediction mode of a current block, it can also be determined whether to use the current block as an inter-frame merge candidate. For example, a block encoded / decoded based on an affine motion model can be set as unavailable to be used as an inter-frame merge candidate. Thereby, even if the current block is encoded / decoded by inter-frame prediction, when an inter-frame prediction mode of the current block is an affine prediction mode, an inter-frame prediction motion information list is not updated based on the current block.

[0235] Alternatively, an inter-frame merge candidate can also be derived based on at least one of sub-block vectors included in sub-blocks of a block encoded / decoded based on an affine motion model. For example, a sub-block located at a top-left of a current block, a sub-block located at a center, or a sub-block located at a top-right can be used to derive an inter-frame merge candidate. Alternatively, an average of sub-block vectors of a plurality of sub-blocks can also be set as a motion vector of an inter-frame merge candidate.

[0236] Alternatively, an inter-frame merge candidate can also be derived based on an average of affine seed vectors of a block encoded / decoded based on an affine motion model. For example, an average of at least one of a first affine seed vector, a second affine seed vector, or a third affine seed vector of a current block is set as a motion vector of an inter-frame merge candidate.

[0237] Alternatively, the inter motion information lists can be configured for different inter prediction modes. For example, at least one of an inter motion information list for blocks coded / decoded by intra block copy, an inter motion information list for blocks coded / decoded based on a translational motion model, or an inter motion information list for blocks coded / decoded based on an affine motion model can be defined. Depending on the inter prediction mode of the current block, any of the multiple inter motion information lists can be selected.

[0238] FIG. 19 An example of generating inter motion information lists for different inter prediction modes is shown.

[0239] When a block is coded / decoded based on a non-affine motion model, an inter merge candidate mvCand derived based on the block can be added to an inter non-affine motion information list HmvpCandList. On the other hand, when a block is coded / decoded based on an affine motion model, an inter merge candidate mvAfCand derived based on the block can be added to an inter affine motion information list HmvpAfCandList.

[0240] An affine seed vector of a block can be stored in an inter merge candidate derived from a block coded / decoded based on an affine motion model. Thereby, the inter merge candidate can be used as a merge candidate for deriving an affine seed vector of the current block.

[0241] In addition to the described inter motion information lists, further inter motion information lists can be defined. In addition to the described inter motion information lists (hereinafter referred to as first inter motion information lists), long-term motion information lists (hereinafter referred to as second inter motion information lists) can be defined. Therein, the long-term motion information lists comprise long-term merge candidates.

[0242] When both the first inter motion information lists and the second inter motion information lists are empty, inter merge candidates can first be added to the second inter motion information lists. After the number of inter merge candidates available in the second inter motion information lists reaches a maximum number, inter merge candidates can be added to the first inter motion information lists.

[0243] Alternatively, one inter merge candidate can also be added to both the second inter motion information lists and the first inter motion information lists.

[0244] In this case, the second inter motion information lists can no longer be updated. Alternatively, the second inter motion information lists can be updated when the decoded area is above a predetermined ratio of a slice. Alternatively, the second inter motion information lists can be updated every N coding tree unit rows.

[0245] On the other hand, the first inter motion information list can be updated every time a block encoded / decoded with inter prediction is generated. However, the inter merge candidate added to the second inter motion information list can be set not to be used to update the first inter motion information list.

[0246] Information for selecting either the first inter motion information list or the second inter motion information list can be signaled through a bitstream. When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidate included in the inter motion information list indicated by the information can be added to the merge candidate list.

[0247] Alternatively, the inter motion information list can be selected based on the size, shape, inter prediction mode, bi-prediction or not, motion vector refinement or not, or triangle partition or not of the current block.

[0248] Alternatively, even when the number of merge candidates included in the merge candidate list is less than the maximum number of merges, the inter merge candidate included in the second inter motion information list can be added to the merge candidate list, although the inter merge candidate included in the first inter motion information list is added to the merge candidate list.

[0249] FIG. 20 FIG. 1 is a diagram illustrating an example of adding an inter merge candidate included in a long-term motion information list to a merge candidate list.

[0250] When the number of merge candidates included in the merge candidate list is less than the maximum number, the inter merge candidate included in the first inter motion information list HmvpCandList can be added to the merge candidate list. Even when the number of merge candidates included in the merge candidate list is less than the maximum number, although the inter merge candidate included in the first inter motion information list is added to the merge candidate list, the inter merge candidate included in the long-term motion information list HmvpLTCandList can be added to the merge candidate list.

[0251] The inter merge candidate can be set to include additional information in addition to the motion information. For example, size, shape, or partition information of a block can be added in the inter merge candidate. When constructing a merge candidate list of a current block, only the inter merge candidate having the same or similar size, shape, or partition information as the current block can be used in the inter merge candidate, or the inter merge candidate having the same or similar size, shape, or partition information as the current block can be preferentially added to the merge candidate list.

[0252] Alternatively, the inter motion information list can be generated for different block sizes, shapes, or partition information. The inter motion information list corresponding to the shape, size, or partition information of the current block can be used to generate the merge candidate list of the current block.

[0253] When the number of the merge candidates included in the merge candidate list of the current block is less than a threshold, the inter merge candidates included in the inter motion information list can be added to the merge candidate list. The adding process is performed in ascending order or descending order of the index. For example, the inter merge candidate with the largest index can be added to the merge candidate list.

[0254] When the inter merge candidates included in the inter motion information list are to be added to the merge candidate list, a redundancy check between the inter merge candidates and the merge candidates already stored in the merge candidate list can be performed.

[0255] The redundancy check can also be performed only on some of the inter merge candidates included in the inter motion information list. For example, the redundancy check can be performed only on the inter merge candidates with an index above or below a threshold. Alternatively, the redundancy check can be performed only on the N merge candidates with the largest index or the N merge candidates with the smallest index.

[0256] Alternatively, the redundancy check can be performed only on some of the merge candidates already stored in the merge candidate list. For example, the redundancy check can be performed only on the merge candidates with an index above or below a threshold or the merge candidates derived from blocks at specific positions. The specific positions can include at least one of a left neighboring block, an above neighboring block, a top-right neighboring block, or a bottom-left neighboring block of the current block.

[0257] FIG. 21 FIG. 4 is a diagram illustrating an example of performing the redundancy check only on some of the merge candidates.

[0258] When the inter merge candidate HmvpCand[j] is to be added to the merge candidate list, a redundancy check between the inter merge candidate and the two merge candidates with the largest index, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], can be performed. NumMerge can represent the number of the spatial merge candidates and the temporal merge candidates available.

[0259] Unlike the example shown in the figure, when an inter merge candidate HmvpCand[j] is to be added to the merge candidate list, a redundancy check between the inter merge candidate and the 2 merge candidates with the smallest index can also be performed. For example, it can be confirmed whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, the redundancy check can be performed only on the merge candidates derived from a specific location. For example, the redundancy check can be performed only on at least one of the merge candidates derived from the neighboring block located at the left side of the current block or the merge candidates derived from the neighboring block located above the current block. When there is no merge candidate derived from a specific location in the merge candidate list, the inter merge candidate can be added to the merge candidate list without the redundancy check.

[0260] In the case where the same merge candidate as the first inter merge candidate is found, when the redundancy check is performed on the second inter merge candidate, the redundancy check of the same merge candidate as the first inter merge candidate can be skipped.

[0261] FIG. 22 is a figure showing an example of skipping the redundancy check of a specific merge candidate.

[0262] When an inter merge candidate HmvpCand[i] with index i is to be added to the merge candidate list, a redundancy check between the inter merge candidate and the merge candidates already stored in the merge candidate list can be performed. In this case, in the case where the same merge candidate mergeCandList[j] as the inter merge candidate HmvpCand[i] is found, the inter merge candidate HmvpCand[i] will not be added to the merge candidate list, and a redundancy check between the inter merge candidate HmvpCand[i-1] with index i-1 and the merge candidate can be performed. In this case, the redundancy check between the inter merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be skipped.

[0263] For example, in the example shown in FIG. 22 In the example shown in FIG. 6, it is determined that HmvpCand[i] is the same as mergeCandList[2]. Thus, HmvpCand[i] is not added to the merge candidate list, and the redundancy check can be performed on HmvpCand[i-1]. In this case, the redundancy check between HvmpCand[i-1] and mergeCandList[2] can be skipped.

[0264] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, at least one of a pair-wise merge candidate or a zero merge candidate can be included in addition to the inter merge candidate. The pair-wise merge candidate refers to a merge candidate having an average value of motion vectors of two or more merge candidates as a motion vector, and the zero merge candidate refers to a merge candidate having a motion vector of 0.

[0265] The merge candidate list of the current block can add merge candidates in the following order.

[0266] spatial merge candidate - temporal merge candidate - inter merge candidate - (inter affine merge candidate) - pair-wise merge candidate - zero merge candidate

[0267] The spatial merge candidate refers to a merge candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal merge candidate refers to a merge candidate derived from a previous reference picture. The inter affine merge candidate list indicates an inter merge candidate derived from a block encoded / decoded in an affine motion model.

[0268] The inter motion information list can also be used in the advanced motion vector prediction mode. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold value, an inter merge candidate included in the inter motion information list is set as a motion vector prediction candidate related to the current block. Specifically, the motion vector of the inter merge candidate is set as the motion vector prediction candidate.

[0269] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as a motion vector predictor of the current block. Thereafter, after the motion vector residual value of the current block is decoded, the motion vector of the current block can be obtained by adding the motion vector predictor and the motion vector residual value.

[0270] The motion vector prediction candidate list of the current block can be constructed in the following order.

[0271] spatial motion vector prediction candidate - temporal motion vector prediction candidate - inter decoded region merge candidate - (inter decoded region affine merge candidate) - zero motion vector prediction candidate

[0272] The spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from a previous reference picture. The inter affine merge candidate list indicates an inter motion vector prediction candidate derived from a block encoded / decoded in an affine motion model. The zero motion vector prediction candidate indicates a candidate having a value of 0 for a motion vector.

[0273] The coding block can be divided into a plurality of prediction units, and prediction can be performed on the divided prediction units respectively. Among them, the prediction unit represents a basic unit for prediction.

[0274] The coding block can be divided using at least one of a vertical line, a horizontal line, an oblique line, or a diagonal line. Information for determining at least one of the number, the angle, or the position of the line dividing the coding block can be signaled through a bitstream. For example, information indicating any one of the division type candidates of the coding block can be signaled through the bitstream, or information specifying any one of the plurality of line candidates dividing the coding block can be signaled through the bitstream. Alternatively, information for determining the number or the type of the line candidates dividing the coding block can be signaled through the bitstream. For example, using a 1-bit flag, it can be determined whether an oblique line having an angle greater than a diagonal line and / or an oblique line having an angle less than a diagonal line is available as a line candidate.

[0275] Alternatively, based on at least one of an intra prediction mode of the coding block, an inter prediction mode, a position of a merge candidate that can be used, or a division type of a neighboring block, at least one of the number, the angle, or the position of the line dividing the coding block can be adaptively determined.

[0276] If the coding block is divided into a plurality of prediction units, intra prediction or inter prediction can be performed on each of the divided prediction units.

[0277] FIG. 23 FIG. 1 is a diagram illustrating an example of dividing a coding block into a plurality of prediction units using a diagonal line.

[0278] As illustrated in the examples of FIG. 23 (a) and FIG. 23 (b), the coding block can be divided into two triangular prediction units using a diagonal line.

[0279] In FIG. 23 (a) and FIG. 23 (b), it is illustrated that the coding block is divided into two prediction units using a diagonal line connecting two vertices of the coding block. However, the coding block can be divided into two prediction units using an oblique line of which at least one end does not pass through a vertex of the coding block.

[0280] FIG. 24 FIG. 1 is a diagram illustrating an example of dividing a coding block into a plurality of prediction units using a diagonal line.

[0281] As illustrated in the examples of FIG. 24 (a) and FIG. 24 (b), the coding block can be divided into two prediction units using an oblique line of which both ends respectively contact an upper boundary and a lower boundary of the coding block.

[0282] Alternatively, as illustrated in the examples of FIG. 24 (c) andFIG. 24 (d) In the example shown, the coding block can be divided into two prediction units using a diagonal line contacting the left boundary and the right boundary of the coding block, respectively.

[0283] Alternatively, the coding block can be divided into two prediction blocks having different sizes. For example, the diagonal line dividing the coding block is set to contact two boundary surfaces forming one vertex, whereby the coding block can be divided into two prediction units having different sizes.

[0284] FIG. 25 A diagram showing an example of dividing a coding block into a plurality of prediction blocks having different sizes.

[0285] As shown in FIG. 25 (a) and FIG. 25 (b), a coding block can be divided into two prediction units having different sizes by setting a diagonal line connecting the upper left corner or the lower right corner of the coding block to pass through the left boundary, the right boundary, the upper boundary, or the lower boundary of the coding block, rather than the upper left corner or the lower right corner of the coding block.

[0286] Alternatively, as shown in FIG. 25 (c) and FIG. 25 (d), a coding block can be divided into two prediction units having different sizes by setting a diagonal line connecting the upper right corner or the lower left corner of the coding block to pass through the left boundary, the right boundary, the upper boundary, or the lower boundary of the coding block, rather than the upper left corner or the lower right corner of the coding block.

[0287] Each prediction unit generated by dividing a coding block is referred to as an "Nth prediction unit". For example, in the example shown in FIG. 23 to FIG. 25 PU1 can be defined as a first prediction unit, and PU2 can be defined as a second prediction unit. The first prediction unit can refer to a prediction unit including a sample located at the lower left or a sample located at the upper left within a coding block, and the second prediction unit can refer to a prediction unit including a sample located at the upper right or a sample located at the lower right within the coding block.

[0288] In contrast to the above, a prediction unit including a sample located at the upper right or a sample located at the lower right within a coding block can be defined as a first prediction unit, and a prediction unit including a sample located at the lower left or a sample located at the upper left within the coding block can be defined as a second prediction unit.

[0289] The embodiments described later mainly explain an example of division using a diagonal line. In particular, a process of dividing a coding block into 2 prediction units using a diagonal line is referred to as diagonal line division or triangle division, and a prediction unit generated based on diagonal line division is referred to as a triangle prediction unit. However, of course, a division example using a diagonal line having an angle different from a vertical line, a horizontal line, or a diagonal line can be used in the embodiments described later.

[0290] Whether to apply diagonal line division to a coding block can be determined based on at least one of a slice type, a maximum number of merge candidates that a merge candidate list can include, a size of a coding block, a shape of a coding block, a prediction encoding mode of a coding block, or a partition type of a parent node.

[0291] For example, whether to apply diagonal line division to a coding block can be determined based on whether a current slice is a B type. Diagonal line division can be allowed only when the current slice is a B type.

[0292] Alternatively, whether to apply diagonal line division to a coding block can be determined based on whether a maximum number of merge candidates included in a merge candidate list is more than 2. Diagonal line division can be allowed only in a case where the maximum number of merge candidates included in the merge candidate list is more than 2.

[0293] Alternatively, when at least one of a width or a height of hardware is greater than 64, a disadvantage that a data processing unit of 64x64 size is redundantly accessed can occur. Thus, when at least one of a width or a height of a coding block is greater than a threshold value, it can be possible not to divide the coding block into a plurality of prediction blocks. For example, when at least one of a width and a height of a coding block is greater than 64 (e.g., when at least one of the width and the height is 128), diagonal line division can not be used.

[0294] Alternatively, diagonal line division can not be allowed for a coding block for which a number of samples is greater than a threshold value, in consideration of a maximum number of samples that can be processed at the same time in a hardware implementation. For example, diagonal line division can not be allowed for a coding tree block for which a number of samples is greater than 4096.

[0295] Alternatively, diagonal line division can not be allowed for a coding block for which a number of samples included in the coding block is less than a threshold value. For example, diagonal line division can be set not to be applied to a coding block when a number of samples included in the coding block is less than 64.

[0296] Alternatively, whether to apply diagonal line division to a coding block can be determined based on whether a width-to-height ratio of the coding block is less than a first threshold value or whether the width-to-height ratio of the coding block is greater than a second threshold value. Here, a width-to-height ratio whRatio of a coding block can be determined as a ratio of a width CbW to a height CbH of the coding block, as shown in Equation 5 below.

[0297] [Formula 5]

[0298] whRatio = CbW / CbH

[0299] The second threshold can be the inverse of the first threshold. For example, when the first threshold is k, the second threshold can be 1 / k.

[0300] The diagonal partition can be applied to the coding block only when the width-to-height ratio of the coding block is between the first threshold and the second threshold.

[0301] Alternatively, the triangular partition can be used only when the width-to-height ratio of the coding block is less than the first threshold or greater than the second threshold. For example, when the first threshold is 16, the diagonal partition can not be allowed for a coding block of size 64x4 or 4x64.

[0302] Alternatively, based on the partition type of the parent node, it can be determined whether the diagonal partition is allowed. For example, when the coding block as the parent node is partitioned based on the quad-tree partition, the diagonal partition can be applied to the coding block as the leaf node. On the other hand, when the coding block as the parent node is partitioned based on the binary or ternary tree partition, the coding block as the leaf node is set to not allow the diagonal partition.

[0303] Alternatively, based on the prediction coding mode of the coding block, it can be determined whether the diagonal partition is allowed. For example, when the coding block is coded with intra prediction, the diagonal partition can be allowed only in the case where the coding block is coded with inter prediction or in the case where the coding block is coded with a predefined inter prediction mode. The predefined inter prediction mode can represent at least one of a merge mode, an advanced motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0304] Alternatively, based on the size of the parallel processing region, it can be determined whether the diagonal partition is allowed. For example, when the size of the coding block is greater than the size of the parallel processing region, it is possible that the diagonal partition is not used.

[0305] It is also possible to consider two or more of the enumerated conditions to determine whether the diagonal partition is applied to the coding block.

[0306] As another example, information indicating whether the diagonal partition is applied to the coding block can be signaled through a bitstream. The information can be signaled at a sequence level, a picture level, a slice level, or a block level. For example, a flag triangle_partition_flag indicating whether the triangular partition is applied to the coding block can be signaled at a coding block level.

[0307] When it is determined to apply diagonal partitioning to the coding block, information indicating the number of lines or the position of the lines partitioning the coding block can be signaled through the bitstream.

[0308] For example, when the coding block is partitioned by a diagonal line, information indicating the direction of the diagonal line partitioning the coding block can be signaled through the bitstream. For example, a flag triangle_partition_type_flag indicating the direction of the diagonal line can be signaled through the bitstream. The flag indicates whether the coding block is partitioned by a diagonal line connecting the top-left and bottom-right or by a diagonal line connecting the top-right and bottom-left. Partitioning the coding block by a diagonal line connecting the top-left and bottom-right can be referred to as a left triangle partition type, and partitioning the coding block by a diagonal line connecting the top-right and bottom-left can be referred to as a right triangle partition type. For example, a value of 0 of the flag indicates that the partition type of the coding block is the left triangle partition type, and a value of 1 of the flag indicates that the partition type of the coding block is the right triangle partition type.

[0309] In addition, information indicating whether the prediction units have the same size or information indicating the position of the diagonal line used to partition the coding block can be signaled through the bitstream. For example, if the information indicating the size of the prediction units indicates that the prediction units have the same size, encoding of the information indicating the position of the diagonal line is skipped, and the coding block can be partitioned into two prediction units using a diagonal line passing through two vertices of the coding block. On the other hand, when the information indicating the size of the prediction units indicates that the prediction units do not have the same size, the position of the diagonal line used to partition the coding block can be determined based on the information indicating the position of the diagonal line. For example, when the left triangle partition type is applied to the coding block, the position information can indicate whether the diagonal line is in contact with the left boundary and the lower boundary or the upper boundary and the right boundary of the coding block. Alternatively, when the right triangle partition type is applied to the coding block, the position information can indicate whether the diagonal line is in contact with the right boundary and the lower boundary or the upper boundary and the left boundary of the coding block.

[0310] The information indicating the partition type of the coding block can be signaled at the coding block level. Thereby, the partition type can be determined for different coding blocks to which diagonal partitioning is applied.

[0311] As another example, for a sequence, a picture, a slice, a tile, or a coding tree unit, information indicating the partition type can be signaled. In this case, the partition type of the coding block to which diagonal partitioning is applied can be set to be the same within the sequence, the picture, the slice, the tile, or the coding tree unit.

[0312] Alternatively, for a first coding unit within a coding tree unit to which diagonal partitioning is applied, information used to determine the partition type is encoded and signaled, and a second and subsequent coding unit to which diagonal partitioning is applied is set to use the same partition type as the first coding unit.

[0313] As another example, a partition type of the coding block can be determined based on a partition type of a neighboring block. The neighboring block can include at least one of a neighboring block adjacent to a top-left corner of the coding block, a neighboring block adjacent to a top-right corner, a neighboring block adjacent to a bottom-left corner, a neighboring block located above, or a neighboring block located to the left. For example, the partition type of the current block can be set to be the same as the partition type of the neighboring block. Alternatively, the partition type of the current block can be determined based on whether the top-left neighboring block applies a left-triangle partition type, whether the top-right neighboring block or the bottom-left neighboring block applies a right-triangle partition type.

[0314] In order to perform motion prediction compensation on the first triangle prediction unit and the second triangle prediction unit, motion information of the first triangle prediction unit and the second triangle prediction unit can be derived, respectively. In this case, the motion information of the first triangle prediction unit and the second triangle prediction unit can be derived from merge candidates included in a merge candidate list. In order to distinguish a general merge candidate list and a merge candidate list used when deriving motion information of a triangle prediction unit, the merge candidate list used when deriving motion information of a triangle prediction unit is referred to as a triangle merge candidate list, and a merge candidate included in the triangle merge candidate list is referred to as a triangle merge candidate. However, the aforementioned merge candidate derivation method and merge candidate list construction method are also included in the spirit of the present application.

[0315] Information for determining a number of maximum triangle merge candidates that the triangle merge candidate list can include can be signaled through a bitstream. The information can represent a difference between a number of maximum merge candidates that a merge candidate list can include and a number of maximum triangle merge candidates that a triangle merge candidate list can include.

[0316] A triangle merge candidate can be derived from spatial neighboring blocks and temporal neighboring blocks of a coding block.

[0317] FIG. 26 FIG. 1 is a diagram illustrating neighboring blocks used to derive a triangle merge candidate.

[0318] The triangular merge candidate can be derived using at least one of a neighboring block located above the coding block, a neighboring block located left of the coding block, or a co-located block included in a different picture than the coding block. The upper neighboring block can include at least one of a block including a sample (xCb+CbW-1, yCb-1) located above the coding block, a block including a sample (xCb+CbW, yCb-1) located above the coding block, or a block including a sample (xCb-1, yCb-1) located above the coding block. The left neighboring block can include at least one of a block including a sample (xCb-1, yCb+CbH-1) located left of the coding block or a block including a sample (xCb-1, yCb+CbH) located left of the coding block. The co-located block can be determined as either one of a block including a sample (xCb+CbW, yCb+CbH) adjacent to a top right corner of the coding block within a co-located picture or a block including a sample (xCb / 2, yCb / 2) located at a center of the coding block.

[0319] The neighboring blocks can be searched in a predefined order, and the triangular merge candidates can be constructed into a triangular merge candidate list in the predefined order. For example, the triangular merge candidate list can be constructed by searching the triangular merge candidates in an order of B1, A1, B0, A0, C0, B2, and C1.

[0320] Motion information of the triangular prediction unit can be derived based on the triangular merge candidate list. That is, the triangular prediction unit can share one triangular merge candidate list.

[0321] To derive the motion information of the triangular merge unit, information for specifying at least one of the triangular merge candidates included in the triangular merge candidate list can be signaled through a bitstream. For example, index information merge_triangle_idx for specifying at least one of the triangular merge candidates can be signaled through the bitstream.

[0322] The index information can specify a combination of the merge candidate of the first triangular prediction unit and the merge candidate of the second triangular prediction unit. For example, Table 1 below shows an example of the combination of the merge candidates according to the index information merge_triangle_idx.

[0323] Table 1

[0324] merge_triangle_idx 0 1 2 3 4 5 6 7 8 first prediction unit 1 0 0 0 2 0 0 1 3 second prediction unit 0 1 2 1 0 3 4 0 0 merge_triangle_idx 9 10 11 12 13 14 15 16 17 first prediction unit 4 0 1 1 0 0 1 1 1 second prediction unit 0 2 2 2 4 3 3 4 4 merge_triangle_idx 18 19 20 21 22 23 24 25 26 first triangle prediction unit 1 2 2 2 4 3 3 3 4 second triangle prediction unit 3 1 0 1 3 0 2 4 0 merge_triangle_idx 27 28 29 30 31 32 33 34 35 first triangle prediction unit 3 2 4 4 2 4 3 4 3 second triangle prediction unit 1 3 1 1 3 2 2 3 1 merge_triangle_idx 36 37 38 39 first triangle prediction unit 2 2 4 3 second triangle prediction unit 4 4 2 4

[0325] The value of the index information merge_triangle_idx of 1 indicates that the motion information of the first triangle prediction unit is derived from the merge candidate of index 1 and the motion information of the second triangle prediction unit is derived from the merge candidate of index 0. The triangle merge candidate used to derive the motion information of the first triangle prediction unit and the triangle merge candidate used to derive the motion information of the second triangle prediction unit can be determined by the index information merge_triangle_idx.

[0326] The partition type of the coding block to which the diagonal partition is applied can also be determined according to the index information. That is, the index information can specify the combination of the merge candidate of the first triangle prediction unit, the merge candidate of the second triangle prediction unit, and the partition direction of the coding block. When the partition type of the coding block is determined according to the index information, the information triangle_partition_type_flag indicating the direction of the diagonal partitioning the coding block can not be encoded. Table 2 shows the partition type of the coding block with respect to the index information merge_triangle_idx.

[0327] Table 2

[0328] merge_triangle_idx 0 1 2 3 4 5 6 7 8 TriangleDir 0 1 1 0 0 1 1 1 0 merge_triangle_idx 9 10 11 12 13 14 15 16 17 TriangleDir 0 0 0 1 0 0 0 0 1 merge_triangle_idx 18 19 20 21 22 23 24 25 26 TriangleDir 1 1 1 0 0 1 1 1 1 merge_triangle_idx 27 28 29 30 31 32 33 34 35 TriangleDir 1 1 1 0 0 1 0 1 0 merge_triangle_idx 36 37 38 39 TriangleDir 0 1 0 0

[0329] The variable TriangleDir is 0 when the coding block applies the left triangle partition type and is 1 when the coding block applies the right triangle partition type. By combining Table 1 and Table 2, it can be set to specify the combination of the merge candidate of the first triangle prediction unit, the merge candidate of the second triangle prediction unit, and the partition direction of the coding block according to the index information merge_triangle_idx.

[0330] As another example, index information for only any one of the first triangle prediction unit and the second triangle prediction unit can be signaled, and an index of a triangle merge candidate for the other one of the first triangle prediction unit and the second triangle prediction unit can be determined based on the index information. For example, a triangle merge candidate of the first triangle prediction unit can be determined based on index information merge_triangle_idx representing an index of any one of the triangle merge candidates. And a triangle merge candidate of the second triangle prediction unit can be specified based on the merge_triangle_idx. For example, the triangle merge candidate of the second triangle prediction unit can be derived by adding or subtracting an offset to or from the index information merge_triangle_idx. The offset can be an integer such as 1 or 2. For example, the triangle merge candidate of the second triangle prediction unit can be determined as a triangle merge candidate having a value of merge_triangle_idx plus 1 as an index. When the merge_triangle_idx indicates a triangle merge candidate having a largest index value among the triangle merge candidates, the motion information of the second triangle prediction unit can be derived from a triangle merge candidate having an index of 0 or a triangle merge candidate having a value obtained by subtracting 1 from the merge_triangle_idx as an index.

[0331] Alternatively, the motion information of the second triangle prediction unit can be derived from a triangle merge candidate having a same reference picture as a triangle merge candidate of the first triangle prediction unit specified according to the index information. Wherein the triangle merge candidate having a same reference picture as the triangle merge candidate of the first triangle prediction unit can represent at least one of a L0 reference picture or a L1 reference picture of the triangle merge candidate having a same triangle merge candidate as the triangle merge candidate of the first triangle prediction unit. When there are multiple triangle merge candidates having a same reference picture as the triangle merge candidate of the first triangle prediction unit, any one of the triangle merge candidates can be selected based on at least one of whether the merge candidate includes bi-directional motion information or a difference value between an index of the merge candidate and the index information.

[0332] As another example, for the first triangle prediction unit and the second triangle prediction unit, index information can be signaled respectively. For example, first index information 1st_merge_idx for a triangle merge candidate of the first triangle prediction unit and second index information 2nd_merge_idx for a triangle merge candidate of the second triangle prediction unit can be signaled through a bitstream. Motion information of the first triangle prediction unit can be derived from the triangle merge candidate determined based on the first index information 1st_merge_idx, and motion information of the second triangle prediction unit can be derived from the triangle merge candidate determined based on the second index information 2nd_merge_idx.

[0333] The first index information 1st_merge_idx can indicate an index of any one of the triangle merge candidates included in the triangle merge candidate list. The triangle merge candidate of the first triangle prediction unit can be determined as the triangle merge candidate indicated by the first index information 1st_merge_idx.

[0334] The triangle merge candidate indicated by the first index information 1st_merge_idx is set to be unavailable as the triangle merge candidate of the second triangle prediction unit. Thus, the second index information 2nd_merge_idx of the second triangle prediction unit can indicate an index of any one of the remaining triangle merge candidates other than the triangle merge candidate indicated by the first index information. When a value of the second index information 2nd_merge_idx is less than a value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangle prediction unit can be determined as the triangle merge candidate having the index indicated by the second index information 2nd_merge_idx. On the other hand, when the value of the second index information 2nd_merge_idx is the same as or greater than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangle prediction unit can be determined as the triangle merge candidate having a value of the value of the second index information 2nd_merge_idx plus 1 as the index.

[0335] Alternatively, the signaling of the second index information can be determined based on the number of triangle merge candidates included in the triangle merge candidate list. For example, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is no more than 2, the signaling of the second index information can be skipped. When the signaling of the second index information is skipped, the second triangle merge candidate can be derived by adding or subtracting an offset to the first index information. For example, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information is index 0, the second triangle merge candidate can be derived by adding 1 to the first index information. Alternatively, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information is 1, the second triangle merge candidate can be derived by subtracting 1 from the first index information.

[0336] Alternatively, when the signaling of the second index information is skipped, the second index information can be set to a default value. The default value can be 0. By comparing the first index information and the second index information, the second triangle merge candidate can be derived. For example, when the second index information is smaller than the first index information, the merge candidate with index 0 is set as the second triangle merge candidate, and when the second index information is the same or larger than the first index information, the merge candidate with index 1 is set as the second triangle merge candidate.

[0337] When the triangle merge candidate has uni-directional motion information, the uni-directional motion information of the triangle merge candidate is set as the motion information of the triangle prediction unit. On the other hand, when the triangle merge candidate has bi-directional motion information, only one of the L0 motion information or the L1 motion information is set as the motion information of the triangle prediction unit. Which one of the L0 motion information or the L1 motion information to be obtained can be determined based on the index of the triangle merge candidate or the motion information of another triangle prediction unit.

[0338] For example, when the index of the triangle merge candidate is even, the L0 motion information of the triangle prediction unit is set to 0, and the L1 motion information of the triangle merge candidate is set to the L1 motion information of the triangle prediction unit. On the other hand, when the index of the triangle merge candidate is odd, the L1 motion information of the triangle prediction unit is set to 0, and the L0 motion information of the triangle merge candidate is set to 0. In contrast to the above, when the index of the triangle merge candidate is even, the L0 motion information of the triangle merge candidate is set to the L0 motion information of the triangle prediction unit, and when the index of the triangle merge candidate is odd, the L1 motion information of the triangle merge candidate can also be set to the L1 motion information of the triangle prediction unit. Alternatively, for the first triangle prediction unit, when the triangle merge candidate is even, the L0 motion information of the triangle merge candidate is set to the L0 motion information of the first triangle prediction unit, and on the other hand, for the second triangle prediction unit, when the triangle merge candidate is odd, the L1 motion information of the triangle merge candidate is set to the L1 motion information of the second triangle prediction unit.

[0339] Alternatively, when the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit is set to 0, and the L1 motion information of the triangle merge candidate is set to the L1 information of the second triangle prediction unit. On the other hand, when the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit is set to 0, and the L0 motion information of the triangle merge candidate is set to the L0 motion signal of the second triangle prediction unit.

[0340] The triangle merge candidate list used to derive the motion information of the first triangle prediction unit and the triangle merge candidate list used to derive the motion information of the second triangle prediction unit can also be set to be different.

[0341] For example, when the triangle merge candidate used to derive the motion information of the first triangle prediction unit in the triangle merge candidate list is specified based on index information related to the first triangle prediction unit, the motion information of the second triangle prediction unit can be derived using the triangle merge list including the remaining triangle merge candidates other than the triangle merge candidate pointed to by the index information. Specifically, the motion information of the second triangle prediction unit can be derived from any one of the remaining triangle merge candidates.

[0342] Thus, the number of maximum triangle merge candidates included in the triangle merge candidate list of the first triangle prediction unit and the number of maximum triangle merge candidates included in the triangle merge candidate list of the second triangle prediction unit can be different. For example, when the triangle merge candidate list of the first triangle prediction unit includes M merge candidates, the triangle merge candidate list of the second triangle prediction unit can include M-1 merge candidates except for the triangle merge candidate pointed by the index information of the first triangle prediction unit.

[0343] As another example, the merge candidates of each triangle prediction unit can be derived based on neighboring blocks neighboring the coding block, and the availability of the neighboring blocks can be determined by considering the shape or position of the triangle prediction unit.

[0344] FIG. 27 is a diagram for describing an example of determining the availability of neighboring blocks for each triangle prediction unit.

[0345] The neighboring blocks that are not neighboring the first triangle prediction unit can be set as the neighboring blocks unavailable for the first triangle prediction unit, and the neighboring blocks that are not neighboring the second triangle prediction unit can be set as the neighboring blocks unavailable for the second triangle prediction unit.

[0346] For example, as in the example shown in FIG. 27 (a), when the left triangle partition type is applied to the coding block, it can be determined that the blocks A1, A0, and A2 neighboring the first triangle prediction unit among the neighboring blocks neighboring the coding block are available for the first triangle prediction unit, and the blocks B0 and B1 are unavailable for the first triangle prediction unit. Thus, the triangle merge candidate list related to the first triangle prediction unit includes the triangle merge candidates derived from the blocks A1, A0, and A2, and does not include the triangle merge candidates derived from the blocks B0 and B1.

[0347] As in the example shown in FIG. 27 (b), when the left triangle partition type is applied to the coding block, it can be determined that the blocks B0 and B1 neighboring the second triangle prediction unit are available for the second triangle prediction unit, and the blocks A1, A0, and A2 are unavailable for the second triangle prediction unit. Thus, the triangle merge candidate list related to the second triangle prediction unit includes the triangle merge candidates derived from the blocks B0 and B1, and does not include the triangle merge candidates derived from the blocks A1, A0, and A2.

[0348] Thus, the number of triangle merge candidates available for the triangle prediction unit or the range of triangle merge candidates available for the triangle prediction unit can be determined based on at least one of the position of the triangle prediction unit or the partition type of the coding block.

[0349] As another example, the merge mode can be applied only to either one of the first triangular prediction unit and the second triangular prediction unit. Also, the other one of the first triangular prediction unit and the second triangular prediction unit can be set to have the same motion information as the triangular prediction unit to which the merge mode is applied, or the motion information of the triangular prediction unit to which the merge mode is applied can be refined to derive.

[0350] For example, the motion vector and the reference picture index of the first triangular prediction unit can be derived based on a triangular merge candidate, and the motion vector of the second triangular prediction unit can be derived by refining the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit can be derived by adding or subtracting a refinement motion vector {Rx, Ry} to or from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit can be set to be the same as the reference picture index of the first triangular prediction unit.

[0351] Information for determining a refinement motion vector representing a difference between the motion vector of the first triangular prediction unit and the motion vector of the second triangular prediction unit can be signaled through a bitstream. The information can include at least one of information representing a size of the refinement motion vector or information representing a sign of the refinement motion vector.

[0352] Alternatively, the sign of the refinement motion vector can be derived based on at least one of a position, an index of the triangular prediction unit, or a split type applied to a coding block.

[0353] As another example, the motion vector and the reference picture index of either one of the first triangular prediction unit and the second triangular prediction unit can be signaled. The motion vector of the other one of the first triangular prediction unit and the second triangular prediction unit can be derived by refining the signaled motion vector.

[0354] For example, based on information signaled from a bitstream, the motion vector and the reference picture index of the first triangular prediction unit can be determined. Also, the motion vector of the second triangular prediction unit can be derived by refining the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit can be derived by adding or subtracting a refinement motion vector {Rx, Ry} to or from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit can be set to be the same as the reference picture index of the first triangular prediction unit.

[0355] Motion prediction compensation prediction can be performed on the coding block based on the motion information of the first triangular prediction unit and the motion information of the second triangular prediction unit. In this case, quality degradation can occur in the boundary region of the first triangular prediction unit and the second triangular prediction unit. For example, the edge (Edge) existing on the boundary of the first triangular prediction unit and the second triangular prediction unit can cause deterioration in the continuity of the quality. In order to reduce the quality degradation of the boundary region, the prediction samples can be derived by Smoothing filtering or weighted prediction.

[0356] The prediction samples to which the diagonal split is applied within the coding block can be derived according to a weighted sum operation of the first prediction samples obtained based on the motion information of the first triangular prediction unit and the second prediction samples obtained based on the motion information of the second triangular prediction unit. Alternatively, the prediction samples of the first triangular prediction unit are derived from a first prediction block determined based on the motion information of the first triangular prediction unit, the prediction samples of the second triangular prediction unit are derived from a second prediction block determined based on the motion information of the second triangular prediction unit, and the prediction samples located in the boundary region of the first triangular prediction unit and the second triangular prediction unit can be derived based on a weighted sum operation of the first prediction samples included in the first prediction block and the second prediction samples included in the second prediction block. For example, Equation 6 below shows an example of deriving the prediction samples of the first triangular prediction unit and the second triangular prediction unit.

[0357] [Equation 6]

[0358] P(x, y) = w1*P1(x, y) + (1-w1)*P2(x, y)

[0359] In Equation 6, P1 denotes the first prediction sample, and P2 denotes the second prediction sample. w1 denotes a weighting value applied to the first prediction sample, and (1-w1) denotes a weighting value applied to the second prediction sample. As in the example shown in Equation 6, the weighting value applied to the second prediction sample can be derived by subtracting a constant from the weighting value applied to the first prediction sample.

[0360] In the case where the left triangular split type is applied to the coding block, the boundary region can include prediction samples having the same x-axis coordinate and y-axis coordinate. On the other hand, in the case where the right triangular split type is applied to the coding block, the boundary region can include prediction samples whose sum of the x-axis coordinate and the y-axis coordinate is above a first threshold value and below a second threshold value.

[0361] The size of the boundary region can be determined based on at least one of the size of the coding block, the shape of the coding block, the motion information of the triangular prediction unit, the motion vector difference of the triangular prediction unit, the output order of the reference picture, or the difference between the first prediction sample and the second prediction sample in the diagonal boundary.

[0362] FIG. 28 and FIG. 29 is a diagram showing an example of deriving a prediction sample based on a weighted sum operation of a first prediction sample and a second prediction sample. FIG. 28 An example of applying a left triangle partition type to a coding block is shown, and FIG. 29 An example of applying a right triangle partition type to a coding block is shown. In addition, FIG. 28 (a) and FIG. 29 (a) is a diagram showing a prediction pattern of a luma component, and FIG. 28 (b) and FIG. 29 (b) is a diagram showing a prediction pattern of a chroma component.

[0363] In the illustrated diagrams, a number written in a prediction sample indicates a weighting value applied to a first prediction sample near a boundary of a first prediction unit and a second prediction unit. For example, when the number written in the prediction sample is N, N / 8 of a weighting value is applied to the first prediction sample, and (1-(N / 8)) of a weighting value is applied to the second prediction sample, whereby the prediction sample can be derived.

[0364] In a non-boundary region, a first prediction sample or a second prediction sample can be determined as a prediction sample. Referring to FIG. 28 , in a region belonging to a first triangle prediction unit in a region in which an absolute value of a difference between an x-axis coordinate and a y-axis coordinate is greater than a threshold value, a first prediction sample derived based on motion information of the first triangle prediction unit can be determined as a prediction sample. On the other hand, in a region belonging to a second triangle prediction unit in a region in which the difference between the x-axis coordinate and the y-axis coordinate is greater than the threshold value, a second prediction sample derived based on motion information of the second triangle prediction unit can be determined as a prediction sample.

[0365] Referring to FIG. 28 , in a region in which a sum of an x-axis coordinate and a y-axis coordinate is less than a first threshold value, a first prediction sample derived based on motion information of a first triangle prediction unit can be determined as a prediction sample. On the other hand, in a region in which the sum of the x-axis coordinate and the y-axis coordinate is greater than a second threshold value, a second prediction sample derived based on motion information of a second triangle prediction unit can be determined as a prediction sample.

[0366] A threshold value for determining a non-boundary region can be determined based on at least one of a size of a coding block, a shape of a coding block, or a color component. For example, when a threshold value related to a luma component is set to N, a threshold value related to a chroma component can be set to N / 2.

[0367] The prediction sample included in the boundary region can be derived based on a weighted sum operation of the first prediction sample and the second prediction sample. In this case, the weight values applied to the first prediction sample and the second prediction sample can be determined based on at least one of a position of the prediction sample, a size of the coding block, a shape of the coding block, or a color component.

[0368] For example, as in the example shown in FIG. 29 (a), the prediction sample at a position having the same x-axis coordinate and y-axis coordinate can be derived by applying the same weight value to the first prediction sample and the second prediction sample. The weight value ratio applied to the first prediction sample and the second prediction sample can be set to (3:1) or (1:3) to derive the prediction sample having an absolute value of 1 for a difference between the x-axis coordinate and the y-axis coordinate. Also, the weight value ratio applied to the first prediction sample and the second prediction sample can be set to (7:1) or (1:7) to derive the prediction sample having an absolute value of 2 for a difference between the x-axis coordinate and the y-axis coordinate.

[0369] Alternatively, as in the example shown in FIG. 29 (b), the prediction sample at a position having the same x-axis coordinate and y-axis coordinate can be derived by applying the same weight value to the first prediction sample and the second prediction sample, and the prediction sample having an absolute value of 1 for a difference between the x-axis coordinate and the y-axis coordinate can be derived by setting the weight value ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0370] For example, as in the example shown in ​ (a), the prediction sample having a sum of the x-axis coordinate and the y-axis coordinate that is smaller than a width or a height of the coding block by 1 can be derived by applying the same weight value to the first prediction sample and the second prediction sample. The weight value ratio applied to the first prediction sample and the second prediction sample can be set to (3:1) or (1:3) to derive the prediction sample having a sum of the x-axis coordinate and the y-axis coordinate that is the same as or smaller than the width or the height of the coding block by 2. The weight value ratio applied to the first prediction sample and the second prediction sample can be set to (7:1) or (1:7) to derive the prediction sample having a sum of the x-axis coordinate and the y-axis coordinate that is greater than the width or the height of the coding block by 1 or smaller than the width or the height of the coding block by 3.

[0371] Alternatively, as in the example shown in ​ (b), the prediction sample having a sum of the x-axis coordinate and the y-axis coordinate that is smaller than a width or a height of the coding block by 1 can be derived by applying the same weight value to the first prediction sample and the second prediction sample. The weight value ratio applied to the first prediction sample and the second prediction sample can be set to (7:1) or (1:7) to derive the prediction sample having a sum of the x-axis coordinate and the y-axis coordinate that is the same as or smaller than the width or the height of the coding block by 2.

[0372] As another example, the position of the prediction sample or the shape of the coding block can be considered to determine the weighting value. Equations 7 to 9 show an example of deriving the weighting value when the left triangle partition type is applied to the coding block. Equation 7 shows an example of deriving the weighting value applied to the first prediction sample when the coding block is a square shape.

[0373] [Equation 7]

[0374] w1 = (x - y + 4) / 8

[0375] In Equation 7, x and y denote the position of the prediction sample. When the coding block is a non-square shape, the weighting value applied to the first prediction sample can be derived as shown in Equations 8 or 9. Equation 8 shows a case where the width of the coding block is greater than the height, and Equation 9 shows a case where the width of the coding block is less than the height.

[0376] [Equation 8]

[0377] w1 = ((x / whRatio) - y + 4) / 8

[0378] [Equation 9]

[0379] w1 = (x - (y * whRatio) + 4) / 8

[0380] When the right triangle partition type is applied to the coding block, the weighting value applied to the first prediction sample can be determined as shown in Equations 10 to 12. Equation 10 shows an example of deriving the weighting value applied to the first prediction sample when the coding block is a square shape.

[0381] [Equation 10]

[0382] w1 = (CbW - 1 - x - y) + 4) / 8

[0383] In Equation 10, CbW denotes the width of the coding block. When the coding block is a non-square shape, the weighting value applied to the first prediction sample can be derived as shown in Equations 11 or 12. Equation 11 shows a case where the width of the coding block is greater than the height, and Equation 12 shows a case where the width of the coding block is less than the height.

[0384] [Equation 11]

[0385] w1 = (CbH - 1 - (x / whRatio) - y) + 4) / 8

[0386] [Equation 12]

[0387] w1 = (CbW - 1 - x - (y * whRation) + 4) / 8

[0388] In Equation 11, CbH denotes a height of the coding block.

[0389] As an example as shown, for the prediction samples within the boundary region, the samples included in the first triangular prediction unit can be derived by assigning a greater weighting value to the first prediction sample than to the second prediction sample, and the samples included in the second triangular prediction unit can be derived by assigning a greater weighting value to the second prediction sample than to the first prediction sample.

[0390] When the diagonal partition is applied to the coding block, the coding block can be set to not apply the combined prediction mode of the intra prediction mode and the merge mode combination.

[0391] Intra prediction is to predict the current block using the reconstructed samples of the neighboring blocks of the current block that have been coded / decoded. In this case, the intra prediction of the current block can use the reconstructed samples before the in-loop filter is applied.

[0392] The intra prediction technique includes a matrix-based intra prediction and a general intra prediction that considers the directionality with the neighboring reconstructed samples. Information indicating the intra prediction technique of the current block can be signaled through a bitstream. The information can be a 1-bit flag. Alternatively, the intra prediction technique of the current block can be determined based on at least one of the location, size, shape of the current block, or the intra prediction technique of the neighboring block. For example, when the current block exists across the image boundary, the current block is set to not apply the matrix-based intra prediction.

[0393] The matrix-based intra prediction is a method of obtaining a prediction block of the current block based on a matrix multiplication between a matrix stored in the encoder and the decoder and the reconstructed samples of the neighboring blocks of the current block. Information for specifying any one of a plurality of stored matrices can be signaled through a bitstream. The decoder can determine the matrix for the intra prediction of the current block based on the information and the size of the current block.

[0394] The general intra prediction is a method of obtaining a prediction block related to the current block based on a non-angular intra prediction mode or an angular intra prediction mode.

[0395] The derived residual image can be derived by subtracting the predicted image from the original image. In this case, when the residual image is changed to the frequency domain, the subjective quality of the video does not significantly decrease even if the high frequency components among the frequency components are removed. Thus, if the value of the high frequency component is made smaller or set to 0, there is an effect of improving the compression efficiency without causing a significant visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into 2-dimensional frequency components. The transformation can be performed using a transform technique such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST).

[0396] In the residual image, a 2-dimensional image transform can not be performed on a part of the blocks. The non-performance of the 2-dimensional image transform can be referred to as Transform Skip. When the Transform Skip is applied, quantization can be applied to the residual values on which the transform is not performed.

[0397] After the current block is transformed using the DCT or the DST, the transformed current block can be transformed again. In this case, the DCT or DST-based transform can be defined as a primary transform, and the process of transforming the block to which the primary transform is applied again can be referred to as a secondary transform.

[0398] The primary transform can be performed using any one of a plurality of transform kernel candidates. For example, the primary transform can be performed using any one of DCT2, DCT8, or DCT7.

[0399] Different transform kernels can be used for the horizontal direction and the vertical direction. Information indicating a combination of the transform kernel for the horizontal direction and the transform kernel for the vertical direction can also be signaled through a bitstream.

[0400] The execution units of the primary transform and the secondary transform can be different. For example, the primary transform can be performed on an 8x8 block, and the secondary transform can be performed on a 4x4 sub-block among the transformed 8x8 block. In this case, the transform coefficients of the remaining area on which the secondary transform is not performed can also be set to 0.

[0401] Alternatively, the primary transform can be performed on a 4x4 block, and the secondary transform can be performed on an 8x8 area including the transformed 4x4 block.

[0402] Information indicating whether the secondary transform is performed can be signaled through a bitstream.

[0403] Alternatively, it can be determined whether to perform the secondary transform based on whether the horizontal direction transform core and the vertical direction transform core are the same. For example, the secondary transform can be performed only when the horizontal direction transform core and the vertical direction transform core are the same. Alternatively, the secondary transform can be performed only when the horizontal direction transform core and the vertical direction transform core are not the same.

[0404] Alternatively, the secondary transform can be allowed only when the horizontal direction transform and the vertical direction transform utilize a predefined transform core. For example, the secondary transform can be allowed when the horizontal direction transform and the vertical direction transform use a DCT2 transform core.

[0405] Alternatively, it can be determined whether to perform the secondary transform based on the number of non-zero transform coefficients of the current block. For example, it can be set not to use the secondary transform when the number of non-zero transform coefficients of the current block is less than or equal to a threshold, and it can be set to use the secondary transform when the number of non-zero transform coefficients of the current block is greater than the threshold. It can also be set to use the secondary transform only when the current block is encoded with intra prediction.

[0406] An inverse transform of the secondary transform (a second inverse transform) can be performed in the decoder, and an inverse transform of the primary transform (a first inverse transform) can be performed on a result thereof. As a result of the performance of the second inverse transform and the first inverse transform, a residual signal of the current block can be obtained.

[0407] If the transform and the quantization are performed in the encoder, the decoder can obtain a residual block through inverse quantization and inverse transform. The prediction block and the residual block can be added in the decoder, and a reconstructed block of the current block can be obtained.

[0408] If the reconstructed block of the current block is obtained, information loss generated in the quantization and the encoding process can be reduced through in-loop filtering. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

[0409] Embodiments described with the decoding process or the encoding process as a center are also included in the scope of the present application. Embodiments described in a predetermined order can be changed in an order different from that described, and are also included in the scope of the present application.

[0410] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the time sequence of the invention, and can be performed simultaneously or in another order as needed. Also, in the above-described embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagrams can also be respectively implemented as hardware devices or software, and a plurality of structural elements can be implemented in combination as a single hardware device or software. The embodiments can be implemented in the form of program instructions executable by various computer components and recorded in computer-readable recording media. The computer-readable recording media can individually or in combination include program instructions, data files, data structures, etc. Examples of the computer-readable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, and the like, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROMs, RAMs, flash memories, and the like. The hardware devices can be configured to operate as one or more software modules to perform processes according to the present invention, and vice versa.

[0411] Industrial applicability

[0412] The present invention can be applied to an electronic device that encodes / decodes a video.

Claims

1. A method of video decoding, comprising: deriving a merge candidate list for a current block; deriving first motion information of a prediction unit containing a first triangle based on a first merge candidate included in the merge candidate list, wherein the first merge candidate is determined based on first index information; deriving second motion information of a prediction unit containing a second triangle based on a second merge candidate included in the merge candidate list, wherein the second merge candidate is determined based on second index information; and determining prediction samples of the current block based on a weighted sum operation of first prediction samples and second prediction samples, wherein the first prediction samples are derived based on the first motion information and the second prediction samples are derived based on the second motion information; wherein the first index information and the second index information are obtained by decoding a bitstream, and when a value of the second index information is equal to or larger than a value of the first index information, the second merge candidate is determined as a merge candidate having an index obtained by adding 1 to the value of the second index information.

2. The method of claim 1, wherein when the value of the second index information is smaller than the value of the first index information, the second merge candidate is determined as a merge candidate having an index indicated by the second index information.

3. The method of claim 1, wherein a maximum number of merge candidates included in the merge candidate list is determined based on information obtained by decoding the bitstream.

4. The method of claim 1, further comprising: determining a position of a diagonal line indicating a partitioning of the current block based on information indicating the position of the diagonal line, wherein the information is obtained by decoding the bitstream.

5. The method of claim 1, wherein whether a partitioning-based mode is applied to the current block is determined based on at least one of a slice type, a size of the current block, a shape of the current block, and a prediction mode of the current block.

6. The method of claim 5, wherein whether the partitioning-based mode is applied to the current block is determined based on whether a ratio of a width to a height of the current block is lower than a first threshold.

7. The method of claim 5, wherein whether the partitioning-based mode is applied to the current block is determined according to whether the prediction mode of the current block is inter prediction.

8. A method of video encoding, comprising: deriving a merge candidate list for a current block; deriving first motion information of a prediction unit containing a first triangle based on a first merge candidate included in the merge candidate list, wherein the first merge candidate corresponds to first index information; deriving second motion information of a prediction unit containing a second triangle based on a second merge candidate included in the merge candidate list, wherein the second merge candidate corresponds to second index information; and determining prediction samples of the current block based on a weighted sum operation of first prediction samples and second prediction samples, wherein the first prediction samples are derived based on the first motion information and the second prediction samples are derived based on the second motion information. encode the first index information and the second index information into a bitstream, wherein when a value of the second index information is equal to or greater than a value of the first index information, the second index information is encoded using a value obtained by reducing 1 from an index of the second merge candidate.

9. The method of claim 8, wherein when the value of the second index information is less than the value of the first index information, the second index information is encoded using a value equal to the index of the second merge candidate.

10. The method of claim 8, wherein information for determining a maximum number of merge candidates included in the merge candidate list is written into a bitstream.

11. The method of claim 8, further comprising: information for indicating a position of a diagonal line partitioning the current block is written into a bitstream.

12. The method of claim 8, wherein whether to apply the partition-based mode to the current block is determined based on at least one of a slice type, a size of the current block, a shape of the current block, and a prediction mode of the current block.

13. The method of claim 12, wherein whether to apply the partition-based mode to the current block is determined based on whether a ratio of a width to a height of the current block is lower than a first threshold.

14. The method of claim 12, wherein whether to apply the partition-based mode to the current block is determined according to whether a prediction mode of the current block is inter prediction.

15. A video decoding apparatus comprising an inter prediction section configured to perform the method according to any one of claims 1 to 7.

16. A video encoding apparatus comprising an inter prediction section configured to perform the method according to any one of claims 8 to 14.

17. A computer readable storage medium storing a bitstream generated by a video encoding apparatus performing the video encoding method according to any one of claims 8 to 14.

Images