Video signal encoding / decoding method and apparatus therefor

Abstract

The video decoding method according to the present application includes the steps of generating a merge candidate list for a first block, selecting any one of merge candidates included in the merge candidate list, and performing motion compensation on the first block based on motion information of the selected merge candidate. At this time, inter merge candidates included in an inter motion information list can be added to the merge candidate list based on the number of spatial merge candidates and temporal merge candidates included in the merge candidate list.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201980059801.6, filed in the State Intellectual Property Office on September 20, 2019, for "Video signal encoding / decoding method and apparatus thereof", which is a national phase entry of PCT International Patent Application No. PCT / KR2019 / 012292, filed on September 20, 2019, claiming priority to Korean Patent Application No. 10-2018-0114348, filed on September 21, 2018, the entire contents of which are incorporated herein by reference.

[0002] Cross Reference to Related Applications

[0003] This application is based on and claims priority under 35 U.S.C. § 119 from Korean Patent Application No. 10-2018-0114348, filed on September 21, 2018, the entire contents of which are incorporated herein by reference.

[0004] This application is based on and claims priority under 35 U.S.C. § 119 from Korean Patent Application No. 10-2018-0114349, filed on September 21, 2018, the entire contents of which are incorporated herein by reference.

[0005] This application is based on and claims priority under 35 U.S.C. § 119 from Korean Patent Application No. 10-2018-0148858, filed on November 27, 2018, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0006] The present application relates to a video signal encoding / decoding method and apparatus thereof. BACKGROUND

[0007] With the trend of larger display panels, there is a growing need for higher quality video services. The biggest problem with high-definition video services is the large increase in data volume, and in order to solve this problem, research is actively being conducted to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). The JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding) and was approved on January 25, 2013, and its compression performance is about 2 times that of H.264 / AVC. With the rapid development of high-definition video services, the performance of HEVC has gradually revealed its limitations. SUMMARY

[0008] Technical Problem to be Solved

[0009] The object of the present application is to provide a method of deriving merge candidates other than merge candidates derived from candidate blocks adjacent to a current block when encoding / decoding a video signal and an apparatus for performing the same.

[0010] The object of the present application is to provide a method of deriving merge candidates using an inter motion information list when encoding / decoding a video signal and an apparatus for performing the same.

[0011] The object of the present application is to provide a method of deriving merge candidates of blocks included in a merge processing region when encoding / decoding a video signal and an apparatus for performing the same.

[0012] The technical problems to be solved by the present application are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present application pertains from the following description.

[0013] Technical Solution

[0014] The video signal decoding / encoding method according to the present application includes the steps of generating a merge candidate list for a first block, selecting any one of merge candidates included in the merge candidate list, and performing motion compensation on the first block based on motion information of the selected merge candidate. At this time, inter-merge candidates included in an inter-motion information list can be added to the merge candidate list based on the number of spatial merge candidates and temporal merge candidates included in the merge candidate list.

[0015] In the video signal decoding / encoding method according to the present application, the inter-motion information list can include inter-merge candidates derived based on motion information of blocks decoded before the first block. At this time, the inter-motion information list can not be updated based on motion information of a second block included in the same merge processing region as the first block.

[0016] In the video signal decoding / encoding method according to the present application, when the first block is included in a merge processing region, a temporary merge candidate derived based on motion information of the first block is added to a temporary motion information list, and when decoding of all blocks included in the merge processing region is completed, the temporary merge candidate is updated in the inter-motion information list.

[0017] In the video signal decoding / encoding method according to the present application, based on a result of determining whether a first inter-merge candidate included in the inter-motion information list is identical to at least one merge candidate included in the merge candidate list, it can be determined whether to add the first inter-merge candidate to the merge candidate list.

[0018] In the video signal decoding / encoding method according to the present application, the determination can be performed by comparing at least one merge candidate having an index value less than or equal to a threshold value with the first inter-merge candidate.

[0019] In the video signal decoding / encoding method according to the present application, when it is determined that there is a merge candidate identical to the first inter-merge candidate, the first inter-merge candidate is not added to the merge candidate list, and based on a result of determining whether a first inter-merge candidate included in the inter-motion information list is identical to at least one merge candidate included in the merge candidate list, it can be determined whether to add a second inter-merge candidate to the merge candidate list. At this time, a determination as to whether the second inter-merge candidate is identical to a merge candidate identical to the first inter-merge candidate can be skipped.

[0020] The features briefly summarized above for the application are merely exemplary implementations of the detailed description of the application that will be described later and do not limit the scope of the application.

[0021] Effects of Invention

[0022] According to the present application, inter prediction efficiency can be improved by providing a method of deriving merge candidates other than merge candidates derived from neighboring candidate blocks of a current block.

[0023] According to the present application, inter prediction efficiency can be improved by providing a method of deriving merge candidates using an inter motion information list.

[0024] According to the present application, inter prediction efficiency can be improved by providing a method of deriving merge candidates of blocks included in a merge processing region.

[0025] Effects obtainable in the present application are not limited to the above-mentioned effects and other effects not mentioned above will become apparent to those skilled in the art from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a block diagram of a video encoder (encoder) according to an embodiment of the present application.

[0027] Figure 2 is a block diagram of a video decoder (decoder) according to an embodiment of the present application.

[0028] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present application.

[0029] Figure 4 is a diagram illustrating various partition types of a coding block.

[0030] Figure 5 is a diagram illustrating a partition mode of a coding tree unit.

[0031] Figure 6 is a flowchart of an inter prediction method according to an embodiment of the present application.

[0032] Figure 7 is a diagram illustrating non-linear motion of an object.

[0033] Figure 8 is a flowchart illustrating an affine motion based inter prediction method according to an embodiment of the present application.

[0034] Figure 9 is a diagram illustrating an example of affine seed vectors for each affine motion model.

[0035] Figure 10 is a diagram illustrating an example of affine vectors for sub-blocks under a 4-parameter motion model.

[0036] Figure 11is a flowchart of a process of deriving motion information of a current block in merge mode.

[0037] Figure 12 is a diagram illustrating an example of a candidate block used to derive a merge candidate.

[0038] Figure 13 is a diagram illustrating a position of reference samples.

[0039] Figure 14 is a diagram illustrating an example of a candidate block used to derive a merge candidate.

[0040] Figure 15 is a diagram illustrating an example of changing a position of reference samples.

[0041] Figure 16 is a diagram illustrating an example of changing a position of reference samples.

[0042] Figure 17 is a flowchart illustrating an update state of an inter motion information list.

[0043] Figure 18 is a diagram illustrating an embodiment of updating an inter merge candidate list.

[0044] Figure 19 is a diagram illustrating an example of updating an index of a stored inter merge candidate.

[0045] Figure 20 is a diagram illustrating a position of a representative sub-block.

[0046] Figure 21 is a diagram illustrating an example of generating an inter motion information list for each inter prediction mode.

[0047] Figure 22 is a diagram illustrating an example of adding an inter merge candidate included in a long-term motion information list to a merge candidate list.

[0048] Figure 23 is a diagram illustrating an example of performing redundancy detection only on a part of merge candidates.

[0049] Figure 24 is a diagram illustrating an example of skipping redundancy detection for a specific merge candidate.

[0050] Figure 25 is a diagram illustrating an example of setting a candidate block included in the same merge processing region as a current block to be unavailable as a merge candidate.

[0051] Figure 26 is a diagram illustrating a temporary motion information list.

[0052] Figure 27 This is a diagram illustrating an example of merging the inter-frame motion information list and the temporary motion information list.

[0053] Figure 28 This is a flowchart of an intra-frame prediction method according to an embodiment of the present invention.

[0054] Figure 29 This is a diagram showing the reference samples included in each reference sample line.

[0055] Figure 30 This is a diagram illustrating the intra-frame prediction mode.

[0056] Figure 31 and Figure 32 This is a diagram illustrating an example of a one-dimensional array of reference samples arranged in a row.

[0057] Figure 33 This is a diagram showing the angle formed by the intra-frame prediction mode and a straight line parallel to the x-axis.

[0058] Figure 34 This is a diagram showing the predicted samples obtained when the current block is not a square.

[0059] Figure 35 This is a diagram illustrating the wide-angle intra-frame prediction mode.

[0060] Figure 36 This is a diagram illustrating the application of PDPC.

[0061] Figure 37 This shows an example of specifying a second merge candidate by taking into account the search order of the candidate blocks.

[0062] Figure 38 This illustrates an example of selecting a first and second merge candidate from merge candidates derived from non-adjacent blocks.

[0063] Figure 39 This is a diagram illustrating an example of determining the weighting applied to the prediction block based on the shape of the candidate block.

[0064] Figure 40 This is a diagram illustrating an example of setting a non-affine merge candidate, rather than an affine merge candidate, as the second merge candidate.

[0065] Figure 41 This is a diagram showing an example of replacing merge candidates.

[0066] Figure 42 This is a flowchart illustrating the process of determining block strength.

[0067] Figure 43 This is a graph showing predefined filter candidates. DETAILED DESCRIPTION

[0068] Hereinafter, an embodiment of the present application will be described in detail with reference to the accompanying drawings.

[0069] Encoding and decoding of a video are performed in units of blocks. For example, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, or reconstruction can be performed on a coding block, a transform block, or a prediction block.

[0070] Hereinafter, a block to be encoded / decoded will be referred to as a "current block". For example, the current block can represent a coding block, a transform block, or a prediction block according to a current encoding / decoding process step.

[0071] In addition, the term "unit" used in the present specification indicates a basic unit for performing a specific encoding / decoding process, and a "block" can be understood to indicate an array of samples of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described later, a coding block and a coding unit can be understood to have the same meaning.

[0072] Figure 1 is a block diagram of a video encoder (encoder) according to an embodiment of the present application.

[0073] Referring to Figure 1 , the video encoding apparatus 100 can include a picture partitioning section 110, prediction sections 120, 125, a transform section 130, a quantization section 135, a rearrangement section 160, an entropy encoding section 165, an inverse quantization section 140, an inverse transform section 145, a filter section 150, and a memory 155.

[0074] Figure 1 The components shown in the drawing are separately shown in order to indicate mutually different characteristic functions in the video encoding apparatus, and do not indicate that the components are constituted by separate hardware or a single software component. That is, for each component, in order to facilitate explanation, at least two of the components are combined into one component or one component is divided into a plurality of components in a manner of arranging the components, and functions are performed thereby, and such an embodiment of integrating the components and an embodiment of separating the components do not depart from the essence of the present application and belong to the scope of the present application.

[0075] Further, some structural elements are not essential structural elements for performing the essence of the present application, but are optional structural elements for improving performance. The present application can be implemented by including only components necessary for implementing the essence of the present application other than the structural elements for improving performance, and a structure including only essential structural elements other than the optional structural elements for improving performance also belongs to the scope of the present application.

[0076] The image division section 110 can divide an input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image division section 110 divides one image into a combination of a plurality of coding units, prediction units, and transform units, and can select one combination of coding units, prediction units, and transform units to encode the image based on a predetermined criterion (e.g., a cost function).

[0077] For example, one image can be divided into a plurality of coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used, and a video or a largest coding unit can be divided into other coding units as a root, and the coding units can have as many child nodes as the number of divided coding units. A coding unit that is no longer divided according to certain restrictions will become a leaf node. That is, when it is assumed that a coding unit can be divided only in a square shape, one coding unit can be divided into at most 4 other coding units.

[0078] Hereinafter, in the embodiments of the present application, a coding unit can mean a unit that performs encoding, and can also mean a unit that performs decoding.

[0079] A prediction unit within one coding unit can be split into at least one isometric shape such as a square or a rectangle of the same size, or one prediction unit within one coding unit can be split into another prediction unit having a different shape and / or size.

[0080] When a prediction unit that performs intra prediction based on a coding unit is not a minimum coding unit, intra prediction can be performed without being split into a plurality of prediction units NxN.

[0081] The prediction units 120, 125 can include an inter-prediction unit 120 that performs inter-prediction and an intra-prediction unit 125 that performs intra-prediction. It can be determined whether to use inter-prediction or perform intra-prediction for a prediction unit, and specific information (e.g., an intra-prediction mode, a motion vector, a reference picture, etc.) is determined based on each prediction method. In this case, the processing unit that performs prediction can be different from the processing unit that determines the prediction method and the specifics. For example, the prediction method and the prediction mode, etc. can be determined by the prediction unit, and the prediction can be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Also, the prediction mode information, the motion vector information, etc. used for prediction can be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When a specific encoding mode is used, the original block can also be encoded directly and transmitted to the decoder without generating a prediction block by the prediction unit 120, 125.

[0082] The inter-prediction unit 120 can predict a prediction unit based on information of at least one of a previous picture or a next picture of the current picture, and in some cases, based on information of a part of an already encoded region within the current picture. The inter-prediction unit 120 can include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0083] The reference picture interpolation unit receives reference picture information from the memory 155, and can generate pixel information of an integer pixel or less from the reference picture. For a luminance pixel, in order to generate pixel information of an integer pixel or less in a 1 / 4-pixel unit, a DCT-based 8th order interpolation filter (DCT-based Interpolation Filter) having different filter coefficients can be used. For a chrominance signal, in order to generate pixel information of an integer pixel or less in a 1 / 8-pixel unit, a DCT-based 4th order interpolation filter (DCT-based Interpolation Filter) having different filter coefficients can be used.

[0084] The motion prediction section can perform motion prediction based on a reference picture interpolated by the reference picture interpolation section. Methods for calculating a motion vector can use a full search-based block matching algorithm (FBMA), a three step search method (TSS), a new three-step search algorithm (NTS), and the like. A motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. A current prediction unit can be predicted in the motion prediction section by taking different motion prediction methods. Motion prediction methods can use a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, and the like.

[0085] The intra prediction section 125 can generate a prediction unit based on reference pixel information of a current block periphery as pixel information within a current picture. In a case where a neighboring block of a current prediction unit is a block for which inter prediction has been performed and a reference pixel is a pixel for which inter prediction has been performed, the reference pixel included in the block for which inter prediction has been performed can be used as reference pixel information of a periphery of a block for which intra prediction has been performed. That is, in a case where reference pixels are not available, at least one of the available reference pixels can be used instead of the unavailable reference pixel information.

[0086] In intra prediction, a prediction mode can have an angular prediction mode using reference pixel information in a prediction direction and a non-angular mode not using direction information when performing prediction. A mode for predicting luminance information and a mode for predicting chrominance information can be different, and for predicting chrominance information, intra prediction mode information used for predicting luminance information or predicted luminance signal information can be applied.

[0087] When performing intra prediction, if a size of a prediction unit is the same as a size of a transform unit, intra prediction can be performed on the prediction unit based on pixels present on the left side of the prediction unit, pixels present on the upper left side, and pixels present on the upper side. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using reference pixels based on the transform unit. Also, intra prediction using N x N partitioning only for a minimum coding unit can be applied.

[0088] The intra prediction method can generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of adaptive intra smoothing filter for the reference pixels can be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of the prediction units existing in the periphery of the current prediction unit. In the case of predicting the prediction mode of the current prediction unit using the mode information predicted from the peripheral prediction units, if the intra prediction modes of the current prediction unit and the peripheral prediction units are the same, information indicating that the prediction modes of the current prediction unit and the peripheral prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the peripheral prediction units are different, the prediction mode information of the current block can be encoded by performing entropy encoding.

[0089] Also, a residual block including residual value information including a difference between the prediction unit generated based on the prediction unit generated in the prediction sections 120 and 125 and the original block of the prediction unit can be generated. The generated residual block can be input to the transform section 130.

[0090] The transform section 130 can transform the original block and the residual block including the residual coefficient information of the prediction unit generated by the prediction sections 120 and 125 using a transform method such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a K-L transform (KLT). Whether to apply DCT or DST or to apply KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0091] The quantization section 135 can quantize the values transformed into the frequency domain in the transform section 130. The quantization coefficients can vary according to the importance of the block or the video. The values calculated in the quantization section 135 can be provided to the inverse quantization section 140 and the rearrangement section 160.

[0092] The rearrangement section 160 can perform rearrangement of the coefficient values on the quantized residual values.

[0093] The rearranging unit 160 can change the 2-dimensional block-shaped coefficients into a 1-dimensional vector form by a coefficient scanning method. For example, the rearranging unit 160 can scan the DC coefficient and the coefficients of a high frequency domain using a zig-zag scan method and change them into a 1-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of the zig-zag scan, a vertical scan that scans the 2-dimensional block-shaped coefficients in a column direction and a horizontal scan that scans the 2-dimensional block-shaped coefficients in a row direction can be used. That is, which scan method to use among the zig-zag scan, the vertical scan, and the horizontal scan can be determined depending on the size of the transform unit and the intra prediction mode.

[0094] The entropy encoding unit 165 can perform entropy encoding based on the values calculated by the rearranging unit 160. For example, the entropy encoding can use various encoding methods such as an exponential Golomb, a context-adaptive variable length coding (CAVLC), a context-adaptive binary arithmetic coding (CABAC), etc.

[0095] The entropy encoding unit 165 can encode various information such as residual value coefficient information and block type information of the encoding unit from the rearranging unit 160 and the prediction unit 120, 125, prediction mode information, division unit information, prediction unit information, and transmission unit information, motion vector information, reference frame information, interpolation information of a block, filter information, etc.

[0096] The coefficient values of the encoding unit input from the rearranging unit 160 can be entropy encoded in the entropy encoding unit 165.

[0097] The inverse quantization unit 140 and the inverse transform unit 145 inverse quantize the plurality of values quantized in the quantization unit 135 and inverse transform the values transformed in the transform unit 130. The residual values generated in the inverse quantization unit 140 and the inverse transform unit 145 can generate a reconstructed block by merging with the prediction unit predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction unit 120, 125.

[0098] The filter unit 150 can include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0099] A deblocking filter can remove blocking distortion generated in a reconstructed image due to boundaries between blocks. In order to determine whether to perform deblocking, it is possible to determine whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. In the case of applying a deblocking filter to a block, a strong filter or a weak filter can be applied according to a required deblocking filter strength. Also, in a process of using a deblocking filter, horizontal direction filtering and vertical direction filtering can be processed in synchronization when performing vertical filtering and horizontal filtering.

[0100] The offset correction section can correct an offset between a video in which deblocking is performed and an original video in a pixel unit. In order to perform offset correction on a designated image, a method of determining a region in which offset is to be performed after dividing pixels included in a video into a predetermined number of regions and applying offset to the corresponding region, or a method of applying offset while considering edge information of each pixel can be used.

[0101] Adaptive loop filtering (ALF) can be performed based on a value obtained by comparing a filtered reconstructed image and an original video. After dividing pixels included in a video into predetermined groups, filtering can be performed differently for each group by determining one filter to be used for the corresponding group. Information related to whether to apply adaptive loop filtering and a luminance signal can be transmitted per coding unit (CU), and a shape of an adaptive loop filter to be applied and a filter coefficient can be different for each block. Also, it is possible to apply an adaptive loop filter of the same type (fixed type) regardless of characteristics of a block to which it is applied.

[0102] The memory 155 can store a reconstructed block or image calculated by the filter section 150, and can provide the stored reconstructed block or image to the prediction sections 120 and 125 when performing inter prediction.

[0103] Figure 2 is a block diagram of a video decoder (decoder) according to an embodiment of the present application.

[0104] Referring to Figure 2 , the video decoder 200 can include an entropy decoding section 210, a rearranging section 215, an inverse quantization section 220, an inverse transform section 225, a prediction section 230, a prediction section 235, a filter section 240, and a memory 245.

[0105] When a video bitstream is input from a video encoder, the input bitstream can be decoded in a step opposite to that of the video encoder.

[0106] The entropy decoding unit 210 can perform entropy decoding in steps opposite to those performed in the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc. can be applied in correspondence with the methods performed in the video encoder.

[0107] The entropy decoding unit 210 can decode information related to intra prediction and inter prediction performed by the encoder.

[0108] The rearranging unit 215 can perform rearranging based on the method of rearranging the bitstream entropy-decoded by the entropy decoding unit 210 in the encoding unit. The plurality of coefficients represented in the form of a 1-dimensional vector can be rearranged into coefficients in the shape of a 2-dimensional block. The rearranging unit 215 receives information related to the coefficient scanning performed in the encoding unit, and can perform rearranging by a method of performing inverse scanning based on the scanning order performed in the corresponding encoding unit.

[0109] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values of the rearranged block.

[0110] The inverse transform unit 225 can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, with respect to the transform, i.e., DCT, DST, and KLT, performed by the transform unit, with respect to the quantization result performed by the video encoder. The inverse transform can be performed based on the transmission unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, the transform method (e.g., DCT, DST, KLT) can be selectively performed according to the prediction method, the size of the current block, the prediction direction, and the like.

[0111] The prediction units 230, 235 can generate a prediction block based on the information related to the generation of the prediction block provided by the entropy decoding unit 210 and the previously decoded block or image information provided by the memory 245.

[0112] As described above, when intra prediction is performed in the same manner as in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixels present on the left side of the prediction unit, the pixels present on the upper left side, and the pixels present on the upper side, and if the size of the prediction unit when intra prediction is performed is different from the size of the transform unit, intra prediction can be performed using reference pixels based on the transform unit. Also, intra prediction using N×N partitioning only for the minimum coding unit can be applied.

[0113] The prediction units decision section receives various information such as prediction unit information input from the entropy decoding section 210, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, and determines whether the prediction unit is performing inter prediction or intra prediction according to the current coding unit. The inter prediction section 230 can use information required for inter prediction of the current prediction unit provided by the video encoder, and perform inter prediction on the current prediction unit based on information included in at least one of the previous picture or the next picture of the current picture to which the current prediction unit belongs. Alternatively, inter prediction can also be performed based on information of a part of the region that has been reconstructed within the current picture to which the current prediction unit belongs.

[0114] In order to perform inter prediction, it can be determined based on the coding unit which mode among Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), Intra Block Copy Mode is used for motion prediction of the prediction unit included in the corresponding coding unit.

[0115] The intra prediction section 235 can generate a prediction block based on pixel information within the current picture. When the prediction unit is a prediction unit on which intra prediction has been performed, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction section 235 can include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation section, a DC filter. The Adaptive Intra Smoothing filter is a section that performs filtering on reference pixels of the current block, and whether to apply the filter can be determined according to the prediction mode of the current prediction unit. The Adaptive Intra Smoothing filter can be applied to the reference pixels of the current block using the prediction mode of the prediction unit and the Adaptive Intra Smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode in which the Adaptive Intra Smoothing filter is not applied, the Adaptive Intra Smoothing filter can not be applied.

[0116] For the reference pixel interpolation section, if the prediction mode of the prediction unit is a prediction unit in which intra prediction is performed based on pixel values obtained by interpolating reference pixels, the reference pixels of the integer value or less can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode in which the prediction block is generated without interpolating the reference pixels, the reference pixels can not be interpolated. If the prediction mode of the current block is a DC mode, the DC filter can generate the prediction block by filtering.

[0117] The reconstructed block or image can be provided to the filter section 240. The filter section 240 can include a deblocking filter, an offset correction section, an ALF.

[0118] Information related to whether to apply a deblocking filter to a corresponding block or image and information related to whether to apply a strong filter or a weak filter when applying a deblocking filter can be received from a video encoder. The information related to a deblocking filter provided by a video encoder is received from a deblocking filter of a video decoder, and a deblocking filter can be performed on a corresponding block by the video decoder.

[0119] The offset correction section can perform offset correction on a reconstructed video based on a type of offset correction for a video at the time of encoding and offset amount information, etc.

[0120] An ALF can be applied to a coding unit based on information related to whether to apply an ALF, ALF coefficient information, etc., provided by an encoder. Such ALF information can be provided by being included in a specific parameter set.

[0121] The memory 245 stores a reconstructed image or block so that the image or block can be used as a reference image or a reference block, and a reconstructed image can be provided to an output section.

[0122] Figure 3 FIG. 1 is a diagram illustrating a basic coding tree unit of an embodiment of the present application.

[0123] A coding block having the largest size can be defined as a coding tree block. One image can be divided into a plurality of coding tree units (CTU). A coding tree unit is a coding unit having the largest size, and can also be referred to as a largest coding unit (LCU). Figure 3 An example of dividing one image into a plurality of coding tree units is illustrated.

[0124] The size of a coding tree unit can be defined at an image level or a sequence level. To this end, information indicating the size of a coding tree unit can be signaled through a picture parameter set or a sequence parameter set.

[0125] For example, the size of a coding tree unit of an entire image within a sequence can be set to 128x128. Alternatively, either one of 128x128 or 256x256 at an image level can be determined as the size of a coding tree unit. For example, the size of a coding tree unit in a first image can be set to 128x128, and the size of a coding tree unit in a second image can be set to 256x256.

[0126] A coding block can be generated by partitioning a coding tree unit. The coding block represents a basic unit for coding / decoding processing. For example, prediction or transform can be performed per different coding block, or a prediction coding mode can be determined per different coding block. The prediction coding mode represents a method of generating a prediction picture. For example, the prediction coding mode can include Intra Prediction (IP), Inter Prediction (IP), Current Picture Referencing (CPR), Intra Block Copy (IBC), or Combined Prediction. For a coding block, at least one of the prediction coding modes, i.e., IP, IP, CPR, or Combined Prediction, can be used to generate a prediction block related to the coding block.

[0127] Information representing a prediction coding mode of a current block can be signaled through a bitstream. For example, the information can be a 1-bit flag indicating whether the prediction coding mode is an Intra mode or an Inter mode. The CPR or the Combined Prediction can be used only in a case where the prediction coding mode of the current block is determined to be the Inter mode.

[0128] The CPR is used to set a current picture as a reference picture and obtain a prediction block of a current block from a coded / decoded area within the current picture. The current picture means a picture including the current block. Information representing whether the CPR is applied to the current block can be signaled through a bitstream. For example, the information can be a 1-bit flag. When the flag is true, the prediction coding mode of the current block can be determined to be the CPR, and when the flag is false, the prediction mode of the current block can be determined to be the Inter Prediction.

[0129] Alternatively, the prediction coding mode of the current block can be determined based on a reference picture index. For example, when the reference picture index points to the current picture, the prediction coding mode of the current block can be determined to be the CPR. When the reference picture index points to another picture other than the current picture, the prediction coding mode of the current block can be determined to be the Inter Prediction. That is, the CPR is a prediction method using information of a coded / decoded area within the current picture, and the Inter Prediction is a prediction method using information of a coded / decoded other picture.

[0130] A combined prediction indicates an encoding mode in which two or more of intra prediction, inter prediction, and current picture reference are combined. For example, in a case where combined prediction is applied, a first prediction block can be generated based on one of intra prediction, inter prediction, or current picture reference, and a second prediction block can be generated based on another one. If the first prediction block and the second prediction block are generated, a final prediction block can be generated through an average operation or a weighted sum operation of the first prediction block and the second prediction block. Information indicating whether combined prediction is applied can be signaled through a bitstream. The information can be a 1-bit flag.

[0131] Figure 4 is a diagram illustrating various partition types of a coding block.

[0132] A coding block can be partitioned into a plurality of coding blocks based on a quad tree partition, a binary tree partition, or a ternary tree partition. The partitioned coding block can also be partitioned into a plurality of coding blocks based on a quad tree partition, a binary tree partition, or a ternary tree partition.

[0133] A quad tree partition refers to a partition technique in which a current block is partitioned into 4 blocks. As a result of the quad tree partition, the current block can be partitioned into 4 square sub-partitions (refer to Figure 4 “SPLIT_QT” in ).

[0134] A binary tree partition refers to a partition technique in which a current block is partitioned into 2 blocks. A process in which the current block is partitioned into two blocks along a vertical direction (i.e., using a vertical line that passes through the current block) can be referred to as a vertical direction binary tree partition, and a process in which the current block is partitioned into two blocks along a horizontal direction (i.e., using a horizontal line that passes through the current block) can be referred to as a horizontal direction binary tree partition. The current block can be partitioned into 2 non-square sub-partitions after the binary tree partition. Figure 4 “SPLIT_BT_VER” in indicates a vertical direction binary tree partition result, and Figure 4 “SPLIT_BT_HOR” in indicates a horizontal direction binary tree partition result.

[0135] A ternary tree partition refers to a partition technique in which a current block is partitioned into 3 blocks. A process in which the current block is partitioned into three blocks along a vertical direction (i.e., using two vertical lines that pass through the current block) can be referred to as a vertical direction ternary tree partition, and a process in which the current block is partitioned into three blocks along a horizontal direction (i.e., using two horizontal lines that pass through the current block) can be referred to as a horizontal direction ternary tree partition. The current block can be partitioned into 3 non-square sub-partitions after the ternary tree partition. In this case, a width / height of a sub-partition located at the center of the current block can be 2 times a width / height of the other sub-partitions. Figure 4 “SPLIT_TT_VER” in indicates a vertical direction ternary tree partition result, and Figure 4“SPLIT_TT_HOR” in the table represents the horizontal direction ternary tree partitioning result.

[0136] The number of partitioning of a coding tree unit can be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at a sequence or picture level. Thus, the maximum partitioning depth of a coding tree unit can be different according to different sequences or pictures.

[0137] Alternatively, the maximum partitioning depth for each of the plurality of partitioning techniques can be determined separately. For example, the maximum partitioning depth allowing quad-tree partitioning can be different from the maximum partitioning depth allowing binary-tree partitioning and / or ternary-tree partitioning.

[0138] The encoder can signal information representing at least one of the partitioning shape or the partitioning depth of the current block through a bitstream. The decoder can determine the partitioning shape and the partitioning depth of the coding tree unit based on the information parsed from the bitstream.

[0139] Figure 5 is a diagram showing the partitioning mode of a coding tree unit.

[0140] The process of partitioning a coding block using partitioning techniques such as quad-tree partitioning, binary-tree partitioning, and / or ternary-tree partitioning, etc. can be referred to as multi-tree partitioning.

[0141] The coding block generated by applying multi-tree partitioning to a coding block can be referred to as a plurality of downstream coding blocks. When the partitioning depth of the coding block is k, the partitioning depth of the plurality of downstream coding blocks is set to k+1.

[0142] On the other hand, for the plurality of coding blocks with a partitioning depth of k+1, the coding block with a partitioning depth of k can be referred to as an upstream coding block.

[0143] The partitioning type of the current coding block can be determined based on at least one of the partitioning shape of the upstream coding block or the partitioning type of the neighboring coding block. The neighboring coding block is adjacent to the current coding block, which can include at least one of the top neighboring block, the left neighboring block, or the top-left corner neighboring block of the current coding block. The partitioning type can include at least one of whether to partition a quad-tree, whether to partition a binary-tree, a binary-tree partitioning direction, whether to partition a ternary-tree, or a ternary-tree partitioning direction.

[0144] In order to determine the partitioning shape of the coding block, information representing whether the coding block is partitioned or not can be signaled through a bitstream. The information is a 1-bit flag “split_cu_flag”, which is true when the coding block is partitioned by a multi-tree partitioning technique.

[0145] When "split_cu_flag" is true, information indicating whether or not a coding block is quad-tree divided can be signaled through a bitstream. The information is a 1-bit flag "split_qt_flag", and when the flag is true, the coding block can be divided into 4 blocks.

[0146] For example, in the example illustrated in FIG. 10, quad-tree division is applied to a coding tree unit to generate 4 coding blocks having a division depth of 1. Also, quad-tree division is applied again to a first coding block and a fourth coding block among the 4 coding blocks generated as a result of the quad-tree division. Finally, 4 coding blocks having a division depth of 2 can be generated. Figure 5

[0147] Also, coding blocks having a division depth of 3 can be generated by applying quad-tree division again to the coding blocks having a division depth of 2.

[0148] When quad-tree division is not applied to a coding block, whether to perform binary-tree division or ternary-tree division on the coding block can be determined by considering at least one of a size of the coding block, whether the coding block is located at an image boundary, a maximum division depth, or a division shape of a neighboring block. When it is determined to perform binary-tree division or ternary-tree division on the coding block, information indicating a division direction can be signaled through a bitstream. The information can be a 1-bit flag "mtt_split_cu_vertical_flag". Whether the division direction is a vertical direction or a horizontal direction can be determined based on the flag. In addition, information indicating which one of binary-tree division or ternary-tree division is applied to the coding block can be signaled through a bitstream. The information can be a 1-bit flag "mtt_split_cu_binary_flag". Whether binary-tree division or ternary-tree division is applied to the coding block can be determined based on the flag.

[0149] For example, in the example illustrated in FIG. 11, vertical direction binary-tree division is applied to a coding block having a division depth of 1, vertical direction ternary-tree division is applied to a left coding block among the coding blocks generated as a result of the division, and vertical direction binary-tree division is applied to a right coding block. Figure 5

[0150] Inter prediction refers to a prediction encoding mode in which information of a previous image is used to predict a current block. For example, a block having the same position as a current block within a previous image (hereinafter, referred to as a collocated block) can be set as a prediction block of the current block. Hereinafter, a prediction block generated based on a block having the same position as a current block will be referred to as a collocated prediction block.

[0151] ​​On the other hand, if an object existing in the previous picture has moved to another position in the current picture, motion of the object can be used to effectively predict the current block. For example, if a moving direction and a size of the object can be known by comparing the previous picture and the current picture, a prediction block (or a predicted picture) of the current block can be generated considering motion information of the object. Hereinafter, the prediction block generated using the motion information can be referred to as a motion prediction block.

[0152] A residual block can be generated by subtracting the prediction block from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using the motion prediction block instead of the collocated prediction block, so that compression performance of the residual block can be improved.

[0153] As described above, a process of generating a prediction block using motion information can be referred to as motion-compensated prediction. In most inter-prediction, a prediction block can be generated based on motion-compensated prediction.

[0154] Motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weighting value index. The motion vector indicates a moving direction and a size of an object. The reference picture index specifies a reference picture of the current block among a plurality of reference pictures included in a reference picture list. The prediction direction refers to any one of a uni-prediction L0, a uni-prediction L1, or a bi-prediction (L0 prediction and L1 prediction). At least one of motion information of an L0 direction or motion information of an L1 direction can be used according to a prediction direction of the current block. The bi-directional weighting value index specifies a weighting value for an L0 prediction block and a weighting value applied to an L1 prediction block.

[0155] Figure 6 is a flowchart of an inter-prediction method according to an embodiment of the present application.

[0156] Reference Figure 6 The inter-prediction method includes the steps of determining an inter-prediction mode of a current block (S601), obtaining motion information of the current block according to the determined inter-prediction mode (S602), and performing motion-compensated prediction on the current block based on the obtained motion information (S603).

[0157] Among the inter prediction modes, there are inter prediction modes using translation motion information and inter prediction modes using affine motion information for determining motion information of a current block. For example, the inter prediction modes using translation motion information can include a merge mode and an advanced motion vector prediction mode, and the inter prediction modes using affine motion information can include an affine merge mode and an affine motion vector prediction mode. According to the inter prediction modes, the motion information of the current block can be determined based on neighboring blocks adjacent to the current block or information parsed from a bitstream.

[0158] Hereinafter, an inter prediction method using affine motion information will be described in detail.

[0159] Figure 7 is a graph showing non-linear motion of an object.

[0160] Motion of an object within a video can be non-linear motion. For example, as shown in an example of Figure 7 , non-linear motion of an object such as zoom-in, zoom-out, rotation, or affine transformation can occur. When non-linear motion of an object occurs, the motion of the object cannot be effectively represented by a translation motion vector. Thus, in a portion where non-linear motion of an object occurs, affine motion can be used instead of translation motion, thereby improving coding efficiency.

[0161] Figure 8 is a flowchart illustrating an affine motion based inter prediction method according to an embodiment of the present application.

[0162] Whether to apply the affine motion based inter prediction technique to the current block can be determined based on information parsed from a bitstream. Specifically, whether to apply the affine motion based inter prediction technique to the current block can be determined based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0163] When the affine motion based inter prediction technique is applied to the current block, an affine motion model of the current block can be determined (S801). The affine motion model can be determined by at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent affine motion, and the 4-parameter affine motion model uses 4 parameters to represent affine motion.

[0164] Equation 1 is a case where affine motion is represented using 6 parameters. The affine motion represents translation motion of a predetermined region determined by an affine seed vector.

[0165] [Equation 1]

[0166] v x = ax - by + e

[0167] v y = cx + dy + f

[0168] When the affine motion is represented using 6 parameters, a complex motion can be represented, but the number of bits required to encode the respective parameters increases, which reduces the coding efficiency. Thus, the affine motion can also be represented using 4 parameters. Equation 2 is for the case where the affine motion is represented using 4 parameters.

[0169] [Equation 2]

[0170] v x = ax - by + e

[0171] v y = bx + ay + f

[0172] Information for determining the affine motion model of the current block can be encoded and signaled through the bitstream. For example, the information can be a 1-bit flag "affine_type_flag". The value of the flag is 0 indicates that the 4-parameter affine motion model is applied, and the value of the flag is 1 indicates that the 6-parameter affine motion model is applied. The flag can be encoded in units of slice, tile, or block (e.g., coding block or coding tree unit). When the flag is signaled at the slice level, the affine motion model determined at the slice level can be applied to all blocks belonging to the slice.

[0173] Alternatively, the affine motion model of the current block can be determined based on the affine inter prediction mode of the current block. For example, when the affine merge mode is applied, the affine motion model of the current block can be determined as the 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, the information for determining the affine motion model of the current block can be encoded and signaled through the bitstream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0174] Next, the affine seed vectors of the current block can be derived (S802). When the 4-parameter affine motion model is selected, the motion vectors at two control points of the current block can be derived. On the other hand, when the 6-parameter affine motion model is selected, the motion vectors at three control points of the current block can be derived. The motion vectors at the control points can be referred to as affine seed vectors. The control points can include at least one of the top-left corner, the top-right corner, or the bottom-left corner of the current block.

[0175] Figure 9is a diagram showing examples of affine seed vectors for each affine motion model.

[0176] In the 4-parameter affine motion model, affine seed vectors related to two of the top-left corner, the top-right corner, or the bottom-left corner can be derived. For example, as shown in the example of part (a), when the 4-parameter affine motion model is selected, affine vectors can be derived by using an affine seed vector sv0 related to the top-left corner (e.g., the top-left sample (x0, y0)) of the current block and an affine seed vector sv1 related to the top-right corner (e.g., the top-right sample (x1, y1)) of the current block. An affine seed vector related to the bottom-left corner can also be used instead of the affine seed vector related to the top-left corner, or an affine seed vector related to the bottom-left corner can also be used instead of the affine seed vector related to the top-right corner. Figure 9

[0177] In the 6-parameter affine motion model, affine seed vectors related to the top-left corner, the top-right corner, and the bottom-left corner can be derived. For example, as shown in the example of part (b), when the 6-parameter affine motion model is selected, affine vectors can be derived by using an affine seed vector sv0 related to the top-left corner (e.g., the top-left sample (x0, y0)) of the current block, an affine seed vector sv1 related to the top-right corner (e.g., the top-right sample (x1, y1)) of the current block, and an affine seed vector sv2 related to the top-left corner (e.g., the top-left sample (x2, y2)) of the current block. Figure 9

[0178] In the embodiments described later, under the 4-parameter affine motion model, the affine seed vectors of the top-left control point and the top-right control point are referred to as a first affine seed vector and a second affine seed vector, respectively. In the embodiments described later that use the first affine seed vector and the second affine seed vector, at least one of the first affine seed vector and the second affine seed vector can be replaced with an affine seed vector of the bottom-left control point (a third affine seed vector) or an affine seed vector of the bottom-right control point (a fourth affine seed vector).

[0179] Also, under the 6-parameter affine motion model, the affine seed vectors of the top-left control point, the top-right control point, and the bottom-left control point are referred to as a first affine seed vector, a second affine seed vector, and a third affine seed vector, respectively. In the embodiments described later that use the first affine seed vector, the second affine seed vector, and the third affine seed vector, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with an affine seed vector of the bottom-right control point (a fourth affine seed vector).

[0180] ​​Affine vectors can be derived for different sub-blocks using an affine seed vector (S803). Here, the affine vector represents the translational motion vector derived from the affine seed vector. The affine vector of a sub-block can be called the affine sub-block motion vector or the sub-block motion vector.

[0181] Figure 10 This is a diagram showing an example of the affine vectors of a sub-block under a 4-parameter motion model.

[0182] The affine vector of a sub-block can be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine sub-block vector.

[0183] [Formula 3]

[0184]

[0185]

[0186] In Formula 3, (x, y) represents the position of the sub-block. The position of the sub-block refers to the position of the reference sample included within it. The reference sample can be the sample located at the top left corner of the sub-block, or at least one sample located at the center of the sub-block in either the x-axis or y-axis coordinate system. (x0, y0) represents the position of the first control point, and (sv... 0x sv 0y Let (x1, y1) represent the first affine seed vector. Additionally, (x1, y1) represents the position of the second control point, and (sv... 1x sv 1y ) represents the second affine seed vector.

[0187] When the first control point and the second control point correspond to the top left corner and the top right corner of the current block, respectively, x1-x0 can be set to the same value as the width of the current block.

[0188] Then, motion compensation prediction can be performed on each sub-block using the affine vectors of each sub-block (S804). After performing motion compensation prediction, prediction blocks associated with each sub-block can be generated. The prediction blocks of the sub-blocks can be set as the prediction blocks of the current block.

[0189] Next, we will explain in detail the inter-frame prediction method that uses translational motion information.

[0190] Motion information for the current block can be derived from the motion information of other blocks. These other blocks can be those that are prioritized for inter-frame prediction encoding / decoding compared to the current block. Setting the motion information of the current block to be the same as that of other blocks is defined as a merging mode. Furthermore, setting the motion vectors of other blocks to the predicted values ​​of the motion vectors of the current block is defined as a motion vector prediction mode.

[0191] Figure 11 is a flowchart of a process of deriving motion information of a current block in a merge mode.

[0192] A merge candidate of the current block can be derived (S1101). The merge candidate of the current block can be derived from a block that is coded / decoded in inter prediction before the current block.

[0193] Figure 12 is a diagram illustrating an example of a candidate block used to derive a merge candidate.

[0194] The candidate block can include at least one of a neighboring block including samples neighboring the current block or a non-neighboring block including samples not neighboring the current block. Hereinafter, a sample used to determine the candidate block is designated as a reference sample. In addition, a reference sample neighboring the current block is referred to as a neighboring reference sample, and a reference sample not neighboring the current block is referred to as a non-neighboring reference sample.

[0195] The neighboring reference sample can include a sample in an adjacent column of a leftmost column of the current block or a sample in an adjacent row of an uppermost row of the current block. For example, if a coordinate of a top-left sample of the current block is (0, 0), at least one of a block including a reference sample at a position of (-1, H-1), a block including a reference sample at a position of (W-1, -1), a block including a reference sample at a position of (W, -1), a block including a reference sample at a position of (-1, H), or a block including a reference sample at a position of (-1, -1) can be used as the candidate block. Referring to the drawings, at least one of the neighboring blocks indexed 0 to 4 can be used as the candidate block.

[0196] The non-neighboring reference sample indicates a sample having at least one of an x-axis distance or a y-axis distance from a neighboring reference sample of the current block of a predefined value. For example, at least one of a block including a reference sample having an x-axis distance from a left-side reference sample of a predefined value, a block including a non-neighboring sample having a y-axis distance from an upper reference sample of a predefined value, or a block including a non-neighboring sample having x-axis and y-axis distances from a top-left reference sample of predefined values can be used as the candidate block. The predefined value can be an integer of 4, 8, 12, 16, etc. Referring to the drawings, at least one of the blocks indexed 5 to 26 can be used as the candidate block.

[0197] A sample not on the same vertical line, horizontal line, or diagonal line as the neighboring reference sample can be set as the non-neighboring reference sample.

[0198] Figure 13 is a diagram illustrating positions of reference samples.

[0199] As Figure 13In the example shown, the x-coordinate of the upper non-adjacent reference sample can be set to be different from the x-coordinate of the upper adjacent reference sample. For example, when the position of the upper adjacent reference sample is (W-1, -1), the position of the upper non-adjacent reference sample that is separated from the upper adjacent reference sample by N along the y-axis can be set to ((W / 2)-1, -1-N), and the position of the upper non-adjacent reference sample that is separated from the upper adjacent reference sample by 2N along the y-axis can be set to (0, -1-2N). That is, the position of the non-adjacent reference sample can be determined based on the position of the adjacent reference sample and the distance between the adjacent reference sample.

[0200] Hereinafter, a candidate block including an adjacent reference sample in a candidate block is referred to as an adjacent block, and a block including a non-adjacent reference sample is referred to as a non-adjacent block.

[0201] When the distance between the current block and the candidate block is greater than or equal to a threshold value, the candidate block can be set to be unusable as a merge candidate. The threshold value can be determined based on the size of the coding tree unit. For example, the threshold value can be set to the height of the coding tree unit (ctu_height), or a value obtained by adding or subtracting an offset value to or from the height of the coding tree unit (e.g., ctu_height±N). The offset value N is a value predefined in the encoder and the decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0202] When the difference between the y-axis coordinate of the current block and the y-axis coordinate of the sample included in the candidate block is greater than a threshold value, the candidate block can be determined to be unusable as a merge candidate.

[0203] Alternatively, the candidate block that does not belong to the same coding tree unit as the current block can be set to be unusable as a merge candidate. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, the candidate block including the reference sample can be set to be unusable as a merge candidate.

[0204] If, when the upper boundary of the current block is adjacent to the upper boundary of the coding tree unit, a plurality of candidate blocks are determined to be unusable as a merge candidate, the coding / decoding efficiency of the current block can be reduced. To resolve the above problem, the candidate blocks can be set such that the number of candidate blocks located above the current block is greater than the number of candidate blocks located to the left of the current block.

[0205] Figure 14 FIG. 1 is a diagram illustrating an example of a candidate block used to derive a merge candidate.

[0206] As Figure 14 In the example shown, the upper block belonging to the N block columns above the current block and the left block belonging to the M block columns to the left of the current block can be set as candidate blocks. At this time, by setting M to be greater than N, the number of left candidate blocks can be set to be greater than the number of upper candidate blocks.

[0207] For example, a difference between a y-axis coordinate of a reference sample within the current block and a y-axis coordinate of an upper block available as a candidate block can be set to not more than N times a height of the current block. In addition, a difference between an x-axis coordinate of a reference sample within the current block and an x-axis coordinate of a left block available as a candidate block can be set to not more than M times a width of the current block.

[0208] For example, as shown in an example of Figure 14 , it is shown that blocks belonging to two block columns above the current block and blocks belonging to five block columns left of the current block are set as candidate blocks.

[0209] As another example, when a candidate block does not belong to the same coding tree unit as the current block, a block belonging to the same coding tree unit as the current block, or a block including a reference sample adjacent to a boundary of the coding tree unit can be utilized instead of the candidate block to derive a merge candidate.

[0210] Figure 15 is a diagram showing an example of changing a position of a reference sample.

[0211] When a reference sample is included in a different coding tree unit from the current block, and the reference sample is not adjacent to a boundary of the coding tree unit, a reference sample adjacent to the boundary of the coding tree unit can be utilized instead of the reference sample to determine a candidate block reference sample.

[0212] For example, in examples shown in Figure 15 (a) and Figure 15 (b), when an upper boundary of the current block and an upper boundary of the coding tree unit are in contact with each other, a reference sample above the current block belongs to a different coding tree unit from the current block. A reference sample not adjacent to the upper boundary of the coding tree unit among the reference samples belonging to the different coding tree unit can be replaced with a sample adjacent to the upper boundary of the coding tree unit.

[0213] For example, as shown in an example of Figure 15 (a), a reference sample at position 6 is replaced with a sample at position 6' located at the upper boundary of the coding tree unit, as shown in an example of Figure 15 (b), a reference sample at position 15 is replaced with a sample at position 15' located at the upper boundary of the coding tree unit. At this time, a y-coordinate of the replacement sample can be changed to an adjacent position of the coding tree unit, and an x-coordinate of the replacement sample can be set to be the same as that of the reference sample. For example, the sample at position 6' can have the same x-coordinate as the sample at position 6, and the sample at position 15' can have the same x-coordinate as the sample at position 15.

[0214] Alternatively, a value obtained by adding or subtracting an offset value to the x-coordinate of the reference sample can be set as the x-coordinate of the replacement sample. For example, when the x-coordinates of the neighboring reference sample and the non-neighboring reference sample located above the current block are the same, a value obtained by adding or subtracting an offset value to the x-coordinate of the reference sample can be set as the x-coordinate of the replacement sample. This is to prevent the replacement sample for replacing the non-neighboring reference sample from being located at the same position as other non-neighboring reference samples or neighboring reference samples.

[0215] Figure 16 FIG. 6 is a diagram illustrating an example of changing the position of a reference sample.

[0216] When a reference sample included in a coding tree unit different from the current block and not adjacent to the upper boundary of the coding tree unit is replaced with a sample located at the boundary of the coding tree unit, a value obtained by adding or subtracting an offset value to the x-coordinate of the reference sample can be set as the x-coordinate of the replacement sample.

[0217] For example, in the example shown in FIG. 6, the reference sample at position 6 and the reference sample at position 15 can be replaced with a sample at position 6' and a sample at position 15', respectively, which have the same y-coordinate as the row adjacent to the upper boundary of the coding tree unit. At this time, the x-coordinate of the sample at the 6' position can be set to a value having a difference of W / 2 from the x-coordinate of the reference sample at the 6 position, and the x-coordinate of the sample at the 15' position can be set to a value having a difference of W-1 from the x-coordinate of the reference sample at the 15 position. Figure 16 Unlike the example shown in FIG. 6, the y-coordinate of the row located above the uppermost row of the current block or the y-coordinate of the upper boundary of the coding tree unit can also be set as the y-coordinate of the replacement sample.

[0218] Figure 15 and Figure 16 Unlike the example shown in FIG. 6, the y-coordinate of the row located above the uppermost row of the current block or the y-coordinate of the upper boundary of the coding tree unit can also be set as the y-coordinate of the replacement sample.

[0219] Although not illustrated, a sample replacing a reference sample can also be determined based on the left boundary of the coding tree unit. For example, when a reference sample is not included in the same coding tree unit as the current block and is not adjacent to the left boundary of the coding tree unit, the reference sample can be replaced with a sample adjacent to the left boundary of the coding tree unit. At this time, the replacement sample can have the same y-coordinate as the reference sample, or can have a y-coordinate obtained by adding or subtracting an offset value to the y-coordinate of the reference sample.

[0220] Thereafter, the block including the replacement sample can be set as a candidate block, and a merge candidate of the current block can be derived based on the candidate block.

[0221] A merge candidate can also be derived from a temporally neighboring block included in a different picture from the current block. For example, a merge candidate can be derived from a co-located block included in a co-located picture. ​

[0222] The motion information of the merge candidate can be set to be the same as the motion information of the candidate block. For example, at least one of the motion vector, the reference picture index, the prediction direction, or the bi-directional weighted value index of the candidate block can be set as the motion information of the merge candidate.

[0223] A merge candidate list including the merge candidate can be generated (S1102). The merge candidate can be classified into a neighboring merge candidate derived from a neighboring block adjacent to the current block and a non-neighboring merge candidate derived from a non-neighboring block.

[0224] Indices of the plurality of merge candidates in the merge candidate list can be assigned in a predetermined order. For example, the indices assigned to the neighboring merge candidates can have smaller values than the indices assigned to the non-neighboring merge candidates. Alternatively, the indices can be assigned based on Figure 12 or Figure 14 The indices can be assigned to each of the merge candidates based on the indices of the blocks as shown in Table 1.

[0225] When the plurality of merge candidates are included in the merge candidate list, at least one of the plurality of merge candidates can be selected (S1103). At this time, information indicating whether the motion information of the current block is derived from the neighboring merge candidate can be signaled through a bitstream. The information can be a 1-bit flag. For example, a syntax element isAdjancentMergeFlag indicating whether the motion information of the current block is derived from the neighboring merge candidate can be signaled through the bitstream. When the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block can be derived based on the neighboring merge candidate. On the other hand, when the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block can be derived based on the non-neighboring merge candidate.

[0226] Table 1 shows a syntax table including the syntax element isAdjancentMergeFlag.

[0227]

Table 1

[0228]

[0229] Information for specifying any one of the plurality of merge candidates can be signaled through a bitstream. For example, information indicating an index of any one of the merge candidates included in the merge candidate list can be signaled through the bitstream.

[0230] When the isAdjacentMergeflag is 1, a syntax element merge_idx for determining any one of the adjacent merge candidates can be signaled. A maximum value of the syntax element merge_idx can be set to a value which is 1 less than the number of the adjacent merge candidates.

[0231] When the isAdjacentMergeflag is 0, a syntax element NA_merge_idx for determining any one of the non-adjacent merge candidates can be signaled. The syntax element NA_merge_idx indicates a value obtained by subtracting the number of the adjacent merge candidates from an index of the non-adjacent merge candidate. The decoder can select the non-adjacent merge candidate by adding the number of the adjacent merge candidates to the index determined according to the NA_merge_idx.

[0232] When the number of the merge candidates included in the merge candidate list is less than a threshold value, the merge candidates included in the inter motion information list can be added to the merge candidate list. The threshold value can be a maximum number of the merge candidates that the merge candidate list can include or a value obtained by subtracting an offset from the maximum number of the merge candidates. The offset can be an integer such as 1 or 2. The inter motion information list can include the merge candidates derived based on blocks encoded / decoded before the current block.

[0233] The inter motion information list includes the merge candidates derived from the blocks encoded / decoded with intra inter prediction within the current picture. For example, the motion information of the merge candidates included in the inter motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on the inter prediction. The motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weighted value index.

[0234] For convenience of explanation, the merge candidates included in the inter motion information list are referred to as inter merge candidates.

[0235] The maximum number of the merge candidates that the inter motion information list can include can be predefined in the encoder and the decoder. For example, the maximum number of the merge candidates that the inter motion information list can include can be 1, 2, 3, 4, 5, 6, 7, 8, or more (e.g., 16).

[0236] Alternatively, information indicating the maximum number of the merge candidates of the inter motion information list can be signaled through a bitstream. The information is signaled at a sequence level, a picture level, or a slice level.

[0237] Alternatively, the maximum number of the merge candidates of the inter motion information list can be determined according to a size of a picture, a size of a slice, or a size of a coding tree unit.

[0238] The inter motion information list can be initialized in units of a picture, a slice, a tile, a brick, a coding tree unit, or a coding tree unit line (row or column). For example, when a slice is initialized, the inter motion information list is also initialized, and the inter motion information list can not include any merge candidate.

[0239] Alternatively, information indicating whether to initialize the inter motion information list can also be signaled through a bitstream. The information can be signaled at a slice level, a tile level, a brick level, or a block level. Before the information indicates to initialize the inter motion information list, a configured inter motion information list can be used.

[0240] Alternatively, information related to the inter merge candidate can be signaled through a picture parameter set or a slice header. Even if a slice is initialized, the inter motion information list can include an initial inter merge candidate. Thus, the inter merge candidate can be used for a block that is first encoded / decoded within a slice.

[0241] According to an encoding / decoding order, blocks are encoded / decoded, and a plurality of blocks that are encoded / decoded based on inter prediction can be sequentially set as the inter merge candidate according to the encoding / decoding order.

[0242] Figure 17 is a flowchart illustrating an update state of the inter motion information list.

[0243] When inter prediction is performed on a current block (S1701), an inter merge candidate can be derived based on the current block (S1702). Motion information of the inter merge candidate can be set to be the same as motion information of the current block.

[0244] When the inter motion information list is empty (S1703), the inter merge candidate derived based on the current block can be added to the inter motion information list (S1704).

[0245] When the inter motion information list already includes the inter merge candidate (S1703), a redundancy check can be performed on motion information of the current block (or the inter merge candidate derived based on the current block) (S1705). The redundancy check is used to determine whether motion information of the inter merge candidate already stored in the inter motion information list is the same as motion information of the current block. The redundancy check can be performed on all of the inter merge candidates already stored in the inter motion information list. Alternatively, the redundancy check can be performed on the inter merge candidates having an index equal to or greater than a threshold or an index less than the threshold among the inter merge candidates already stored in the inter motion information list.

[0246] When the inter prediction merge candidate having the same motion information as the motion information of the current block is not included, the inter merge candidate derived based on the current block can be added to the inter motion information list (S1708). Whether the inter merge candidate is the same can be determined based on whether the motion information (e.g., motion vector and / or reference picture index, etc.) of the inter merge candidate is the same.

[0247] At this time, when the maximum number of inter merge candidates has been stored in the inter motion information list (S1706), the oldest inter merge candidate is deleted (S1707), and the inter merge candidate derived based on the current block can be added to the inter motion information list (S1708).

[0248] The plurality of inter merge candidates can be identified according to respective indexes. When the inter merge candidate derived from the current block is added to the inter motion information list, the inter merge candidate is assigned the lowest index (e.g., 0), and the indexes of the stored inter merge candidates can be increased by 1, respectively. In this case, when the maximum number of inter merge candidates has been stored in the inter motion information list, the inter merge candidate having the largest index is removed.

[0249] Alternatively, when the inter merge candidate derived from the current block is added to the inter motion information list, the inter merge candidate can be assigned the largest index. For example, when the number of inter merge candidates stored in the inter motion information list is less than the maximum value, the inter merge candidate can be assigned an index having the same value as the number of stored inter merge candidates. Alternatively, when the number of inter merge candidates stored in the inter motion information list is equal to the maximum value, the inter merge candidate can be assigned an index of 1 less than the maximum value. Also, the inter merge candidate having the smallest index is removed, and the indexes of the remaining stored plurality of inter merge candidates are decreased by 1, respectively.

[0250] Figure 18 FIG. 1 is a diagram illustrating an embodiment of updating an inter merge candidate list.

[0251] It is assumed that the inter merge candidate derived from the current block is added to the inter merge candidate list, and the inter merge candidate is assigned the largest index. Also, it is assumed that the maximum number of inter merge candidates has been stored in the inter merge candidate list.

[0252] When the inter merge candidate HmvpCand[n+1] derived from the current block is added to the inter merge candidate list HmvpCandList, the inter merge candidate HmvpCand[0] having the smallest index is deleted from the stored inter merge candidates, and the indexes of the remaining plurality of inter merge candidates are decreased by 1, respectively. In addition, the index of the inter merge candidate HmvpCand[n+1] derived from the current block can be set to the maximum value (in the example of FIG. 1, 3).Figure 18 n) in the illustrated example.

[0253] When the inter merge candidate derived based on the current block is already stored (S1705), the inter merge candidate derived based on the current block can not be added to the inter motion information list (S1709).

[0254] Alternatively, as the inter merge candidate derived based on the current block is added to the inter motion information list, the stored inter merge candidate identical to the inter merge candidate can also be removed. In this case, the effect that the index of the stored inter merge candidate is updated is caused.

[0255] Figure 19 is a diagram illustrating an example of updating the index of the stored inter merge candidate.

[0256] When the index of the stored inter merge candidate identical to the inter merge candidate derived based on the current block is hIdx, the stored inter merge candidate is deleted, and the index of the inter merge candidate greater than hIdx can be reduced by 1 each. For example, when Figure 19 In the illustrated example, it is shown that HmvpCand[2] identical to mvCand is deleted from the inter motion information list HvmpCandList, and the index of HmvpCand[3] to HmvpCand[n] is reduced by 1 each.

[0257] And, the inter merge candidate mvCand derived based on the current block can be added to the end of the inter motion information list.

[0258] Alternatively, the index assigned to the stored inter merge candidate identical to the inter merge candidate derived based on the current block can be updated. For example, the index of the stored inter merge candidate can be changed to the minimum value or the maximum value.

[0259] The motion information of the block included in the predetermined region can be set to be unable to be added to the inter motion information list. For example, the inter merge candidate derived based on the motion information of the block included in the merge processing region can not be added to the inter motion information list. Since the encoding / decoding order of the block included in the merge processing region is not defined, it is not appropriate to use the motion information of any of these blocks for the inter prediction of other blocks. Thus, the inter merge candidate derived based on the block included in the merge processing region can not be added to the inter motion information list.

[0260] When motion-compensated prediction is performed by sub-block units, a plurality of sub-blocks included in the current block can derive inter merge candidates based on motion information of a representative sub-block. For example, when sub-block merge candidates are used for the current block, inter merge candidates can be derived based on motion information of a representative sub-block among the sub-blocks.

[0261] The motion vector of the sub-block can be derived in the following order. First, any one of the merge candidates included in the merge candidate list of the current block can be selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. Also, by adding the position (xSb, ySb) of the reference sample (e.g., the top-left sample or the center position sample) of each sub-block within the coding block to the initial shift vector, a shifted sub-block whose position of the reference sample is (xColSb, yColSb) can be derived. Equation 4 below shows a formula for deriving the shifted sub-block.

[0262] [Equation 4]

[0263] (xColSb, yColSb) = (xSb + shVector[0] » 4, ySb + shVector[1] » 4)

[0264] Next, the motion vector of the co-located block corresponding to the center position of the sub-block including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0265] The representative sub-block can mean a sub-block including the top-left sample or the center sample of the current block.

[0266] Figure 20 is a diagram showing the position of the representative sub-block.

[0267] Figure 20 (a) shows an example of setting a sub-block located at the top-left of the current block as the representative sub-block, Figure 20 (b) shows an example of setting a sub-block located at the center of the current block as the representative sub-block. When motion-compensated prediction is performed in sub-block units, inter merge candidates of the current block can be derived based on the motion vector of the sub-block including the top-left sample of the current block or the sub-block including the center sample of the current block.

[0268] Based on the inter prediction mode of the current block, it can also be determined whether to use the current block as an inter merge candidate. For example, a block encoded / decoded based on an affine motion model can be set as unavailable to be used as an inter merge candidate. Thus, even if the current block is encoded / decoded by inter prediction, when the inter prediction mode of the current block is an affine prediction mode, the inter prediction motion information list is not updated based on the current block.

[0269] Alternatively, an inter merge candidate can also be derived based on at least one of the sub-block vectors included in the sub-blocks of the block being coded / decoded based on the affine motion model. For example, an inter merge candidate can be derived using the sub-block located at the top-left of the current block, the sub-block located at the center or the sub-block located at the top-right. Alternatively, an average of the sub-block vectors of multiple sub-blocks can also be set as the motion vector of the inter merge candidate.

[0270] Alternatively, an inter merge candidate can also be derived based on an average of the affine seed vectors of the block being coded / decoded based on the affine motion model. For example, an average of at least one of the first affine seed vector, the second affine seed vector or the third affine seed vector of the current block can be set as the motion vector of the inter merge candidate.

[0271] Alternatively, the inter motion information lists can be configured for different inter prediction modes. For example, at least one of the inter motion information list for a block being coded / decoded by intra block copy, the inter motion information list for a block being coded / decoded based on the translational motion model or the inter motion information list for a block being coded / decoded based on the affine motion model can be defined. Depending on the inter prediction mode of the current block, any of the multiple inter motion information lists can be selected.

[0272] Figure 21 FIG. 1 is a diagram illustrating an example of generating inter motion information lists for each inter prediction mode.

[0273] When a block is coded / decoded based on a non-affine motion model, an inter merge candidate mvCand derived based on the block can be added to the inter non-affine motion information list HmvpCandList. On the other hand, when a block is coded / decoded based on an affine motion model, an inter merge candidate mvAfCand derived based on the block can be added to the inter affine motion information list HmvpAfCandList.

[0274] An affine seed vector of a block can be stored in an inter merge candidate derived from the block being coded / decoded based on the affine motion model. Thereby, the inter merge candidate can be used as a merge candidate for deriving the affine seed vector of the current block.

[0275] In addition to the described inter motion information lists, another inter motion information list can also be defined. In addition to the described inter motion information lists (hereinafter referred to as first inter motion information lists), a long-term motion information list (hereinafter referred to as second inter motion information list) can also be defined. Among others, the long-term motion information list includes long-term merge candidates.

[0276] When both the first inter motion information list and the second inter motion information list are empty, first inter merge candidates can be added to the second inter motion information list. After the number of inter merge candidates available in the second inter motion information list reaches the maximum number, inter merge candidates can be added to the first inter motion information list.

[0277] Alternatively, one inter merge candidate can be added to both the second inter motion information list and the first inter motion information list.

[0278] In this case, the second inter motion information list can not be updated any more. Alternatively, the second inter motion information list can be updated when the decoded area is above a predetermined ratio of a slice. Alternatively, the second inter motion information list can be updated every N coding tree unit row.

[0279] On the other hand, the first inter motion information list can be updated every time a block encoded / decoded with inter prediction is generated. However, the inter merge candidate added to the second inter motion information list can be set not to be used to update the first inter motion information list.

[0280] Information for selecting either the first inter motion information list or the second inter motion information list can be signaled through a bitstream. When the number of merge candidates included in the merge candidate list is less than a threshold value, the merge candidate included in the inter motion information list indicated by the information can be added to the merge candidate list.

[0281] Alternatively, the inter motion information list can be selected based on the size, shape, inter prediction mode, bi-prediction or not, motion vector refinement or not, or triangle partitioning or not of the current block.

[0282] Alternatively, even when the number of merge candidates included in the merge candidate list is less than the maximum number of merges after the inter merge candidate included in the first inter motion information list is added, the inter merge candidate included in the second inter motion information list can be added to the merge candidate list.

[0283] Figure 22 FIG. 1 is a diagram illustrating an example of adding an inter merge candidate included in a long-term motion information list to a merge candidate list.

[0284] When the number of merge candidates included in the merge candidate list is less than the maximum number, the inter merge candidates included in the first inter motion information list HmvpCandList can be added to the merge candidate list. Even when the inter merge candidates included in the first inter motion information list are added to the merge candidate list, if the number of merge candidates included in the merge candidate list is less than the maximum number, the inter merge candidates included in the long-term motion information list HmvpLTCandList can be added to the merge candidate list.

[0285] Table 2 illustrates a process of adding the inter merge candidates included in the long-term motion information list to the merge candidate list.

[0286] [Table 2]

[0287]

[0288] The inter merge candidate can be set to include additional information in addition to the motion information. For example, the size, shape, or partition information of a block can be added in the inter merge candidate. In constructing the merge candidate list of the current block, only the inter merge candidate having the same or similar size, shape, or partition information as the current block can be used in the inter merge candidate, or the inter merge candidate having the same or similar size, shape, or partition information as the current block can be preferentially added to the merge candidate list.

[0289] Alternatively, the inter motion information list can be generated for different block sizes, shapes, or partition information. The merge candidate list of the current block can be generated using the inter motion information list corresponding to the shape, size, or partition information of the current block among the plurality of inter motion information lists.

[0290] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, the inter merge candidate included in the inter motion information list can be added to the merge candidate list. The adding process is performed in ascending or descending order of index. For example, the inter merge candidate having the largest index can be added to the merge candidate list.

[0291] When the inter merge candidate included in the inter motion information list is to be added to the merge candidate list, redundancy detection between the inter merge candidate and the plurality of merge candidates already stored in the merge candidate list can be performed.

[0292] For example, Table 3 illustrates a process of adding the inter merge candidate to the merge candidate list.

[0293] [Table 3]

[0294]

[0295] Redundancy detection can also be performed only on some of the inter-merge candidates included in the inter motion information list. For example, redundancy detection can be performed only on inter-merge candidates with index above or below a threshold. Alternatively, redundancy detection can be performed only on the N largest indexed merge candidates or the N smallest indexed merge candidates.

[0296] Alternatively, redundancy detection can be performed only on some of the stored merge candidates in the merge candidate list. For example, redundancy detection can be performed only on merge candidates with index above or below a threshold or on merge candidates derived from a block at a particular position. The particular position can include at least one of a left neighboring block, an above neighboring block, a top-right neighboring block or a bottom-left neighboring block of the current block.

[0297] Figure 23 FIG. 4 is a diagram illustrating an example of performing redundancy detection only on a portion of the merge candidates.

[0298] When an inter-merge candidate HmvpCand[j] is to be added to the merge candidate list, redundancy detection can be performed between the inter-merge candidate and the two largest indexed merge candidates mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1]. NumMerge can represent the number of spatial merge candidates and temporal merge candidates available.

[0299] Unlike the example shown in the diagram, when an inter-merge candidate HmvpCand[j] is to be added to the merge candidate list, redundancy detection can also be performed between the inter-merge candidate and the two smallest indexed merge candidates. For example, it can be confirmed whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, redundancy detection can be performed only on merge candidates derived from a particular position. For example, redundancy detection can be performed only on at least one of a merge candidate derived from a neighboring block located to the left of the current block or a merge candidate derived from a neighboring block located above the current block. When there is no merge candidate derived from the particular position in the merge candidate list, the inter-merge candidate can be added to the merge candidate list without redundancy detection.

[0300] In a case where a merge candidate identical to a first inter-merge candidate is found, redundancy detection of the merge candidate identical to the first inter-merge candidate can be skipped when performing redundancy detection on a second inter-merge candidate.

[0301] Figure 24 FIG. 4 is a diagram illustrating an example of performing redundancy detection only on a portion of the merge candidates.

[0302] When an inter merge candidate HmvpCand[i] is to be added to the merge candidate list, redundancy detection between the inter merge candidate and the merge candidates already stored in the merge candidate list can be performed. In this case, in a case where the same merge candidate mergeCandList[j] as the inter merge candidate HmvpCand[i] is found, the inter merge candidate HmvpCand[i] will not be added to the merge candidate list, and redundancy detection between the inter merge candidate HmvpCand[i-1] indexed i-1 and the merge candidate can be performed. In this case, redundancy detection between the inter merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be skipped.

[0303] For example, in the example shown in FIG. 7, it is determined that HmvpCand[i] is identical to mergeCandList[2]. Thus, HmvpCand[i] is not added to the merge candidate list, and redundancy detection can be performed on HmvpCand[i-1]. In this case, redundancy detection between HvmpCand[i-1] and mergeCandList[2] can be skipped. Figure 24 For example, in the example shown in FIG. 7, it is determined that HmvpCand[i] is identical to mergeCandList[2]. Thus, HmvpCand[i] is not added to the merge candidate list, and redundancy detection can be performed on HmvpCand[i-1]. In this case, redundancy detection between HvmpCand[i-1] and mergeCandList[2] can be skipped.

[0304] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, at least one of a pair-wise merge candidate or a zero merge candidate can be included in addition to the inter merge candidate. The pair-wise merge candidate refers to a merge candidate having an average value of motion vectors of two or more merge candidates as a motion vector, and the zero merge candidate refers to a merge candidate having a motion vector of 0.

[0305] The merge candidate list of the current block can add merge candidates in the following order.

[0306] Spatial merge candidate - temporal merge candidate - inter merge candidate - (inter affine merge candidate) - pair-wise merge candidate - zero merge candidate

[0307] The spatial merge candidate refers to a merge candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal merge candidate refers to a merge candidate derived from a previous reference picture. The inter affine merge candidate refers to an inter merge candidate derived from a block encoded / decoded in an affine motion model.

[0308] The inter motion information list can also be used in an advanced motion vector prediction mode. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold value, an inter merge candidate included in the inter motion information list is set as a motion vector prediction candidate related to the current block. Specifically, a motion vector of the inter merge candidate is set as the motion vector prediction candidate.

[0309] If any of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as the motion vector predictor of the current block. After the motion vector residual value of the current block is decoded, the motion vector of the current block can be obtained by adding the motion vector predictor and the motion vector residual value.

[0310] The motion vector prediction candidate list of the current block can be constructed in the following order.

[0311] Spatial motion vector prediction candidate - temporal motion vector prediction candidate - inter-decoded region merge candidate - (inter-decoded region affine merge candidate) - zero motion vector prediction candidate

[0312] The spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from a previous reference picture. The inter-affine merge candidate list indicates inter motion vector prediction candidates derived from blocks encoded / decoded in an affine motion model. The zero motion vector prediction candidate indicates a candidate whose value of the motion vector is 0.

[0313] A merge processing region whose size is greater than that of the coded block can be designated. The coded blocks included in the merge processing region can be processed in parallel, not sequentially encoded / decoded. Here, the sequential encoding / decoding refers to a designated encoding / decoding order. Thus, the encoding / decoding process of the blocks included in the merge processing region can be independently processed. Alternatively, the blocks included in the merge processing region can share a merge candidate. Here, the merge candidate can be derived based on the merge processing region.

[0314] According to the described feature, the merge processing region can also be referred to as a parallel processing region, a shared merge region (SMR), or a merge estimation region (MER).

[0315] The merge candidate of the current block can be derived based on the coded block. However, when the current block is included in a merge processing region whose size is greater than that of the current block, a candidate block included in the same merge processing region as the current block can be set as unavailable as a merge candidate.

[0316] Figure 25 FIG. 1 is a diagram illustrating an example of setting a candidate block included in the same merge processing region as a current block as unavailable as a merge candidate.

[0317] In Figure 25In the example shown in the left side of FIG. 5, when CU5 is encoded / decoded, blocks including reference samples neighboring CU5 can be set as candidate blocks. At this time, candidate blocks X3 and X4 included in the same merge processing region as CU5 can be set as unavailable as merge candidates of CU5. On the other hand, candidate blocks X0, X1, and X2 not included in the same merge processing region as CU5 can be set as available as merge candidates.

[0318] In Figure 25 In the example shown in the right side of FIG. 5, when CU8 is encoded / decoded, blocks including reference samples neighboring CU8 can be set as candidate blocks. At this time, candidate blocks X6, X7, and X8 included in the same merge processing region as CU8 can be set as unavailable as merge candidates. On the other hand, candidate blocks X5 and X9 not included in the same merge processing region as CU8 can be set as available as merge candidates.

[0319] The merge processing region can be a square or a non-square. Information for determining the merge processing region can be signaled through a bitstream. The information can include at least one of information indicating a shape of the merge processing region and information indicating a size of the merge processing region. When the merge processing region is a non-square, at least one of information indicating the size of the merge processing region, information indicating a width and / or a height of the merge processing region, and information indicating a ratio of the width to the height of the merge processing region can be signaled through the bitstream.

[0320] The size of the merge processing region can be determined based on at least one of information signaled through a bitstream, an image resolution, a size of a slice, or a tile size.

[0321] When performing motion-compensated prediction on blocks included in the merge processing region, inter-frame merge candidates derived based on motion information of blocks on which information list-based motion-compensated prediction has been performed can be added to inter-frame motion.

[0322] However, in a case where the inter prediction candidate derived from the block included in the merge processing region is added to the inter motion information list, there can be a case where the inter prediction candidate derived from the block is used when encoding / decoding another block in the merge processing region which is actually to be encoded / decoded after the block is encoded / decoded. That is, although dependency between blocks should be excluded when encoding / decoding the block included in the merge processing region, there can be a case where motion compensation prediction is performed using motion information of another block included in the merge processing region. To solve the problem, even if the encoding / decoding of the block included in the merge processing region is completed, the motion information of the encoded / decoded block can not be added to the inter motion information list.

[0323] Alternatively, when performing motion compensation prediction on the block included in the merge processing region, the inter prediction candidate derived from the block can be added to the inter motion information list in a predefined order. Here, the predefined order can be determined according to a scanning order of encoding blocks in the merge processing region or the coding tree unit. The scanning order can be at least one of a raster scan, a horizontal scan, a vertical scan, or a zigzag scan. Alternatively, the predefined order can be determined based on the number of blocks having motion information or the same motion information.

[0324] Alternatively, the inter prediction candidate including the uni-directional motion information can be added to the inter region merge row list before the inter prediction candidate including the bi-directional motion information. In contrast, the inter prediction candidate including the bi-directional motion information can also be added to the inter prediction candidate list before the inter prediction candidate including the uni-directional motion information.

[0325] Alternatively, the inter prediction candidate can be added to the inter motion information list according to an order of high or low usage frequency within the merge processing region or the coding tree unit.

[0326] When the current block is included in the merge processing region and the number of the merge candidates included in the merge candidate list of the current block is less than the maximum number, the inter prediction candidate included in the inter motion information list can be added to the merge candidate list. At this time, it can be set not to add the inter prediction candidate derived from the block included in the same merge processing region as the current block to the merge candidate list of the current block.

[0327] Alternatively, when the current block is included in the merge processing region, it can be set not to use the inter prediction candidate included in the inter motion information list. That is, even if the number of the merge candidates included in the merge candidate list of the current block is less than the maximum number, the inter prediction candidate included in the inter motion information list can not be added to the merge candidate list.

[0328] A list of inter motion information for a merge processing region or a coding tree unit can be configured. The list of inter motion information functions to temporarily store motion information of a block included in the merge processing region. In order to distinguish the general list of inter motion information from the list of inter motion information for the merge processing region or the coding tree unit, the list of inter motion information for the merge processing region or the coding tree unit is referred to as a temporary list of motion information. In addition, an inter merge candidate stored in the temporary list of motion information is referred to as a temporary merge candidate.

[0329] Figure 26 FIG. 1 is a diagram illustrating a temporary list of motion information.

[0330] A temporary list of motion information for a coding tree unit or a merge processing region can be configured. When motion-compensated prediction has been performed on a current block included in the coding tree unit or the merge processing region, motion information of the block can not be added to a list of inter prediction motion information HmvpCandList. Instead, a temporary merge candidate derived from the block can be added to the temporary list of motion information HmvpMERCandList. That is, the temporary merge candidate added to the temporary list of motion information can not be added to the list of inter motion information. Thereby, the list of inter motion information can not include an inter merge candidate derived based on motion information of a block included in the coding tree unit or the merge processing region.

[0331] A maximum number of merge candidates that the temporary list of motion information can include can be set to be the same as a maximum number of merge candidates that the list of inter motion information can include. Alternatively, the maximum number of merge candidates that the temporary list of motion information can include can be determined according to a size of the coding tree unit or the merge processing region.

[0332] A current block included in a coding tree unit or a merge processing region can be set not to use a temporary list of motion information for the coding tree unit or the merge processing region. That is, when a number of merge candidates included in a merge candidate list of the current block is less than a threshold value, an inter merge candidate included in a list of inter motion information is added to the merge candidate list, and a temporary merge candidate included in the temporary list of motion information can not be added to the merge candidate list. Thereby, motion information of another block included in the same coding tree unit or the merge processing region as the current block can not be used for motion-compensated prediction of the current block.

[0333] When encoding / decoding of all blocks included in a coding tree unit or a merge processing region is completed, a list of inter motion information and a temporary list of motion information can be merged.

[0334] Figure 27 FIG. 1 is a diagram illustrating a temporary list of motion information.

[0335] When the encoding / decoding of all blocks included in the coding tree unit or the merge processing region is completed, as shown in the example of FIG. 6, the inter motion information list can be updated with the temporary merge candidates included in the temporary motion information list. Figure 27

[0336] At this time, the temporary merge candidates included in the temporary motion information list can be added to the inter motion information list in the order inserted into the temporary motion information list, i.e., in ascending or descending order of index values.

[0337] As another example, the temporary merge candidates included in the temporary motion information list can be added to the inter motion information list in a predefined order.

[0338] Here, the predefined order can be determined according to a scanning order of the blocks encoded in the merge processing region or the coding tree unit. The scanning order can be at least one of raster scanning, horizontal scanning, vertical scanning, or zigzag scanning. Alternatively, the predefined order can be determined based on the number of blocks having motion information or the same motion information.

[0339] Alternatively, the temporary merge candidates including uni-directional motion information can be added to the inter merge list before the temporary merge candidates including bi-directional motion information. On the other hand, the temporary merge candidates including bi-directional motion information can be added to the inter merge candidate list before the temporary merge candidates including uni-directional motion information.

[0340] Alternatively, the temporary merge candidates can be added to the inter motion information list according to an order of high or low usage frequency within the merge processing region or the coding tree unit.

[0341] When the temporary merge candidates included in the temporary motion information list are added to the inter motion information list, a redundancy detection can be performed on the temporary merge candidates. For example, when the same inter merge candidate as the temporary merge candidate included in the temporary motion information list is already stored in the inter motion information list, the temporary merge candidate can not be added to the inter motion information list. At this time, the redundancy detection can be performed on some of the inter merge candidates included in the inter motion information list. For example, the redundancy detection can be performed on the inter prediction merge candidates having an index greater than or equal to a threshold value. For example, when the temporary merge candidate is the same as the inter merge candidate having an index greater than or equal to a predefined value, the temporary merge candidate can not be added to the inter motion information list.

[0342] Intra prediction is to predict a current block using reconstructed samples of a periphery of the current block that have been encoded / decoded. In this case, the intra prediction of the current block can use the reconstructed samples before an in-loop filter is applied. ​

[0343] Intra prediction techniques include matrix-based intra prediction and general intra prediction considering directionality with neighboring reconstructed samples. Information indicating an intra prediction technique of a current block can be signaled through a bitstream. The information can be a 1-bit flag. Alternatively, the intra prediction technique of the current block can be determined based on at least one of a position, a size, a shape of the current block, or an intra prediction technique of a neighboring block. For example, when the current block exists across an image boundary, the current block is set not to apply the matrix-based intra prediction.

[0344] The matrix-based intra prediction is a method of obtaining a prediction block of a current block based on matrix multiplication between a matrix stored in an encoder and a decoder and reconstructed samples around the current block. Information for specifying any one of a plurality of stored matrices can be signaled through a bitstream. The decoder can determine a matrix for intra prediction of the current block based on the information and a size of the current block.

[0345] The general intra prediction is a method of obtaining a prediction block related to a current block based on a non-angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on the general intra prediction is described in more detail with reference to the accompanying drawings.

[0346] Figure 28 is a flowchart of an intra prediction method according to an embodiment of the present application.

[0347] A reference sample line of a current block can be determined (S2801). The reference sample line refers to a set of reference samples included in a Kth row or column deviated from above and / or left of the current block. The reference samples can be derived from reconstructed samples of a current block periphery that have been encoded / decoded.

[0348] Index information identifying a reference sample line of a current block among a plurality of reference sample lines can be signaled through a bitstream. The plurality of reference sample lines can include at least one of a first row / column, a second row / column, a third row / column, or a fourth row / column above and / or left of the current block. Table 4 shows an index assigned to each reference sample line. In Table 4, it is assumed that the first row / column, the second row / column, and the fourth row / column are used as reference sample line candidates.

[0349]

Table 4

[0350] Index Reference sample line 0 First reference sample line 1 Second reference sample line 2 Fourth reference sample line

[0351] The reference sample line of the current block can also be determined based on at least one of a position, a size, a shape of the current block, or a prediction encoding mode of a neighboring block. For example, when the current block is adjacent to a boundary of an image, a tile, a slice, or a coding tree unit, a first reference sample line can be determined as the reference sample line of the current block.

[0352] The reference sample line can include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper and left reference samples can be derived from the reconstructed samples surrounding the current block. The reconstructed samples can be in a state prior to the application of the in-loop filter.

[0353] Figure 29 This is a diagram showing the reference samples included in each reference sample line.

[0354] Based on the intra-prediction mode of the current block, a prediction sample can be obtained using at least one of the reference samples belonging to the reference sample line.

[0355] Next, the intra-prediction mode of the current block can be determined (S2802). For the intra-prediction mode of the current block, at least one of a non-angular intra-prediction mode or an angular intra-prediction mode can be determined as the intra-prediction mode of the current block. Non-angular intra-prediction modes include Planer and DC, and angular intra-prediction modes include 33 or 65 modes from the lower left diagonal to the upper right diagonal.

[0356] Figure 30 This is a diagram illustrating the intra-frame prediction mode.

[0357] Figure 30 (a) shows 35 intra-frame prediction modes. Figure 30 (b) shows 67 intra-frame prediction modes.

[0358] It can also be defined as compared to Figure 30 The number of intra-frame prediction modes shown is either more or less.

[0359] The Most Probable Mode (MPM) can be set based on the intra-prediction modes of neighboring blocks adjacent to the current block. Neighboring blocks can include the left-side neighboring block to the left of the current block and the top-side neighboring block above the current block. When the coordinates of the top-left sample of the current block are (0, 0), the left-side neighboring block can include samples at positions (-1, 0), (-1, H-1), or (-1, (H-1) / 2), where H represents the height of the current block. The top-side neighboring block can include samples at positions (0, -1), (W-1, -1), or ((W-1) / 2, -1), where W represents the width of the current block.

[0360] When the neighboring blocks are coded with general intra prediction, the MPMs can be derived based on the intra prediction modes of the neighboring blocks. Specifically, the intra prediction mode of the left neighboring block can be set as a variable candIntraPredModeA, and the intra prediction mode of the above neighboring block can be set as a variable candIntraPredModeB.

[0361] At this time, when the neighboring blocks are unavailable (for example, when the neighboring blocks have not been coded / decoded or the positions of the neighboring blocks deviate from the image boundary), the variable candIntraPredModeX (where X is A or B) derived based on the intra prediction mode of the neighboring blocks can be set as a default mode in a case where the neighboring blocks are coded with matrix-based intra prediction, in a case where the neighboring blocks are coded with inter prediction, or in a case where the neighboring blocks are included in a coding tree unit different from the current block. The default mode can include at least one of a planar mode, a DC mode, a vertical direction mode, or a horizontal direction mode.

[0362] Alternatively, when the neighboring blocks are coded with matrix-based intra prediction, the intra prediction mode corresponding to the index value used to specify any one of the matrices can be set as candIntraPredModeX. To this end, a look-up table indicating the mapping relationship of the index value used to specify the matrix and the intra prediction mode can be pre-stored in the encoder and the decoder.

[0363] The MPMs can be derived based on the variable candIntraPredModeA and the variable candIntraPredModeB. The number of MPMs included in the MPM list can be predefined in the encoder and the decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs can be signaled through a bitstream. Alternatively, the number of MPMs can be determined based on at least one of the prediction coding mode of the neighboring blocks, the size or shape of the current block.

[0364] In the embodiments described later, it is assumed that the number of MPMs is 3, and the three MPMs will be referred to as MPM[0], MPM[1], and MPM[2]. When the number of MPMs is more than 3, the MPMs can include the three MPMs explained in the embodiments described later.

[0365] When candIntraPredA is the same as candIntraPredB and candIntraPredA is a planar mode or a DC mode, MPM[0] and MPM[1] can be set to the planar mode and the DC mode, respectively. MPM[2] can be set to a vertical intra prediction mode, a horizontal intra prediction mode, or a diagonal intra prediction mode. The diagonal intra prediction mode can be a lower-left diagonal intra prediction mode, an upper-left intra prediction mode, or an upper-right intra prediction mode.

[0366] When candIntraPredA is the same as candIntraPredB and candIntraPredA is an intra prediction mode, MPM[0] can be set to be the same as candIntraPredA. MPM[1] and MPM[2] can be set to intra prediction modes similar to candIntraPredA. The intra prediction modes similar to candIntraPredA can be intra prediction modes with an index difference of ±1 or ±2 from candIntraPredA. The intra prediction modes similar to candIntraPredA can be derived using a modulo operation (%) and an offset.

[0367] When candIntraPredA is the same as candIntraPredB and candIntraPredA is an intra prediction mode, MPM[0] can be set to be the same as candIntraPredA. MPM[1] and MPM[2] can be set to intra prediction modes similar to candIntraPredA. The intra prediction modes similar to candIntraPredA can be intra prediction modes with an index difference of ±1 or ±2 from candIntraPredA. The intra prediction modes similar to candIntraPredA can be derived using a modulo operation (%) and an offset.

[0368] An MPM list including a plurality of MPMs can be generated, and information indicating whether an MPM identical to an intra prediction mode of a current block is included in the MPM list can be signaled through a bitstream. The information is a 1-bit flag, which can be referred to as an MPM flag. When the MPM flag indicates that an MPM identical to the current block is included in the MPM list, index information identifying one of the MPMs can be signaled through the bitstream. The MPM specified by the index information can be set as the intra prediction mode of the current block. When the MPM flag indicates that an MPM identical to the current block is not included in the MPM list, residual mode information indicating any one of residual intra prediction modes other than the MPMs can be signaled through the bitstream. The residual mode information represents an index value corresponding to the intra prediction mode of the current block when the index is reassigned to the residual intra prediction mode other than the MPMs. A decoder can arrange the MPMs in ascending order, and determine the intra prediction mode of the current block by comparing the residual mode information with the MPMs. For example, when the residual mode information is the same as or less than the MPM, the intra prediction mode of the current block can be derived by adding 1 to the residual mode information.

[0369] Instead of the operation of setting a default mode as an MPM, information indicating whether the intra prediction mode of the current block is the default mode can be signaled through a bitstream. The information is a 1-bit flag, and the flag can be referred to as a default mode flag. The default mode flag can be signaled only when the MPM flag indicates that an MPM identical to the current block is included in the MPM list. As described above, the default mode can include at least one of a planar mode, a DC mode, a vertical direction mode, or a horizontal direction mode. For example, when the planar mode is set as the default mode, the default mode flag can indicate whether the intra prediction mode of the current block is the planar mode. When the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information can be set as the intra prediction mode of the current block.

[0370] When a plurality of intra prediction modes are set as default modes, index information indicating any one of the default modes can be further signaled. The intra prediction mode of the current block can be set as the default mode indicated by the index information.

[0371] When the index of the reference sample line of the current block is not 0, the default mode is set not to be used. Thereby, when the index of the reference sample line is not 0, the default mode flag is not signaled, and a value of the default mode flag can be set as a pre-defined value (i.e., false).

[0372] When the intra prediction mode of the current block is determined, a prediction sample of the current block can be obtained based on the determined intra prediction mode (S2803).

[0373] When the DC mode is selected, the prediction samples related to the current block can be generated based on the average value of the reference samples. Specifically, the values of the overall samples within the prediction block can be generated based on the average value of the reference samples. The average value can be derived using at least one of the above reference samples located above the current block and the left reference samples located to the left of the current block.

[0374] According to the shape of the current block, the number or range of the reference samples used in deriving the average value can be different. For example, when the current block is a non-square block with a width greater than a height, only the above reference samples can be used to calculate the average value. On the other hand, when the current block is a non-square block with a width less than a height, only the left reference samples can be used to calculate the average value. That is, when the width and height of the current block are different, only the reference samples adjacent to the side with the longer length can be used to calculate the average value. Alternatively, whether to use only the above reference samples to calculate the average value or to use only the left reference samples to calculate the average value can be determined based on the width and height ratio of the current block.

[0375] When the planar mode is selected, the prediction samples can be obtained using the horizontal direction prediction samples and the vertical direction prediction samples. The horizontal direction prediction samples are obtained based on the left reference samples and the right reference samples located on the same horizontal line as the prediction samples, and the vertical direction prediction samples are obtained based on the above reference samples and the below reference samples located on the same vertical line as the prediction samples. The right reference samples can be generated by copying the reference sample adjacent to the top right corner of the current block, and the below reference samples can be generated by copying the reference sample adjacent to the bottom left corner of the current block. The horizontal direction prediction samples can be obtained based on the weighted sum operation of the left reference samples and the right reference samples, and the vertical direction prediction samples can be obtained based on the weighted sum operation of the above reference samples and the below reference samples. In this case, the weighting values assigned to each reference sample can be determined according to the position of the prediction sample. The prediction samples can be obtained based on the average operation or the weighted sum operation of the horizontal direction prediction samples and the vertical direction prediction samples. When the weighted sum operation is performed, the weighting values assigned to the horizontal direction prediction samples and the vertical direction prediction samples can be determined based on the position of the prediction sample.

[0376] When the angular prediction mode is selected, a parameter representing the prediction direction (or the prediction angle) of the selected angular prediction mode can be determined. Table 5 below shows the intra prediction parameter intraPredAng for each intra prediction mode.

[0377] [Table 5]

[0378]

[0379] Table 5 shows the intra direction parameters of each of the intra prediction modes having an index of any one of 2 to 34 when 35 intra prediction modes are defined. When more than 33 angular intra prediction modes are defined, Table 5 is further subdivided to set the intra direction parameters of each angular intra prediction mode.

[0380] After arranging the above reference samples and the left reference samples of the current block into a column, the prediction samples can be obtained based on the value of the intra direction parameter. In this case, when the value of the intra direction parameter is negative, the left reference samples and the above reference samples can be arranged into a column.

[0381] Figure 31 and Figure 32 is a diagram showing an example of a one-dimensional array of arranging the reference samples into a row.

[0382] Figure 31 shows an example of a vertical direction one-dimensional array of arranging the reference samples in a vertical direction, and Figure 32 shows an example of a horizontal direction one-dimensional array of arranging the reference samples in a horizontal direction. Embodiments of Figure 31 and Figure 32 will be described under the assumption that 35 intra prediction modes are defined.

[0383] When the intra prediction mode index is any one of 11 to 18, a horizontal direction one-dimensional arrangement of the above reference samples can be applied in a counterclockwise direction, and when the intra prediction mode index is any one of 19 to 25, a vertical direction one-dimensional arrangement of the left reference samples can be applied in a clockwise direction. When the reference samples are arranged into a column, the intra prediction mode angle can be considered.

[0384] Based on the intra direction parameter, a reference sample determination parameter can be determined. The reference sample determination parameter can include a reference sample index for specifying the reference sample and a weight parameter for determining a weight value applied to the reference sample.

[0385] The reference sample index iIdx and the weight parameter ifact can be obtained by Equations 5 and 6 below, respectively.

[0386] [Equation 5]

[0387] iIdx = (y + 1) * P ang / 32

[0388] [Equation 6]

[0389] i fact = [(y + 1) * P ang ] & 31

[0390] In Equations 5 and 6, P angrepresents an intra direction parameter. A reference sample specified according to a reference sample index iIdx corresponds to an integer pel.

[0391] To derive a prediction sample, more than one reference sample can be specified. Specifically, considering a slope of a prediction mode, a position of a reference sample used when deriving a prediction sample can be specified. For example, using a reference sample index iIdx, a reference sample used when deriving a prediction sample can be specified.

[0392] In this case, when a slope of an intra prediction mode is not represented by one reference sample, a prediction sample can be generated by interpolating a plurality of reference samples. For example, when a slope of an intra prediction mode is a value between a slope between a prediction sample and a first reference sample and a slope between the prediction sample and a second reference sample, the first reference sample and the second reference sample can be interpolated to obtain the prediction sample. That is, when an angular line following an intra prediction angle does not pass through a reference sample located at an integer pel, reference samples adjacent to a position where the angular line passes, left or right or above or below, can be interpolated to obtain the prediction sample.

[0393] Equation 7 below shows an example of obtaining a prediction sample based on a reference sample.

[0394] [Equation 7]

[0395] P(x, y) = ((32 - i fact ) / 32) * Ref_1D(x + iIdx + 1) + (i fac t / 32) * Ref_1D(x + iIdx + 2)

[0396] In Equation 7, P represents a prediction sample, and Ref_1D represents any one of one-dimensionally arranged reference samples. In this case, a position of a reference sample can be determined according to a position (x, y) of a prediction sample and a reference sample index iIdx.

[0397] When a slope of an intra prediction mode can be represented by one reference sample, a weighting value parameter i fact is set to 0. Accordingly, Equation 7 can be simplified as shown in Equation 8 below.

[0398] [Equation 8]

[0399] P(x, y) = Ref_ID(x + iIdx + 1)

[0400] Intra prediction can also be performed on a current block based on a plurality of intra prediction modes. For example, an intra prediction mode can be derived for different prediction samples, and a prediction sample can be derived based on an intra prediction mode assigned to each prediction sample.

[0401] Alternatively, the intra prediction mode can be derived for different regions, and the intra prediction can be performed for the regions based on the intra prediction modes assigned to the regions. The regions can include at least one sample. At least one of the size or shape of the regions can be adaptively determined based on at least one of the size, shape, or intra prediction mode of the current block. Alternatively, at least one of the size or shape of the regions can be predefined in the encoder and the decoder regardless of the size or shape of the current block.

[0402] Alternatively, the intra prediction can be performed based on a plurality of intra prediction modes, respectively, and a final prediction sample can be derived based on an average operation or a weighted sum operation of a plurality of prediction samples obtained through the plurality of intra predictions. For example, the intra prediction can be performed based on a first intra prediction mode to obtain a first prediction sample, and the intra prediction can be performed based on a second intra prediction mode to obtain a second prediction sample. Thereafter, a final prediction sample can be obtained based on an average operation or a weighted sum operation between the first prediction sample and the second prediction sample. In this case, the weighted values respectively assigned to the first prediction sample and the second prediction sample can be determined considering at least one of whether the first intra prediction mode is a non-angular / angle prediction mode, whether the second intra prediction mode is a non-angular / angle prediction mode, or an intra prediction mode of a neighboring block.

[0403] The plurality of intra prediction modes can be a combination of non-angular intra prediction modes and angle prediction modes, a combination of angle prediction modes, or a combination of non-angular prediction modes.

[0404] Figure 33 is a diagram illustrating an angle formed by an angle intra prediction mode and a straight line parallel to the x-axis.

[0405] As Figure 33 In the example shown in FIG. 6B, the angle prediction mode can exist between the left-down diagonal direction and the right-up diagonal direction. When described as an angle formed by the x-axis and the angle prediction mode, the angle prediction mode can exist between 45 degrees (left-down diagonal direction) and -135 degrees (right-up diagonal direction).

[0406] In the case where the current block is not a square shape, the following case can occur that, depending on the intra prediction mode of the current block, a prediction sample is derived using a reference sample located on an angle line following an intra prediction angle, which is farther from the prediction sample rather than a reference sample closer to the prediction sample.

[0407] Figure 34 is a diagram illustrating a prediction sample obtained in the case where the current block is not a square.

[0408] For example, as in Figure 34In the example shown in (a), it is assumed that the current block is a non-square shape with a width greater than a height, and that the intra prediction mode of the current block is an angular intra prediction mode with an angle between 0 degree and 45 degrees. In this case, when deriving the prediction sample A near the right side column of the current block, a case occurs in which a left side reference sample L among the reference samples located on the angular mode of the angle is used instead of an above reference sample T close to the prediction sample.

[0409] As another example, as in the example shown in Figure 34 In the example shown in (b), it is assumed that the current block is a non-square shape with a height greater than a width, and that the intra prediction mode of the current block is an angular intra prediction mode with an angle between -90 degrees and -135 degrees. In the above case, when deriving the prediction sample A near the below row of the current block, a case occurs in which an above reference sample T among the reference samples located on the angular mode of the angle is used instead of a left side reference sample L close to the prediction sample.

[0410] To resolve the above problem, when the current block is a non-square, the intra prediction mode of the current block can be replaced with an inverse direction intra prediction mode. Thus, for a non-square block, an angular intra prediction mode with an angle greater or less than the angle of the angular prediction mode shown in (a) can be used. Such an angular intra prediction mode can be defined as a wide angle intra prediction mode. The wide angle intra prediction mode indicates an angular intra prediction mode not falling within the range of 45 degrees to -135 degrees. Figure 24

[0411] Figure 35 is a diagram showing wide angle intra prediction modes.

[0412] In the example shown in Figure 35 The intra prediction modes with indices -1 to -14 and the intra prediction modes with indices 67 to 80 indicate wide angle intra prediction modes.

[0413] Although 14 wide angle intra prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide angle intra prediction modes (67 to 80) with angles less than -135 degrees are shown in Figure 35 , a greater or smaller number of wide angle intra prediction modes can be defined.

[0414] When a wide angle intra prediction mode is used, the length of the above reference sample is set to 2W+1, and the length of the left side reference sample is set to 2H+1.

[0415] When a wide angle intra prediction mode is used, the sample A shown in (a) can be predicted using the reference sample T, and the sample B shown in (b) can be predicted using the reference sample L. Figure 34 Figure 34 ​​(b) the illustrated sample A.

[0416] By adding the existing intra prediction modes and the N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 6 shows the intra direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0417]

Table 6

[0418] PredModeIntra -10 -9 -8 -7 -6 -5 -4 -3 -2 intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -1 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 11 12 13 14 15 16 17 18 intraPredAngle 13 11 9 7 5 3 2 1 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -1 -2 -3 -5 -7 -9 -11 -13 -15 PredModeIntra 28 29 30 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -11 -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -1 0 1 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 11 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

[0419] When the current block is non-square and the intra prediction mode of the current block obtained in step S2802 falls within a transform range, the intra prediction mode of the current block can be transformed into a wide-angle intra prediction mode. The transform range can be determined based on at least one of the size, shape, or ratio of the current block. The ratio can represent the ratio between the width and the height of the current block.

[0420] When the current block is a non-square with a width greater than a height, the transform range can be set to the intra prediction mode index of the right-up diagonal direction (e.g., 66) to (the index of the intra prediction mode of the right-up diagonal direction - N). N can be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transform range, the intra prediction mode can be transformed into a wide-angle intra prediction mode. The transform can be performed by subtracting a predefined value from the intra prediction mode, which can be the total number of intra prediction modes other than the wide-angle intra prediction modes (e.g., 67).

[0421] According to the embodiment, the intra prediction modes between the 66th and the 53rd can be transformed into the wide-angle intra prediction modes between the -1st and the -14th, respectively.

[0422] When the current block is a non-square with a height greater than a width, the transform range can be set to the intra prediction mode index of the left-down diagonal direction (e.g., 2) to (the index of the intra prediction mode of the left-down diagonal direction + M). M can be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transform range, the intra prediction mode can be transformed into a wide-angle intra prediction mode. The transform can be performed by adding a predefined value to the intra prediction mode, which can be the total number of angle intra prediction modes other than the wide-angle intra prediction modes (e.g., 65).

[0423] According to the embodiment, the intra prediction modes between the 2nd and the 15th are transformed into the wide-angle intra prediction modes between the 67th and the 80th, respectively.

[0424] Hereinafter, the intra prediction mode falling within the transform range is referred to as a wide-angle intra replacement prediction mode.

[0425] The transform range can be determined based on the ratio of the current block. For example, Table 7 and Table 8 each illustrate the transform range when 35 intra prediction modes and 67 intra prediction modes are defined in addition to the wide-angle intra prediction mode.

[0426]

Table 7

[0427] Condition Replacement intra prediction mode W / H = 2 Modes 2, 3, 4 W / H > 2 Modes 2, 3, 4, 5, 6 W / H = 1 None H / W = 1 / 2 Modes 32, 33, 34 H / W <1 / 2 Modes 30, 31, 32, 33, 34

[0428]

Table 8

[0429] Condition Replacement intra prediction mode W / H = 2 Modes 2, 3, 4, 5, 6, 7 W / H > 2 Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H = 1 None H / W = 1 / 2 Modes 61, 62, 63, 64, 65, 66 H / W <1 / 2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0430] As in the example illustrated in Table 7 and Table 8, the number of wide-angle intra replacement prediction modes falling within the transform range can differ according to the ratio of the current block.

[0431] As the wide-angle intra prediction mode is used in addition to the existing intra prediction mode, the resources required to encode the wide-angle intra prediction mode can increase, thus possibly reducing the encoding efficiency. Thus, instead of directly encoding the wide-angle intra prediction mode, the replacement intra prediction mode related to the wide-angle intra prediction mode is encoded, thus the encoding efficiency can be improved.

[0432] For example, when the current block is encoded using the 67th wide-angle intra prediction mode, the number 2, which is the 67th wide-angle replacement intra prediction mode, can be encoded as the intra prediction mode of the current block. Also, when the current block is encoded using the -1th wide-angle intra prediction mode, the number 66, which is the -1th wide-angle replacement intra prediction mode, can be encoded as the intra prediction mode of the current block.

[0433] The decoder can decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode is included in the transform range. When the decoded intra prediction mode is the wide-angle replacement intra prediction mode, the intra prediction mode can be transformed into the wide-angle intra prediction mode.

[0434] Alternatively, when the current block is encoded in the wide-angle intra prediction mode, the wide-angle intra prediction mode can also be directly encoded.

[0435] The encoding of the intra prediction mode can be implemented based on the MPM list. Specifically, when the neighboring block is encoded in the wide-angle intra prediction mode, the MPM can be set based on the wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode. For example, when the neighboring block is encoded in the wide-angle intra prediction mode, the variable candIntraPredX (X is A or B) can be set to the wide-angle replacement intra prediction mode.

[0436] If the prediction block is generated by the intra prediction result, the prediction samples can be updated based on the position of each prediction sample included in the prediction block. The update method as described above can be referred to as a sample position based intra weighting prediction method (or Position Dependent Prediction Combination, PDPC).

[0437] It can be considered whether to use PDPC based on the intra prediction mode of the current block, the reference sample line of the current block, the size of the current block, or the color component. For example, when the intra prediction mode of the current block is at least one of a planar mode, a DC mode, a vertical direction mode, a horizontal direction mode, a mode with an index value smaller than the vertical direction, or a mode with an index value larger than the horizontal direction, PDPC can be used. Alternatively, PDPC can be used only when at least one of the width or the height of the current block is greater than 4. Alternatively, PDPC can be used only when the index of the reference image line of the current block is 0. Alternatively, PDPC can be used only when the index of the reference image line of the current block is greater than or equal to a predefined value. Alternatively, PDPC can be used only for the luminance component. Alternatively, whether to use PDPC can be determined according to whether two or more of the enumerated conditions are satisfied.

[0438] As another example, information indicating whether to apply PDPC can be signaled through a bitstream.

[0439] If the prediction sample is obtained through the intra prediction sample, based on the position of the obtained prediction sample, a reference sample for correcting the prediction sample can be determined. For convenience of description, in the following embodiments, the reference sample for correcting the prediction sample is referred to as a PDPC reference sample. Moreover, the prediction sample obtained through the intra prediction is referred to as a first prediction sample, and the prediction sample obtained by correcting the first prediction sample is referred to as a second prediction sample.

[0440] Figure 36 is a diagram illustrating application of PDPC.

[0441] The first prediction sample can be corrected using at least one PDPC reference sample. The PDPC reference sample can include at least one of a reference sample adjacent to the top left corner of the current block, an upper reference sample located above the current block, or a left reference sample located to the left of the current block.

[0442] At least one of the reference samples belonging to the reference sample line of the current block can be set as the PDPC reference sample. Alternatively, regardless of the reference sample line of the current block, at least one of the reference samples belonging to the reference sample line having an index of 0 can be set as the PDPC reference sample. For example, even if the first prediction sample is obtained using the reference sample included in the reference sample line having an index of 1 or an index of 2, the second prediction sample can be obtained using the reference sample included in the reference sample line having an index of 0.

[0443] The number or position of the PDPC reference sample used to correct the first prediction sample can be determined in consideration of at least one of the intra prediction mode of the current block, the size of the current block, the shape of the current block, or the position of the first prediction sample.

[0444] For example, when the intra prediction mode of the current block is a planar mode or a DC mode, the second prediction sample can be obtained using the upper reference sample and the left reference sample. At this time, the upper reference sample can be a reference sample perpendicular to the first prediction sample (for example, a reference sample having the same x coordinate), and the left reference sample can be a reference sample horizontal to the first prediction sample (for example, a reference sample having the same y coordinate).

[0445] When the intra prediction mode of the current block is a horizontal intra prediction mode, the second prediction sample can be obtained using the upper reference sample. At this time, the upper reference sample can be a reference sample perpendicular to the first prediction sample.

[0446] When the intra prediction mode of the current block is a vertical intra prediction mode, the second prediction sample can be obtained using the left reference sample. At this time, the left reference sample can be a reference sample horizontal to the first prediction sample.

[0447] When the intra prediction mode of the current block is a lower-left diagonal intra prediction mode or an upper-right diagonal intra prediction mode, the second prediction sample can be obtained based on an upper-left reference sample, an upper reference sample, and a left reference sample. The upper-left reference sample can be a reference sample adjacent to the upper-left corner of the current block (for example, a reference sample at a position of (-1, -1)). The upper reference sample can be a reference sample located in the upper-right diagonal direction of the first prediction sample, and the left reference sample can be a reference sample located in the lower-left diagonal direction of the first prediction sample.

[0448] In summary, when the position of the first prediction sample is (x, y), R(-1, -1) can be set as the upper-left reference sample, and R(x+y+1, -1) or R(x, -1) can be set as the upper reference sample. In addition, R(-1, x+y+1) or R(-1, y) can be set as the left reference sample.

[0449] As another example, at least one of a shape of the current block or whether a wide-angle intra mode is applied can be considered to determine a position of the left reference sample or the above reference sample.

[0450] Specifically, when the intra prediction mode of the current block is the wide-angle intra prediction mode, a reference sample located at a certain offset value from the reference sample in a diagonal direction of the first prediction sample can be set as a PDPC reference sample. For example, the above reference sample R(x+y+k+1, -1) and the left reference sample R(-1, x+y-k+1) can be set as the PDPC reference sample.

[0451] At this time, the offset value k can be determined based on the wide-angle intra prediction mode. Equation 9 and Equation 10 show an example of deriving the offset value based on the wide-angle intra prediction mode.

[0452] [Equation 9]

[0453] k = CurrIntraMode - 66

[0454] if (CurrIntraMode > 66)

[0455] [Equation 10]

[0456] k = -CurrIntraMode

[0457] if (CurrIntraMode < 0)

[0458] The second prediction sample can be determined based on a weighted sum operation between the first prediction sample and the PDPC reference sample. For example, the second prediction sample can be obtained based on Equation 11 below.

[0459] [Equation 11]

[0460] pred(x, y) = (xL * R L + wT * R T - wTL * R TL + (64 - wL - wT + wTL) * pred(x, y) + 32) >> 6

[0461] In Equation 11, RL denotes the left reference sample, RT denotes the above reference sample, and RTL represents the top-left reference sample. pred(x, y) denotes a prediction sample at a position (x, y). wL denotes a weight value assigned to the left reference sample, wT denotes a weight value assigned to the above reference sample, and wTL denotes a weight value assigned to the top-left reference sample. A weight value assigned to the first prediction sample can be derived by subtracting the weight values assigned to the reference samples from a maximum value. For convenience of explanation, the weight value assigned to the PDPC reference sample is referred to as a PDPC weight value.

[0462] The weighting values assigned to the reference samples can be determined based on at least one of an intra prediction mode of the current block or a position of the first prediction sample.

[0463] For example, at least one of wL, wT or wTL can be in a proportional or inverse proportional relationship with at least one of an x-axis coordinate value or a y-axis coordinate value of the prediction sample. Alternatively, at least one of wL, wT or wTL can be in a proportional or inverse proportional relationship with at least one of a width or a height of the current block.

[0464] When the intra prediction mode of the current block is a DC mode, the PDPC weighting values can be determined as shown in Equation 12.

[0465]

Equation 12

[0466] wT = 32 » ((y < 1) » shift)

[0467] wL = 32 » ((x << 1) » shift)

[0468] wTL = (wL » 4) + (wT » 4)

[0469] In Equation 12, x and y denote the position of the first prediction sample.

[0470] In Equation 12, a variable shift used in the shift operation can be derived based on a width or a height of the current block. For example, the variable shift can be derived based on Equation 13 or Equation 14 as follows.

[0471]

Equation 13

[0472] shift = (log2(width) - 2 + log2(height) - 2 + 2) » 2

[0473]

Equation 14

[0474] shift = ((Log2(nTbW) + Log2(nTbH) - 2) » 2)

[0475] Alternatively, the variable shift can also be derived considering an intra direction parameter of the current block.

[0476] The number or kind of parameters used to derive the variable shift can be determined differently according to the intra prediction mode of the current block. For example, when the intra prediction mode of the current block is the planar mode, the DC mode, the vertical direction mode, or the horizontal direction mode, the width and the height of the current block can be used to derive the variable shift as shown in Equation 13 or Equation 14. When the intra prediction mode of the current block is an intra prediction mode having a greater index than the vertical direction intra prediction mode, the height of the current block and the intra direction parameter can be used to derive the variable shift. When the intra prediction mode of the current block is an intra prediction mode having a smaller index than the horizontal direction intra prediction mode, the width of the current block and the intra direction parameter can be used to derive the variable shift.

[0477] When the intra prediction mode of the current block is the planar mode, the value of wTL can be set to 0. wL and wT can be derived based on Equation 15 below.

[0478] [Equation 15]

[0479] wT[y] = 32 » ((y « 1) » nScale)

[0480] wL[x] = 32 » ((x « 1) » nScale)

[0481] When the intra prediction mode of the current block is the horizontal direction intra prediction mode, wT can be set to 0, and wTL and wL can be set to be the same. On the other hand, when the intra prediction mode of the current block is the vertical direction intra prediction mode, wL can be set to 0, and wTL and wT can be set to be the same.

[0482] When the intra prediction mode of the current block is an intra prediction mode pointing in the upper right direction and having an index value greater than the vertical direction intra prediction mode, the PDPC weighting value can be derived as shown in Equation 16 below.

[0483] [Equation 16]

[0484] wT = 16 » ((y « 1) » shift)

[0485] wL = 16 » ((x « 1) » shift)

[0486] wTL = 0

[0487] On the other hand, when the intra prediction mode of the current block is an intra prediction mode pointing in the lower left direction and having an index value smaller than the horizontal direction intra prediction mode, the PDPC weighting value can be derived as shown in Equation 17 below.

[0488] [Equation 17]

[0489] wT16 » ((y « 1) » shift)

[0490] wL = 16 » ((x « 1) » shift)

[0491] wTL = 0

[0492] As described in the above embodiments, the PDPC weight value can be determined based on the positions x and y of the prediction samples.

[0493] As another example, the weight value assigned to each PDPC reference sample can also be determined in units of sub-blocks. The prediction samples included in a sub-block can share the same PDPC weight value.

[0494] The size of the sub-blocks, which are the basic units for determining the weight values, can be predefined in the encoder and the decoder. For example, the weight values can be determined for each sub-block having a size of 2x2 or 4x4.

[0495] Alternatively, the size, shape, or number of the sub-blocks can be determined according to the size or shape of the current block. For example, the coding block can be divided into 4 sub-blocks regardless of the size of the coding block. Alternatively, the coding block can be divided into 4 or 16 sub-blocks according to the size of the coding block.

[0496] Alternatively, the size, shape, or number of the sub-blocks can also be determined based on the intra prediction mode of the current block. For example, when the intra prediction mode of the current block is a horizontal direction mode, N columns (or N rows) can be set as one sub-block, and conversely, when the intra prediction mode of the current block is a vertical direction mode, N rows (or N columns) can be set as one sub-block.

[0497] Equations 18 to 20 show an example of determining the PDPC weight values of a 2x2 size sub-block. Equation 18 shows an example when the intra prediction mode of the current block is a DC mode.

[0498] [Equation 18]

[0499] wT = 32 » (((y « log2K) » log2K) « 1) » shift)

[0500] wL = 32 » (((x « log2K) » log2K) « 1) » shift)

[0501] wTL = (wL » 4) + (wT » 4)

[0502] In Equation 18, K can be determined based on the size of the sub-block.

[0503] Equation 19 shows an example where the intra prediction mode of the current block is an intra prediction mode with an index value greater than the intra prediction mode of the vertical direction, pointing to the upper right direction.

[0504] [Equation 19]

[0505] wT = 16 » (((y « log2K) » log2K) « 1) » shift

[0506] wL = 16 » (((x « log2K) » log2K) « 1) » shift

[0507] wTL = 0

[0508] Equation 20 shows an example where the intra prediction mode of the current block is an intra prediction mode with an index value less than the intra prediction mode of the horizontal direction, pointing to the lower left direction.

[0509] [Equation 20]

[0510] wT = 16 » (((y « log2K) » log2K) « 1) » shift

[0511] wL = 16 » (((x « log2K) » log2K) « 1) » shift

[0512] wTL = 0

[0513] In Equations 18 to 20, x and y represent the positions of the reference samples within the sub-block. The reference samples can be any one of the samples located at the top left of the sub-block, the center of the sub-block, or the bottom right of the sub-block.

[0514] Equations 21 to 23 show examples of determining the PDPC weighting values for a 4x4 size sub-block. Equation 21 shows an example where the intra prediction mode of the current block is the DC mode.

[0515] [Equation 21]

[0516] wT = 32 » (((y « 2) » 2) « 1) » shift

[0517] wL = 32 » (((x « 2) » 2) « 1) » shift

[0518] wTL = (wL » 4) + (wT » 4)

[0519] Equation 22 shows an example where the intra prediction mode of the current block is an intra prediction mode with an index value greater than the intra prediction mode of the vertical direction, pointing to the upper right direction.

[0520] [Equation 22]

[0521] wT = 16 » (((y « 2) » 2) « 1) » shift

[0522] wL = 16 » (((x « 2) » 2) « 1) » shift

[0523] wTL = 0

[0524] Equation 23 shows an example of deriving the PDPC weighting values when the intra prediction mode of the current block is a left-downward pointing intra prediction mode with an index value less than the intra prediction mode of the horizontal direction.

[0525] [Equation 23]

[0526] wT = 16 » (((y « 2) » 2) « 1) » shift

[0527] wL = 16 » (((x « 2) » 2) « 1) » shift

[0528] wTL = 0

[0529] In the above embodiments, the position of the first prediction sample or the prediction samples included in the sub-block is considered to determine the PDPC weighting values. The shape of the current block can be further considered to determine the PDPC weighting values.

[0530] For example, for the DC mode, the derivation method of the PDPC weighting values can be different depending on whether the current block is a non-square block with a width greater than a height or a non-square block with a height greater than a width.

[0531] Equation 24 shows an example of deriving the PDPC weighting values when the current block is a non-square block with a width greater than a height, and Equation 25 shows an example of deriving the PDPC weighting values when the current block is a non-square block with a height greater than a width.

[0532] [Equation 24]

[0533] wT = 32 » (y » shift)

[0534] wL = 32 » (x » shift)

[0535] wTL = (wL » 4) + (wT » 4)

[0536] [Equation 25]

[0537] wT » (y » shift)

[0538] wL = 32 » ((x « 1) » shift)

[0539] wTL = (xL » 4) + (wT » 4)

[0540] In a case where the current block is a non-square, the current block can be predicted using the wide-angle intra prediction mode. In this way, when the wide-angle intra prediction mode is applied, the PDPC can also be applied to update the first prediction samples.

[0541] In a case where the wide-angle intra prediction is applied to the current block, the PDPC weighting value can be determined considering the shape of the coding block.

[0542] For example, in a case where the current block is a non-square having a width greater than a height, depending on the position of the first prediction sample, a case where an upper reference sample located above the first prediction sample is closer to the first prediction sample than a left reference sample located below and left of the first prediction sample can occur. Thus, in correcting the first prediction sample, the weighting value applied to the upper reference sample can be set to have a greater value than the weighting value applied to the left reference sample.

[0543] On the other hand, in a case where the current block is a non-square having a height greater than a width, depending on the position of the first prediction sample, a case where the left reference sample located below and left of the first prediction sample is closer to the first prediction sample than the upper reference sample located above the first prediction sample can occur. Thus, in correcting the first prediction sample, the weighting value applied to the left reference sample can be set to have a greater value than the weighting value applied to the upper reference sample.

[0544] Equation 26 illustrates an example of deriving the PDPC weighting value in a case where the intra prediction mode of the current block is the wide-angle intra prediction mode having an index greater than 66.

[0545] [Equation 26]

[0546] wT = 16 » (y » shift)

[0547] wL = 16 » ((x « 1) » shiftt)

[0548] wTL = 0

[0549] Equation 27 illustrates an example of deriving the PDPC weighting value in a case where the intra prediction mode of the current block is the wide-angle intra prediction mode having an index less than 0.

[0550] [Equation 27]

[0551] wT = 16 » ((y « 1) » shift)

[0552] wL = 16 » (x » shift)

[0553] wTL = 0

[0554] The PDPC weighting value can also be determined based on a ratio of the current block. The ratio of the current block shows a ratio of a width to a height of the current block, and can be defined as shown in Equation 28 below.

[0555] [Equation 28]

[0556] whRatio = CUwidth / CUheight

[0557] The method of deriving the PDPC weighting value can be variably determined according to an intra prediction mode of the current block.

[0558] For example, Equation 29 and Equation 30 show examples of deriving the PDPC weighting value in a case where the intra prediction mode of the current block is a DC mode. Specifically, Equation 29 is an example in a case where the current block is a non-square block having a width greater than a height, and Equation 30 is an example in a case where the current block is a non-square block having a height greater than a width.

[0559] [Equation 29]

[0560] wT = 32 » ((y « 1) » shift)

[0561] wL = 32 » (((x « 1) » whRatio) » shift)

[0562] wTL = (wL » 4) + (wT » 4)

[0563] [Equation 30]

[0564] wT = 32 » (((y « 1) » 1 / whRatio) » shift)

[0565] wL = 32 » ((x « 1) » shift)

[0566] wTL = (wL » 4) + (wT » 4)

[0567] Equation 31 shows an example of deriving the PDPC weighting value in a case where the intra prediction mode of the current block is a wide-angle intra prediction mode having an index greater than 66.

[0568] [Equation 31]

[0569] wT = 16 » (((y « 1) » 1 / whRatio) » shift)

[0570] wL = 16 » ((x « 1) » shift)

[0571] wTL = 0

[0572] Equation 32 shows an example of deriving the PDPC weighting value in the case that the intra prediction mode of the current block is a wide-angle intra prediction mode with an index less than 0.

[0573]

Equation 32

[0574] wr = 16 » ((y « 1) » shift)

[0575] wL = 16 » (((x « 1) » whRatio) » shift

[0576] wTL = 0

[0577] One prediction mode can be applied to the current block multiple times, or multiple prediction modes can be applied to the current block repeatedly. Thus, the prediction method using the same or different prediction modes can be referred to as a combined prediction mode (or a multi-hypothesis prediction mode).

[0578] Information indicating whether the combined prediction mode is applied to the current block can be signaled through a bitstream. For example, the information can be a 1-bit flag.

[0579] In the combined prediction mode, a first prediction block can be generated based on a first prediction mode, and a second prediction block can be generated based on a second prediction mode. Then, a third prediction block can be generated based on a weighted sum operation of the first prediction block and the second prediction block. The third prediction block can be set as the final prediction block of the current block.

[0580] The combined prediction mode can include at least one of a mode combining the merge mode and the merge mode, a mode combining the inter prediction and the intra prediction, a mode combining the merge mode and the advanced motion vector prediction mode, and a mode combining the merge mode and the intra prediction.

[0581] In the mode combining the merge mode and the merge mode, motion-compensated prediction can be performed using a plurality of merge candidates. Specifically, a first prediction block can be generated using a first merge candidate, and a second prediction block can be generated using a second merge candidate. A third prediction block can be generated based on a weighted sum operation of the first prediction block and the second prediction block.

[0582] The information for specifying the first merge candidate and the information for specifying the second merge candidate can be signaled through the bitstream, respectively. For example, index information merge_idx for specifying the first merge candidate and index information merge_2nd_idx for specifying the second merge candidate can be signaled through the bitstream. The second merge candidate can be determined based on the index information merge_2nd_idx and the index information merge_idx.

[0583] The index information merge_idx specifies any one of the merge candidates included in the merge candidate list.

[0584] The index information merge_2nd_idx can specify any one of the remaining merge candidates other than the merge candidate specified by merge_idx. Thus, when the value of merge_2nd_idx is smaller than the value of merge_idx, the merge candidate indexed by the value of merge_2nd_idx can be set as the second merge candidate. When the value of merge_2nd_idx is equal to or greater than the value of merge_idx, the merge candidate indexed by the value of merge_2nd_idx plus 1 can be set as the second merge candidate.

[0585] Alternatively, the second merge candidate can be specified in consideration of the search order of the candidate blocks.

[0586] Figure 37 is an example illustrating the specification of the second merge candidate in consideration of the search order of the candidate blocks.

[0587] In Figure 37 In the example shown in FIG. 14, the indices marked on the neighboring samples and the non-neighboring samples indicate the search order of the candidate blocks. For example, the candidate blocks can be searched in order from the A0 position to the A14 position.

[0588] When the block A4 is selected as the first merge candidate, the merge candidate derived from the candidate block following A4 in the search order can be specified as the second merge candidate. For example, the merge candidate derived from A5 can be selected as the second merge candidate. When the candidate block at the position A5 cannot be used as a merge candidate, the merge candidate derived from the next candidate block can be selected as the second merge candidate.

[0589] The first merge candidate and the second merge candidate can also be selected among the merge candidates derived from the non-neighboring blocks.

[0590] Figure 38 is an example illustrating the selection of the first merge candidate and the second merge candidate among the merge candidates derived from the non-neighboring blocks.

[0591] As Figure 38In the illustrated example, a merge candidate derived from a first candidate block and a second candidate block that are not adjacent to the current block can be selected as a first merge candidate and a second merge candidate, respectively. At this time, the block line to which the first candidate block belongs and the block line to which the second candidate block belongs can be different. For example, the first merge candidate can be derived from any one of the candidate blocks from A5 to A10, and the second merge candidate can be derived from any one of the candidate blocks from A11 to A15.

[0592] Alternatively, it can be set so that the first candidate block and the second candidate block are not included in the same line (e.g., row or column).

[0593] As another example, the second merge candidate can be specified based on the first merge candidate. At this time, the first merge candidate can be specified by index information merge_idx signaled from the bitstream. For example, a merge candidate adjacent to the first merge candidate can be designated as the second merge candidate. Here, the merge candidate adjacent to the first merge candidate can mean a merge candidate having an index difference of 1 from the first merge candidate. For example, a merge candidate having an index value of merge_idx+1 can be set as the second merge candidate. At this time, when the value of merge_idx+1 is greater than the maximum index value (or when the index value of the first merge candidate is the maximum index), a merge candidate having an index value of merge_idx-1 or a merge candidate having an index value of a pre-defined value (e.g., 0) can be set as the second merge candidate.

[0594] Alternatively, the merge candidate adjacent to the first merge candidate can mean a merge candidate derived from a candidate block that is spatially adjacent to a candidate block used to derive the first merge candidate. Here, the adjacent candidate block of the candidate block can mean a block adjacent to the left, right, top, bottom, or in a diagonal direction of the candidate block.

[0595] As another example, the second merge candidate can be specified based on the motion information of the first merge candidate. For example, a merge candidate having the same reference picture as the first merge candidate can be selected as the second merge candidate. When there are a plurality of merge candidates having the same reference picture as the first merge candidate, a merge candidate having the smallest index among the plurality of merge candidates, or a merge candidate having the smallest index difference from the first merge candidate can be selected as the second merge candidate. Alternatively, the second merge candidate can be selected based on index information specifying any one of the plurality of merge candidates.

[0596] Alternatively, when the first merge candidate is a uni-prediction in the first direction, a merge candidate including motion information in the second direction can be set as the second merge candidate. For example, when the first merge candidate has motion information in the direction L0, a merge candidate having motion information in the direction L1 can be set as the second merge candidate. When there are a plurality of merge candidates having motion information in the direction L1, a merge candidate having the smallest index among the plurality of merge candidates or a merge candidate having the smallest difference in index from the first merge candidate can be set as the second merge candidate. Alternatively, the second merge candidate can be selected based on index information specifying any one of the plurality of merge candidates.

[0597] As another example, one of the merge candidates derived from neighboring blocks adjacent to the current block can be set as the first merge candidate, and one of the merge candidates derived from non-neighboring blocks not adjacent to the current block can be set as the second merge candidate.

[0598] As another example, one of the merge candidates derived from candidate blocks located above the current block can be set as the first merge candidate, and one of the merge candidates derived from candidate blocks located to the left can be set as the second merge candidate.

[0599] The combined prediction block can be obtained through a weighted sum operation of the first prediction block derived from the first merge candidate and the second prediction block derived based on the second merge candidate. At this time, the weight value applied to the first prediction block can be set to a larger value than the weight value applied to the second prediction block.

[0600] Alternatively, the weight values can be determined based on the motion information of the first merge candidate and the motion information of the second merge candidate. For example, the weight values applied to the first prediction block and the second prediction block can be determined based on a difference in output order of the reference image and the current image. Specifically, the larger the difference in output order of the reference image and the current image, the smaller the weight value applied to the prediction block can be set.

[0601] Alternatively, the weight values applied to the first prediction block and the second prediction block can be determined in consideration of the size or shape of a candidate block used to derive the first merge candidate (hereinafter referred to as a first candidate block) and a candidate block used to derive the second merge candidate (hereinafter referred to as a second candidate block). For example, the weight value applied to a prediction block derived from a candidate block having a shape similar to that of the current block among the first candidate block or the second candidate block can be set to a larger value. On the other hand, the weight value applied to a prediction block derived from a candidate block having a shape not similar to that of the current block among the first candidate block or the second candidate block can be set to a smaller value.

[0602] Figure 39 is a diagram illustrating an example of determining a weight value applied to a prediction block based on the shape of a candidate block.

[0603] Assume the current block is a non-square with a width greater than its height.

[0604] A first prediction block and a second prediction block can be derived based on a first merge candidate and a second merge candidate, and a combined prediction block can be generated based on a weighted sum of the first prediction block and the second prediction block. At this point, the weighting values ​​applied to the first prediction block and the second prediction block can be determined based on the shapes of the first candidate block and the second candidate block.

[0605] For example, in Figure 39 In the example shown, the first candidate block is a square, and the second candidate block is a non-square with a width greater than its height. Since the second candidate block has the same shape as the current block, the weighting applied to the second prediction block can be set to be greater than the weighting applied to the first prediction block. For example, 5 / 8 of the weighting can be applied to the second prediction block, and 3 / 8 of the weighting can be applied to the first prediction block. Equation 33 shows an example of deriving a combined prediction block based on the weighted sum of the first and second prediction blocks.

[0606] [Formula 33]

[0607] P(x, y)=(3*P1(x, y)+5*P2(x, y))>>3

[0608] P(x,y) represents the combined prediction block, P1(x,y) represents the first prediction block, and P2(x,y) represents the second prediction block.

[0609] As another example, the weighting values ​​applied to the first and second prediction blocks can be determined based on the shape of the current block. For instance, when the current block is a non-square with a width greater than its height, a larger weighting value can be applied to the prediction block generated based on the merge candidate derived from the candidate block above the current block in the first and second merge candidates. When both the first and second merge candidates are derived from the candidate block above the current block, the weighting values ​​applied to the first and second prediction blocks can be set to the same. On the other hand, when the current block is a non-square with a height greater than its width, a larger weighting value can be applied to the prediction block generated based on the merge candidate derived from the candidate block to the left of the current block in the first and second merge candidates. When both the first and second merge candidates are derived from the candidate block to the left of the current block, the weighting values ​​applied to the first and second prediction blocks can be set to the same. When the current block is a square, the weighting values ​​applied to the first and second prediction blocks can be set to the same.

[0610] As another example, the weighting value applied to each prediction block can be determined based on a distance between the current block and the candidate block. Here, the distance can be derived based on a minimum value of a difference value of an x-axis coordinate, a difference value of a y-axis coordinate, or both, from the current block. The weighting value applied to a prediction block derived from a merge candidate having a smaller distance from the current block can be set to have a larger value than the weighting value applied to a prediction block derived from a merge candidate having a larger distance from the current block. For example, in the example shown in FIG. 6, since the x-axis distance between the first candidate block and the current block is smaller than the x-axis distance between the second candidate block and the current block, the weighting value applied to the first prediction block can be set to have a larger value than the weighting value applied to the second prediction block. Figure 37 In this case, since the x-axis distance between the first candidate block and the current block is smaller than the x-axis distance between the second candidate block and the current block, the weighting value applied to the first prediction block can be set to have a larger value than the weighting value applied to the second prediction block.

[0611] Alternatively, when the first merge candidate and the second merge candidate are both derived from non-adjacent blocks, a larger weighting value can be assigned to a prediction block derived from a non-adjacent block having a closer distance from the current block among the non-adjacent blocks. For example, in the example shown in FIG. 7, since the y-axis distance between the first candidate block and the current block is smaller than the y-axis distance between the second candidate block and the current block, the weighting value applied to the first prediction block can be set to have a larger value than the weighting value applied to the second prediction block. Figure 38 In this case, since the y-axis distance between the first candidate block and the current block is smaller than the y-axis distance between the second candidate block and the current block, the weighting value applied to the first prediction block can be set to have a larger value than the weighting value applied to the second prediction block.

[0612] In the combined prediction mode combining the merge mode and the merge mode described above, the merge mode can mean a merge mode based on a translational motion model (hereinafter referred to as a translational merge mode) or a merge mode based on an affine motion model (hereinafter referred to as an affine merge mode). That is, the motion-compensated prediction can be performed by combining the translational merge mode and the translational merge mode, or by combining the affine merge mode and the affine merge mode.

[0613] For example, when the first merge candidate is an affine merge candidate, the second merge candidate can also be set to an affine merge candidate. Here, the affine merge candidate indicates a case where the motion vector of the block including the reference candidate is an affine motion vector. The second merge candidate can be designated by the above-described various embodiments. For example, the second merge candidate can be set to a neighboring merge candidate of the first merge candidate. At this time, when the merge candidate neighboring the first merge candidate is not encoded using the affine motion model, a merge candidate encoded using the affine motion model can be set as the second merge candidate instead of the neighboring merge candidate.

[0614] On the other hand, when the first merge candidate is a non-affine merge candidate, the second merge candidate can also be set as a non-affine merge candidate. At this time, when a merge candidate adjacent to the first merge candidate is coded using an affine motion model, a merge candidate coded using a translational motion model instead of the adjacent merge candidate can be set as the second merge candidate.

[0615] Figure 40 FIG. 4 is a diagram illustrating an example of setting a non-affine merge candidate instead of an affine merge candidate as a second merge candidate.

[0616] When a merge candidate at position A1 is designated as a first merge candidate by merge_idx, a merge candidate A2 having an index value which is 1 greater than the first merge candidate can be selected as a second merge candidate. At this time, in the case where the first merge candidate is a non-affine merge candidate and the second merge candidate is an affine merge candidate, the second merge candidate can be re-set. For example, among merge candidates having indices greater than merge_idx+1, a non-affine merge candidate having the smallest difference from merge_idx+1 can be re-set as the second merge candidate. For example, in the example shown in FIG. 4, the merge candidate A3 having an index of merge_idx+2 is set as the second merge candidate. Figure 18

[0617] As another example, motion-compensated prediction can be performed by combining a translational merge mode with an affine merge mode. That is, either the first merge candidate or the second merge candidate can be an affine merge candidate, and the other can be a non-affine merge candidate.

[0618] Integrated motion information can be derived based on the first merge candidate and the second merge candidate, and motion-compensated prediction can be performed on the current block based on the integrated motion information. For example, a motion vector of the current block can be derived based on an average operation or a weighted sum operation of a motion vector of the first merge candidate and a motion vector of the second merge candidate. At this time, a weighting value applied to the motion vector of the first merge candidate and a weighting value applied to the motion vector of the second merge candidate can be determined through the above-described embodiments.

[0619] In the case where the first merge candidate is a non-affine merge candidate and the second affine merge candidate is an affine merge candidate, a motion vector of the current block can be derived by scaling the motion vector of the second merge candidate. Equation 34 illustrates an example of deriving a motion vector of the current block.

[0620]

Equation 34

[0621] (mvX, mvY) = (mvOx, mvOy) + ((mvIx, mvIy) » M)

[0622] ​In Equation 34, (mvX, mvY) represents the motion vector of the current block, (mv0x, mv0y) represents the motion vector of the first merge candidate, (mv1x, mv1y) represents the motion vector of the second merge candidate. M represents a scaling parameter. M can be predefined in the encoder and the decoder. Alternatively, the value of the scaling parameter M can be determined according to the size of the current block or the candidate block. For example, M can be set to 3 when the width or height of the second candidate block is greater than 32, otherwise M can be set to 2.

[0623] In the prediction mode combining the merge mode and the advanced motion vector prediction mode, the first prediction block can be generated using the motion information derived from the merge candidate, and the second prediction block can be generated using the motion vector derived from the motion vector prediction candidate.

[0624] In the advanced motion vector prediction mode, the motion vector prediction candidate can be derived from a neighboring block adjacent to the current block or a collocated block within a collocated picture. Then, any one of the multiple motion vector prediction candidates can be specified, and the specified motion vector prediction candidate can be set as the motion vector prediction result of the current block. Then, the motion vector of the current block can be derived by adding the motion vector prediction result of the current block and the motion vector difference.

[0625] In the prediction mode combining the merge mode and the advanced motion vector prediction mode, the merge candidate and the motion vector prediction candidate can be derived from the same candidate block. For example, if the merge candidate is specified by merge_idx, the motion vector of the candidate block used to derive the specified merge candidate can be set as the motion vector prediction result. Alternatively, if the motion vector prediction candidate is specified by mvp_flag, the merge candidate derived from the candidate block used to derive the specified merge candidate can be selected.

[0626] Alternatively, the candidate block used to derive the merge candidate and the candidate block used to derive the motion vector prediction candidate can be different. For example, if the merge candidate derived from the candidate block located above the current block is selected, the motion vector prediction candidate derived from the candidate block located left to the current block can be set to be selected.

[0627] Alternatively, when the merge candidate selected by the index information and the motion vector prediction candidate selected by the index information are derived from the same candidate block, the motion vector prediction candidate can be replaced by a motion vector prediction candidate derived from a neighboring candidate block adjacent to the candidate block, or the merge candidate can be replaced by a merge candidate derived from a candidate block adjacent to the candidate block.

[0628] Figure 41 FIG. 1 is a diagram illustrating an example of replacing a merge candidate.

[0629] In Figure 41 the example shown in the left side of FIG. 1, it is shown that a merge candidate and a motion vector prediction candidate are selected from a candidate block at position A2. As shown, when the merge candidate and the motion vector prediction candidate are derived from the same candidate block, a merge candidate or a motion vector prediction candidate derived from a candidate block adjacent to the candidate block can be used instead of the merge candidate or the motion vector prediction candidate. For example, as shown in the right side of FIG. 1, a merge candidate at position A1 can be used instead of a merge candidate at position A2. Figure 41

[0630] A first prediction block can be derived based on a merge candidate of a current block, and a second prediction block can be derived based on the motion vector prediction candidate. Then, a combined prediction block can be derived through a weighted sum operation of the first prediction block and the second prediction block. At this time, a weight value applied to the second prediction block generated using the advanced motion vector prediction mode can be set to a value greater than a weight value applied to the first prediction block generated using the merge mode.

[0631] A derived residual image can be derived by subtracting a prediction image from an original image. In this case, when the residual image is changed to a frequency domain, even if a high frequency component among frequency components is removed, the subjective quality of a video is not greatly reduced. Thus, if a value of the high frequency component is made smaller or set to 0, there is an effect of improving compression efficiency without causing a significant visual distortion. Reflecting the above characteristics, a transform can be performed on a current block to decompose a residual image into 2-dimensional frequency components. The transform can be performed using a transform technique such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST).

[0632] The DCT decomposes (or transforms) a residual image into 2-dimensional frequency components using a cosine transform, and the DST decomposes (or transforms) a residual image into 2-dimensional frequency components using a sine transform. As a result of the transform of the residual image, the frequency components can be expressed as basis pattern components. For example, when a DCT transform is performed on a block having a size of N x N, N 2 basis pattern components can be obtained. The size of each basis pattern component included in a block having a size of N x N can be obtained through the transform. According to the transform technique used, the size of the basis pattern component can be referred to as a DCT coefficient or a DST coefficient.

[0633] The transform technique DCT is mainly used for transforming an image in which a low frequency component is distributed more than 0. The transform technique DST is mainly used for an image in which a high frequency component is distributed more.

[0634] ​Transformation techniques other than DCT or DST can also be used to transform residual images.

[0635] The process of transforming the residual image into two-dimensional frequency components is referred to as a two-dimensional image transformation. Furthermore, the magnitude of the fundamental pattern components obtained from the transformation is called the transformation coefficients. For example, transformation coefficients can refer to DCT coefficients or DST coefficients. When the first and second transformations, which will be described later, are applied simultaneously, the transformation coefficients can represent the magnitude of the fundamental pattern components generated by the result of the second transformation.

[0636] Transform techniques can be determined on a block-by-block basis. A transform technique can be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, when the current block is coded in intra-predictive mode and the size of the current block is less than N×N, the transform technique DST can be used to perform the transform. On the other hand, when the aforementioned conditions cannot be met, the transform technique DCT can be used to perform the transform.

[0637] In the residual image, a portion of the block may not undergo 2D image transformation. This omission of 2D image transformation is called transform skipping. When transform skipping is applied, quantization can be applied to the residual values ​​for which no transformation was performed.

[0638] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the DCT- or DST-based transformation can be defined as the first transformation, and the process of transforming the block again using the first transformation can be called the second transformation.

[0639] The first transform can be performed using any of a number of transform kernel candidates. For example, the first transform can be performed using any of DCT2, DCT8, or DCT7.

[0640] Different transform cores can be used for the horizontal and vertical directions. Information representing combinations of horizontal and vertical transform cores can also be transmitted as signals via bitstreams.

[0641] The execution units for the first and second transformations will be different. For example, the first transformation can be performed on an 8×8 block, and the second transformation can be performed on the 4×4 sub-blocks within the transformed 8×8 block. In this case, the transformation coefficients of the remaining regions where the second transformation is not performed can also be set to 0.

[0642] Alternatively, a first transformation can be performed on a 4×4 block, and a second transformation can be performed on an 8×8 region of the 4×4 block that includes the transformation.

[0643] Information indicating whether to perform the second transformation can be sent via a bitstream signal.

[0644] The decoder can perform the inverse of the second transform (second inverse transform), and the result of the second transform can be subjected to the inverse of the first transform (first inverse transform). The residual signal of the current block can be obtained as the result of the execution of the second inverse transform and the first inverse transform.

[0645] Quantization is used to reduce the energy of the block, and the quantization process involves dividing the transformation coefficients by a specific constant. This constant can be derived from quantization parameters, which can be defined as values ​​between 1 and 63.

[0646] If transform and quantization are performed in the encoder, the decoder can obtain the residual block through inverse quantization and inverse transform. The decoder then adds the predicted block and the residual block together to obtain the reconstructed block of the current block.

[0647] If the reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss during quantization and encoding. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter will be referred to as the first reconstructed block, and the reconstructed block after applying the in-loop filter will be referred to as the second reconstructed block.

[0648] A second reconstructed block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block. In this case, SAO or ALF can be applied after the deblocking filter.

[0649] Deblocking filters are used to mitigate the image quality degradation (blocking artifact) that occurs at block boundaries when quantization is performed on a block-by-block basis. To apply a deblocking filter, the block strength (BS) between the first reconstructed block and its adjacent reconstructed blocks can be determined.

[0650] Figure 42 This is a flowchart illustrating the process of determining block strength.

[0651] exist Figure 30 In the example shown, P represents the first reconstructed block, and Q represents the adjacent reconstructed block. The adjacent reconstructed block can be adjacent to the left or top of the current block.

[0652] exist Figure 42In the illustrated example, the block strength is determined considering the prediction coding mode of P and Q, whether or not a non-zero transform coefficient is included, whether or not the same reference picture is used to perform inter prediction, and whether or not the difference value of the motion vector is greater than or equal to a threshold value.

[0653] Based on the block strength, it can be determined whether or not the deblocking filter is applied. For example, when the block strength is 0, it is possible not to perform filtering.

[0654] SAO is used to alleviate a ringing artifact generated when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset value determined considering a pattern of the first reconstructed image. The determination method of the offset value includes edge offset (EO) or band offset (BO). The EO indicates a method of determining an offset of a current sample according to a pattern of a peripheral pixel. The BO indicates a method of applying a common offset to a set of pixels having similar luminance values within a region. Specifically, the pixel luminance is divided into 32 equal intervals, and pixels having similar luminance values are set as one set. For example, 4 adjacent bands among 32 bands are set as a group, and samples belonging to the 4 bands can apply the same offset value.

[0655] ALF is a method of generating a second reconstructed image by applying a filter of a predefined size / shape to the first reconstructed image or the reconstructed image to which the deblocking filter is applied. Equation 35 below indicates an applicable case of the ALF.

[0656] [Equation 35]

[0657]

[0658] Any one of the predefined filter candidates can be selected in units of an image, a coding tree unit, a coding block, a prediction block, or a transform block. Any one of the sizes or shapes of the respective filter candidates can be different.

[0659] Figure 43 is a diagram illustrating the predefined filter candidates.

[0660] As illustrated in Figure 43 the example, at least one of a 5x5, 7x7, or 9x9 diamond can be selected.

[0661] Only a diamond of a size of 5x5 can be used for a chroma component.

[0662] Embodiments described centering on a decoding process or an encoding process are also included in the scope of the present application. Embodiments described in a predetermined order can also be changed in an order different from that described, and are also included in the scope of the present application.

[0663] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the time sequence of the invention, and can be performed simultaneously or in another order as needed. Also, in the above-described embodiments, the structural elements (for example, units, modules, etc.) constituting the block diagrams can also be respectively implemented as hardware devices or software, and a plurality of structural elements can be implemented in combination as a single hardware device or software. The embodiments can be implemented in the form of program instructions executable by various computer components and recorded in computer-readable recording media. The computer-readable recording media can individually or in combination include program instructions, data files, data structures, etc. Examples of the computer-readable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, and the like, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROMs, RAMs, flash memories, and the like. The hardware devices can be configured to operate as one or more software modules to perform processes according to the present invention, and vice versa.

[0664]

INDUSTRIAL APPLICABILITY

[0665] The present invention can be applied to an electronic device that encodes / decodes a video.

Claims

1. A video decoding method, characterized in that, The method includes: Generate a list of merge candidates for the current block; Select multiple merge candidates from the merge candidate list; Motion compensation is performed on the current block based on the motion information of multiple selected merging candidates to obtain the target prediction block of the current block; wherein, the multiple merging candidates include a first merging candidate and a second merging candidate; The step of performing motion compensation on the current block based on motion information of multiple selected merging candidates to obtain the target prediction block of the current block includes: The first merging candidate is used to generate a first prediction block of the current block, the second merging candidate is used to generate a second prediction block of the current block, and a target prediction block of the current block is generated based on the first prediction block and the second prediction block; The method further includes: obtaining the index information of the first merging candidate and the index information of the second merging candidate by parsing the bit stream; wherein, when the value of the index information of the second merging candidate is equal to or greater than the value of the index information of the first merging candidate, the value of the index information of the second merging candidate is set to the result of adding 1 to the value of the index information of the second merging candidate; The method further includes adding inter-frame merging candidates included in the inter-frame motion information list to the merging candidate list based on the number of spatial merging candidates and temporal merging candidates included in the merging candidate list.

2. The video decoding method according to claim 1, characterized in that, The inter-frame motion information list includes inter-frame merging candidates derived based on motion information of blocks decoded before the current block.

3. The video decoding method according to claim 1, characterized in that, The method further includes: When the current block is included in the merge processing area, temporary merge candidates derived based on the motion information of the current block are added to the temporary motion information list. Once decoding of all blocks included in the merge processing area is complete, the temporary merge candidate is updated in the inter-frame motion information list.

4. The video decoding method according to claim 1, characterized in that, The method further includes: Based on the determination of whether an inter-frame merging candidate included in the inter-frame motion information list is the same as at least one merging candidate included in the merging candidate list, it is determined whether to add the inter-frame merging candidate to the merging candidate list.

5. The video decoding method according to claim 4, characterized in that, The method further includes: If at least one merge candidate with an index value less than or equal to the threshold is different from the inter-frame merge candidate, then it is determined that the inter-frame merge candidate included in the inter-frame motion information list is different from at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is added to the merge candidate list. If at least one of the merge candidates with an index value less than or equal to the threshold contains a merge candidate that is the same as the inter-frame merge candidate, then it is determined that an inter-frame merge candidate included in the inter-frame motion information list is the same as at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is not added to the merge candidate list.

6. The video decoding method according to claim 4, characterized in that, The method further includes: If it is determined that there is a merge candidate in the merge candidate list that is the same as the inter-frame merge candidate, then the inter-frame merge candidate will not be added to the merge candidate list. Based on the determination of whether another inter-frame merging candidate included in the inter-frame motion information list is the same as at least one merging candidate included in the merging candidate list, it is determined whether to add the other inter-frame merging candidate to the merging candidate list. The determination of whether the other inter-frame merge candidate is the same as the merge candidate that is the same as the first inter-frame merge candidate is not performed.

7. The video decoding method according to claim 1, characterized in that, The method further includes: When there is an inter-frame merge candidate in the merge candidate list that has the same motion information as the current block, the index assigned to the inter-frame merge candidate in the inter-frame motion information list will be updated to the maximum value.

8. The video decoding method according to claim 4, wherein, The method further includes: If at least one merge candidate with an index value greater than the threshold is different from the inter-frame merge candidate, then it is determined that the inter-frame merge candidate included in the inter-frame motion information list is different from at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is added to the merge candidate list. If at least one of the merge candidates with an index value greater than the threshold contains a merge candidate that is the same as the inter-frame merge candidate, then it is determined that an inter-frame merge candidate included in the inter-frame motion information list is the same as at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is not added to the merge candidate list.

9. The video decoding method according to claim 4, characterized in that, The method further includes performing the determination by comparing a merge candidate derived from a block at a specific location with the inter-frame merge candidate, wherein the specific location includes at least one of the upper-right adjacent block or the lower-left adjacent block of the current block.

10. The video decoding method according to claim 1, characterized in that, The step of generating the target prediction block for the current block based on the first prediction block and the second prediction block includes: The target prediction block of the current block is generated based on the weighted sum of the first prediction block and the second prediction block.

11. A video encoding method, characterized in that, The method includes: Generate a list of merge candidates for the current block; Select multiple merge candidates from the merge candidate list; Motion compensation is performed on the current block based on the motion information of multiple selected merging candidates to obtain the target prediction block of the current block; wherein, the multiple merging candidates include a first merging candidate and a second merging candidate; The step of performing motion compensation on the current block based on motion information of multiple selected merging candidates to obtain the target prediction block of the current block includes: The first merging candidate is used to generate a first prediction block of the current block, the second merging candidate is used to generate a second prediction block of the current block, and a target prediction block of the current block is generated based on the first prediction block and the second prediction block; The method further includes: transmitting index information for specifying the first merging candidate and index information for the second merging candidate via a bit stream; wherein, when the value of the index information of the second merging candidate is equal to or greater than the value of the index information of the first merging candidate, the value of the index information of the second merging candidate is set to the result of adding 1 to the value of the index information of the second merging candidate; The method further includes adding inter-frame merging candidates included in the inter-frame motion information list to the merging candidate list based on the number of spatial merging candidates and temporal merging candidates included in the merging candidate list.

12. The video encoding method according to claim 11, characterized in that, The inter-frame motion information list includes inter-frame merging candidates derived based on motion information of blocks encoded prior to the current block.

13. The video encoding method according to claim 11, characterized in that, The method further includes: When the current block is included in the merge processing area, temporary merge candidates derived based on the motion information of the current block are added to the temporary motion information list. Once the encoding of all blocks included in the merge processing area is completed, the temporary merge candidate is updated in the inter-frame motion information list.

14. The video encoding method according to claim 11, characterized in that, The method further includes: Based on the determination of whether an inter-frame merging candidate included in the inter-frame motion information list is the same as at least one merging candidate included in the merging candidate list, it is determined whether to add the inter-frame merging candidate to the merging candidate list.

15. The video encoding method according to claim 14, characterized in that, The method further includes: If at least one merge candidate with an index value less than or equal to the threshold is different from the inter-frame merge candidate, then it is determined that the inter-frame merge candidate included in the inter-frame motion information list is different from at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is added to the merge candidate list. If at least one of the merge candidates with an index value less than or equal to the threshold contains a merge candidate that is the same as the inter-frame merge candidate, then it is determined that an inter-frame merge candidate included in the inter-frame motion information list is the same as at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is not added to the merge candidate list.

16. The video encoding method according to claim 14, characterized in that, The method further includes: If it is determined that there is a merge candidate in the merge candidate list that is the same as the inter-frame merge candidate, then the inter-frame merge candidate will not be added to the merge candidate list. Based on the determination of whether another inter-frame merging candidate included in the inter-frame motion information list is the same as at least one merging candidate included in the merging candidate list, it is determined whether to add the other inter-frame merging candidate to the merging candidate list. The determination of whether the other inter-frame merge candidate is the same as the merge candidate that is the same as the first inter-frame merge candidate is not performed.

17. The video encoding method according to claim 11, characterized in that, The method further includes: When there is an inter-frame merge candidate in the merge candidate list that has the same motion information as the current block, the index assigned to the inter-frame merge candidate in the inter-frame motion information list will be updated to the maximum value.

18. The video encoding method according to claim 14, wherein, The method further includes: If at least one merge candidate with an index value greater than the threshold is different from the inter-frame merge candidate, then it is determined that the inter-frame merge candidate included in the inter-frame motion information list is different from at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is added to the merge candidate list. If at least one of the merge candidates with an index value greater than the threshold contains a merge candidate that is the same as the inter-frame merge candidate, then it is determined that an inter-frame merge candidate included in the inter-frame motion information list is the same as at least one merge candidate included in the merge candidate list, and the inter-frame merge candidate is not added to the merge candidate list.

19. The video encoding method according to claim 14, characterized in that, The method further includes performing the determination by comparing a merge candidate derived from a block at a specific location with the inter-frame merge candidate, wherein the specific location includes at least one of the upper-right adjacent block or the lower-left adjacent block of the current block.

20. The video encoding method according to claim 11, characterized in that, The step of generating the target prediction block for the current block based on the first prediction block and the second prediction block includes: The target prediction block of the current block is generated based on the weighted sum of the first prediction block and the second prediction block.

21. A video decoding apparatus, comprising: Memory and processor; characterized in that, The memory is used to store computer programs that can run on the processor; The processor is configured to perform the steps of the method according to any one of claims 1 to 10 when running the computer program.

22. A video encoding apparatus, comprising: Memory and processor; characterized in that, The memory is used to store computer programs that can run on the processor; The processor is configured to perform the steps of the method according to any one of claims 11 to 20 when running the computer program.

23. A method for transmitting a bit stream, characterized in that, The video encoding method according to any one of claims 11 to 20 is used to generate a bitstream; and the bitstream is transmitted.

Images