Image signal encoding / decoding method and apparatus therefor

Abstract

An image decoding method according to the present invention may comprise the steps of: determining whether a combined prediction mode is applied to a current block; when the combined prediction mode is applied to the current block, obtaining first and second prediction blocks with respect to the current block; and, on the basis of a calculation of a weighted sum of the first and second prediction blocks, obtaining a third prediction block with respect to the current block.

Description

Video signal encoding / decoding method and device therefor

[0001] The present invention relates to a method for encoding / decoding a video signal and a device therefor.

[0002] As display panels become increasingly larger, the demand for higher-quality video services is increasing. The biggest problem with high-quality video services is the significant increase in data volume, and to solve this problem, active research is being conducted to improve video compression ratios. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) formed the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed HEVC (High Efficiency Video Coding), a video compression standard with approximately twice the compression performance of H.264 / AVC, and it was approved as a standard on January 25, 2013. With the rapid development of high-quality video services, the performance of HEVC is gradually revealing its limitations.

[0003] The purpose of the present invention is to provide a combined prediction method that combines a plurality of prediction methods in encoding / decoding a video signal, and a device for performing the method.

[0004] The present invention aims to provide a method for dividing a coding block into a plurality of prediction units in encoding / decoding a video signal and a device for performing the method.

[0005] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0006] A video signal decoding / encoding method according to the present invention includes a step of determining whether to apply a combined prediction mode to a current block, a step of obtaining a first prediction block and a second prediction block for the current block when the combined prediction mode is applied to the current block, and a step of obtaining a third prediction block for the current block based on a weighted sum operation of the first prediction block and the second prediction block. At this time, the first prediction block may be obtained based on motion information of a merge candidate of the current block, and the second prediction block may be obtained based on an intra prediction mode of the current block.

[0007] In a video signal decoding / encoding method according to the present invention, when the combined prediction mode is applied to the current block, it may be set so that application of triangular partitioning to the current block is not permitted.

[0008] In the video signal decoding / encoding method according to the present invention, the intra prediction mode of the current block can be set to planar.

[0009] In the video signal decoding / encoding method according to the present invention, the second prediction block can be obtained based on a reference sample line included in an adjacent reference sample line.

[0010] In the video signal decoding / encoding method according to the present invention, when performing the weighted sum operation, the weights applied to the first prediction block and the second prediction block can be determined based on the prediction encoding modes of neighboring blocks adjacent to the current block.

[0011] In a video signal decoding / encoding method according to the present invention, if at least one of the width or height of the current block is greater than a threshold value, the combined prediction mode may not be applied to the current block.

[0012] In a video signal decoding / encoding method according to the present invention, when a flag indicating that the merge mode is applied to the current block is true, the combined prediction mode can be set to be applicable to the current block.

[0013] The features briefly summarized above of the present invention are merely exemplary aspects of the detailed description of the present invention described below and do not limit the scope of the present invention.

[0014] According to the present invention, inter prediction efficiency can be improved by providing a combined prediction method that combines a plurality of prediction methods.

[0015] According to the present invention, inter prediction efficiency can be improved by proposing a method of dividing a coding block into a plurality of prediction blocks and deriving motion information for each of the prediction blocks.

[0016] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0017] FIG. 1 is a block diagram of a video encoder according to one embodiment of the present invention.

[0018] FIG. 2 is a block diagram of an image decoder (decoder) according to one embodiment of the present invention.

[0019] FIG. 3 is a diagram illustrating a basic coding tree unit according to one embodiment of the present invention.

[0020] Figure 4 is a drawing showing various division forms of a coding block.

[0021] Figure 5 is a diagram illustrating the division aspect of a coding tree unit.

[0022] Figure 6 is a drawing showing the form of a basic data unit.

[0023] Figures 7 and 8 are diagrams showing examples in which a coding block is divided into multiple sub-blocks.

[0024] Figure 9 is a flowchart of an inter prediction method according to one embodiment of the present invention.

[0025] Figure 10 is a flowchart of the process of deriving motion information of the current block under merge mode.

[0026] Figure 11 is a diagram illustrating candidate blocks used to derive merge candidates.

[0027] Figure 12 is a diagram showing an example of dividing a coding block into multiple prediction units using a diagonal.

[0028] Figure 13 is a diagram illustrating an example of dividing a coding block into two prediction units.

[0029] Figure 14 shows examples of dividing a coding block into multiple prediction blocks of different sizes.

[0030] Figure 15 is a diagram showing an example in which diagonal partitioning is applied to a conversion unit.

[0031] Figure 16 is a diagram showing neighboring blocks used to derive triangle merge candidates.

[0032] Figure 17 is a diagram illustrating an example of determining the availability of neighboring blocks for each triangle prediction unit.

[0033] Figures 18 and 19 are diagrams showing examples of deriving prediction samples based on a weighted sum operation of a first prediction sample and a second prediction sample.

[0034] FIG. 20 is a flowchart of an intra prediction method according to one embodiment of the present invention.

[0035] Figure 21 is a diagram showing intra prediction modes.

[0036] Figures 22 and 23 are diagrams showing examples of one-dimensional arrays that arrange reference samples in a row.

[0037] Figure 24 is a diagram illustrating the angles formed by directional intra prediction modes with a straight line parallel to the x-axis.

[0038] Figure 25 is a diagram showing how prediction samples are obtained when the current block is a non-square shape.

[0039] Figure 26 is a diagram showing wide-angle intra prediction modes.

[0040] Figure 27 is a drawing showing the application aspect of PDPC.

[0041] Figure 28 is a diagram showing the application of weights.

[0042] Figure 29 illustrates an example in which different weights are assigned to sub-blocks depending on whether the merge candidate has bidirectional motion information.

[0043] Figure 30 shows an example in which different weights are assigned to sub-blocks depending on the intra prediction mode of the current block.

[0044] Figure 31 shows an example in which a second merge candidate is specified by considering the search order of candidate blocks.

[0045] Figure 32 shows an example in which a first merge candidate and a second merge candidate are selected from among merge candidates derived from non-neighboring blocks.

[0046] Figure 33 is a drawing for explaining an example of determining a second merge candidate.

[0047] Figure 34 is a diagram showing an example in which weights applied to prediction blocks are determined based on the shapes of candidate blocks.

[0048] Figure 35 is a diagram showing an example of a merge candidate being replaced.

[0049] Figures 36 and 37 are diagrams showing sub-blocks in which the second transformation is to be performed.

[0050] Figure 38 is a drawing for explaining an example in which the conversion type of the current block is determined.

[0051] Figure 39 is a flowchart showing the process of determining block strength.

[0052] Figure 40 shows predefined filter candidates.

[0053] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0054] Encoding and decoding of images are performed on a block-by-block basis. For example, encoding / decoding processes such as transformation, quantization, prediction, in-loop filtering, or restoration may be performed on coding blocks, transformation blocks, or prediction blocks.

[0055] Hereinafter, the block to be encoded / decoded will be referred to as the "current block." For example, the current block may represent a coding block, a transformation block, or a prediction block, depending on the current encoding / decoding processing step.

[0056] Additionally, the term "unit" used in this specification may be understood to represent a basic unit for performing a specific encoding / decoding process, and "block" may be understood to represent a sample array of a predetermined size. Unless otherwise specified, "block" and "unit" may be used with equivalent meanings. For example, in the embodiments described below, coding block and coding unit may be understood to have equivalent meanings.

[0057] FIG. 1 is a block diagram of a video encoder according to one embodiment of the present invention.

[0058] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0059] Each component shown in Fig. 1 is independently depicted to indicate different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0060] Additionally, some components may not be essential components that perform essential functions of the present invention, but may be optional components merely used to enhance performance. The present invention may be implemented by including only components essential to implementing the essence of the present invention, excluding components used solely for performance enhancement. A structure that includes only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present invention.

[0061] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitting unit (110) can split one picture into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the picture.

[0062] For example, a picture can be split into multiple coding units. A recursive tree structure, such as a quad tree structure, can be used to split a coding unit in a picture. A coding unit that is split into other coding units starting from an image or the largest coding unit as the root can be split into as many child nodes as the number of split coding units. A coding unit that cannot be split any further according to a certain restriction becomes a leaf node. That is, assuming that only a square split is possible for a coding unit, a coding unit can be split into at most four different coding units.

[0063] Hereinafter, in the embodiments of the present invention, the encoding unit may be used to mean a unit that performs encoding or may be used to mean a unit that performs decoding.

[0064] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit.

[0065] When generating a prediction unit that performs intra prediction based on a coding unit, if it is not the minimum coding unit, intra prediction can be performed without dividing it into multiple prediction units NxN.

[0066] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction and an intra prediction unit (125) that performs intra prediction. It may be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. At this time, the processing unit where prediction is performed and the processing unit where the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode may be determined in the prediction unit, and the prediction may be performed in the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoder. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0067] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0068] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0069] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods can be used to derive a motion vector, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods can be used as motion prediction methods, such as the Skip method, the Merge method, the AMVP (Advanced Motion Vector Prediction) method, and the Intra Block Copy method.

[0070] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0071] In intra prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and the intra prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized to predict chrominance information.

[0072] When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit can be performed based on the pixels located to the left of the prediction unit, the pixels located at the upper left, and the pixels located at the upper. However, when performing intra prediction, if the sizes of the prediction unit and the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division can be used only for the minimum coding unit.

[0073] The intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of prediction units existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

[0074] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0075] The transformation unit (130) can transform a residual block including residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) using a transformation method such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform). Here, the DCT transformation core includes at least one of DCT2 or DCT8, and the DST transformation core includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on intra prediction mode information of the prediction unit used to generate the residual block. Transformation for the residual block can also be skipped. A flag indicating whether to skip transformation for the residual block can be encoded. Transformation skipping can be allowed for a residual block whose size is less than or equal to a threshold, a luma component, or a chroma component under a 4:4:4 format.

[0076] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0077] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0078] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0079] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0080] The entropy encoding unit (165) can encode various information such as residual value coefficient information of an encoding unit, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information from the rearrangement unit (160) and the prediction unit (120, 125).

[0081] The entropy encoding unit (165) can entropy encode the coefficient values ​​of the encoding unit input from the rearrangement unit (160).

[0082] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0083] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0084] A deblocking filter can remove block distortion caused by boundaries between blocks in a reconstructed picture. To determine whether to perform deblocking, a deblocking filter can be applied to the current block based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be processed in parallel when performing vertical and horizontal filtering.

[0085] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for an image that has undergone deblocking. To perform offset correction for a specific picture, the pixels contained in the image can be divided into a certain number of regions, the regions to be offset can be determined, and the offset can be applied to those regions. Alternatively, the offset can be applied by considering the edge information of each pixel.

[0086] Adaptive Loop Filtering (ALF) can be performed based on the comparison of the filtered restored image with the original image. After dividing the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined, and filtering can be performed differentially for each group. Information regarding whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. Furthermore, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the target block.

[0087] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0088] FIG. 2 is a block diagram of an image decoder (decoder) according to one embodiment of the present invention.

[0089] Referring to FIG. 2, the image decoder (200) may include an entropy decoding unit (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0090] When a video bitstream is input to a video encoder, the input bitstream can be decoded in the opposite procedure to that of the video encoder.

[0091] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

[0092] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

[0093] The reordering unit (215) can perform reordering based on the method in which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed by the encoding unit and performing reverse scanning based on the scanning order performed by the corresponding encoding unit.

[0094] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0095] The inverse transform unit (225) can perform an inverse transform, i.e., an inverse DCT or inverse DST, on the transform unit, i.e., DCT or DST, for the quantization result performed by the image encoder. Here, the DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Alternatively, if the transform is skipped in the image encoder, the inverse transform unit (225) may not perform the inverse transform. The inverse transform may be performed based on the transmission unit determined by the image encoder. The inverse transform unit (225) of the image decoder may selectively perform a transform technique (e.g., DCT or DST) according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0096] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0097] As described above, when performing intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0098] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit (230) may perform inter prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction unit, using information necessary for inter prediction of the current prediction unit provided from the image encoder. Alternatively, inter prediction may be performed based on information of a pre-restored portion of the current picture including the current prediction unit.

[0099] In order to perform inter prediction, it is possible to determine whether the motion prediction method of the prediction unit included in the encoding unit is skip mode, merge mode, motion vector prediction mode (AMVP mode), or intra block copy mode based on the encoding unit.

[0100] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit (235) can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can determine and apply whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0101] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0102] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0103] Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from the video encoder, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0104] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and offset value information.

[0105] ALF can be applied to a coding unit based on information such as whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

[0106] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0107]

[0108] FIG. 3 is a diagram illustrating a basic coding tree unit according to one embodiment of the present invention.

[0109] The largest coding block can be defined as a coding tree block. A single picture is divided into multiple coding tree units (CTUs). A coding tree unit is the largest coding unit and may also be referred to as an LCU (Largest Coding Unit). Figure 3 illustrates an example of a single picture being divided into multiple coding tree units.

[0110] The size of a coding tree unit can be defined at the picture level or the sequence level. For this purpose, information indicating the size of the coding tree unit can be signaled through a picture parameter set or a sequence parameter set.

[0111] For example, the size of the coding tree unit for the entire picture in the sequence may be set to 128x128. Alternatively, the size of the coding tree unit may be determined to be either 128x128 or 256x256 at the picture level. For example, the size of the coding tree unit may be set to 128x128 in the first picture, and the size of the coding tree unit may be set to 256x256 in the second picture.

[0112] A coding tree unit can be divided to generate a coding block. A coding block represents a basic unit for encoding / decoding processing. For example, prediction or transformation may be performed for each coding block, or a prediction encoding mode may be determined for each coding block. Here, the prediction encoding mode represents a method for generating a prediction image. For example, the prediction encoding mode may include intra prediction, inter prediction, current picture referencing (CPR), or intra block copy (IBC), or combined prediction. For a coding block, a prediction block for the coding block can be generated using at least one prediction encoding mode among intra prediction, inter prediction, current picture referencing, or combined prediction.

[0113] Information indicating the prediction coding mode of the current block may be signaled via the bitstream. For example, the information may be a 1-bit flag indicating whether the prediction coding mode is intra mode or inter mode. Only when the prediction coding mode of the current block is determined to be inter mode, current picture reference or composite prediction may be available.

[0114] Current picture reference is to set the current picture as a reference picture and obtain the prediction block of the current block from an area within the current picture where encoding / decoding has already been completed. Here, the current picture means a picture including the current block. Information indicating whether the current picture reference is applied to the current block can be signaled through the bitstream. For example, the information can be a 1-bit flag. If the flag is true, the prediction encoding mode of the current block is determined as the current picture reference, and if the flag is false, the prediction mode of the current block can be determined as inter prediction.

[0115] Alternatively, the prediction encoding mode of the current block may be determined based on the reference picture index. For example, if the reference picture index points to the current picture, the prediction encoding mode of the current block may be determined as the current picture reference. If the reference picture index points to a picture other than the current picture, the prediction encoding mode of the current block may be determined as the inter prediction. In other words, the current picture reference is a prediction method using information of an area in the current picture in which encoding / decoding has been completed, and the inter prediction is a prediction method using information of another picture in which encoding / decoding has been completed.

[0116] Composite prediction refers to an encoding mode that combines two or more of intra prediction, inter prediction, and current picture reference. For example, when composite prediction is applied, a first prediction block may be generated based on one of intra prediction, inter prediction, or current picture reference, and a second prediction block may be generated based on the other. Once the first and second prediction blocks are generated, a final prediction block may be generated through an average operation or a weighted sum operation of the first and second prediction blocks. Information indicating whether composite prediction is applied may be signaled through the bitstream. The information may be a 1-bit flag.

[0117] Figure 4 is a drawing showing various division forms of a coding block.

[0118] A coding block can be divided into multiple coding blocks based on quad-tree partitioning, binary tree partitioning, or triple-tree partitioning. The divided coding block can also be further divided into multiple coding blocks based on quad-tree partitioning, binary tree partitioning, or triple-tree partitioning.

[0119] Quadtree partitioning is a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block can be partitioned into four square partitions (see 'SPLIT_QT' in (a) of Fig. 4).

[0120] Binary tree partitioning refers to a partitioning technique that partitions the current block into two blocks. Partitioning the current block into two blocks along the vertical direction (i.e., using a vertical line crossing the current block) can be called vertical binary tree partitioning, and partitioning the current block into two blocks along the horizontal direction (i.e., using a horizontal line crossing the current block) can be called horizontal binary tree partitioning. As a result of binary tree partitioning, the current block can be partitioned into two non-square partitions. Fig. 4 (b) 'SPLIT_BT_VER' represents the result of vertical binary tree partitioning, and Fig. 4 (c) 'SPLIT_BT_HOR' represents the result of horizontal binary tree partitioning.

[0121] Triple tree partitioning refers to a partitioning technique that partitions the current block into three blocks. Partitioning the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) can be called vertical triple tree partitioning, and dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines crossing the current block) can be called horizontal triple tree partitioning. As a result of the triple tree partitioning, the current block can be partitioned into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice that of the other partitions. 'SPLIT_TT_VER' in Fig. 4(d) represents the result of the vertical triple tree partitioning, and 'SPLIT_TT_HOR' in Fig. 4(e) represents the result of the horizontal triple tree partitioning.

[0122] The number of times a coding tree unit is split can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Accordingly, the maximum partitioning depth of a coding tree unit can vary for each sequence or feature.

[0123] Alternatively, the maximum splitting depth can be determined individually for each splitting technique. For example, the maximum splitting depth allowed for a quadtree splitting may be different from the maximum splitting depth allowed for a binary tree splitting and / or a triple tree splitting.

[0124] The encoder can signal information indicating at least one of a partition shape or a partition depth of the current block via a bitstream. The decoder can determine the partition shape and the partition depth of the coding tree unit based on the information parsed from the bitstream.

[0125] Figure 5 is a diagram illustrating the division aspect of a coding tree unit.

[0126] Partitioning a coding block using a partitioning technique such as quad tree partitioning, binary tree partitioning, and / or triple tree partitioning can be called multi-tree partitioning.

[0127] The coding blocks generated by applying multi-tree partitioning to a coding block can be called sub-coding blocks. If the partition depth of a coding block is k, the partition depths of the sub-coding blocks are set to k+1.

[0128] Conversely, for coding blocks with a split depth of k+1, the coding block with a split depth of k can be called a higher coding block.

[0129] The split type of the current coding block may be determined based on at least one of the split form of the upper coding block or the split type of the neighboring coding block. Here, the neighboring coding block is adjacent to the current coding block and may include at least one of the upper neighboring block, the left neighboring block, or the neighboring block adjacent to the upper left corner of the current coding block. Here, the split type may include at least one of whether a quad tree split is performed, whether a binary tree split is performed, the binary tree splitting direction, whether a triple tree split is performed, or the triple tree splitting direction.

[0130] To determine the partitioning pattern of a coding block, information indicating whether the coding block is partitioned can be signaled via the bitstream. This information is a 1-bit flag 'split_cu_flag', and a true value of this flag indicates that the coding block is partitioned using a head tree partitioning technique.

[0131] When split_cu_flag is true, information indicating whether the coding block is quad-tree split can be signaled via the bitstream. The information is a 1-bit flag split_qt_flag, and when the flag is true, the coding block can be split into four blocks.

[0132] For example, in the example illustrated in FIG. 5, as the coding tree unit is split into quad trees, four coding blocks with a split depth of 1 are generated. In addition, it is illustrated that quad tree splitting is applied again to the first and fourth coding blocks among the four coding blocks generated as a result of the quad tree splitting. As a result, four coding blocks with a split depth of 2 can be generated.

[0133] Additionally, by applying quad tree partitioning again to a coding block with a partition depth of 2, a coding block with a partition depth of 3 can be generated.

[0134] When quad-tree splitting is not applied to a coding block, it is possible to determine whether binary tree splitting or triple-tree splitting is to be performed on the coding block by considering at least one of the size of the coding block, whether the coding block is located at a picture boundary, the maximum splitting depth, or the splitting shape of a neighboring block. When it is determined that binary tree splitting or triple-tree splitting is to be performed on the coding block, information indicating a splitting direction may be signaled through a bitstream. The information may be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it may be determined whether the splitting direction is vertical or horizontal. Additionally, information indicating whether binary tree splitting or triple-tree splitting is to be applied to the coding block may be signaled through a bitstream. The information may be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it may be determined whether binary tree splitting or triple-tree splitting is to be applied to the coding block.

[0135] For example, in the example illustrated in FIG. 5, vertical binary tree splitting is applied to a coding block with a split depth of 1, and among the coding blocks generated as a result of the splitting, vertical triple tree splitting is applied to a left coding block, and vertical binary tree splitting is applied to a right coding block.

[0136] When implementing a device that encodes or decodes an image, it is difficult to process an area larger than a threshold due to hardware performance. For example, if hardware performance allows simultaneous processing of up to 4096 samples, an area with more than 4096 samples may require processing by repeatedly accessing data units of size 64x64, which may result in problems and inability to process data simultaneously. In this way, the basic unit of data processing can be defined as a pipeline-based data basic unit (Virtual Processing Data Unit, VPDU, hereinafter referred to as data basic unit).

[0137] Data basic units can be classified as square, non-square, or non-rectangular types.

[0138] Figure 6 is a drawing showing the form of a basic data unit.

[0139] Data basic units may include a number of samples that is equal to or less than the maximum number of samples that can be processed simultaneously. For example, as in the example illustrated in (a) of FIG. 6, square blocks of size 64x64 may be set as data basic units. Alternatively, non-square blocks may be set as data basic units. For example, as in the example illustrated in (b) or (c) of FIG. 6, blocks of size 32x128 or blocks of size 64x32 may be set as data basic units.

[0140] Although not shown, data basic units of triangular, L-shaped, or polygonal shape may also be defined.

[0141] Information for determining a data basic unit may be signaled via a bitstream. The information may be for determining at least one of the size or shape of the data basic unit. Based on the information, whether a non-square data basic unit is allowed or a non-rectangular data basic unit is allowed may be determined.

[0142] Alternatively, at least one of the size or shape of the data basic unit in the encoder and decoder may be predefined.

[0143] Considering the size of the data basic unit, whether or not to allow the division of the coding block can be determined. For example, if the coding block generated as a result of the division of the coding block is larger than the data basic unit, the division may not be allowed. Alternatively, if the coding block generated as a result of the division of the coding block is of a non-square shape and is larger than the data basic unit, the division may not be allowed. For example, if the width or height of the coding block is larger than a threshold, or if the number of samples included in the coding block is larger than a threshold, binary tree or triple tree division may not be allowed. Accordingly, encoding of information related to binary tree or triple tree division may be omitted.

[0144] Alternatively, a coding block larger than a data basic unit may be configured to be necessarily split. Alternatively, a coding block larger than a data basic unit may be configured to be necessarily subject to binary tree splitting or triple tree splitting. Accordingly, for a coding block larger than a data basic unit, even if the flag split_flag indicating whether the coding block is split is not encoded, the value of the flag may be derived as 1.

[0145] As another example, a coding block larger than a basic data unit can be divided into multiple sub-blocks. Here, the sub-blocks can be set as prediction units, which are basic units for prediction, or transformation units, which are basic units for transformation and / or quantization. In this case, dividing a coding block into multiple prediction units can be defined as VPDU prediction unit partitioning, and dividing a coding block into multiple transformation units can be defined as VPDU transformation unit partitioning.

[0146] At least one of VPDU prediction unit partitioning or VPDU conversion unit partitioning may be applied to a coding block. The partitioning form of the coding block according to the application of VPDU prediction unit partitioning may be set to be the same as the partitioning form of the coding block according to the application of VPDU conversion unit partitioning.

[0147] When only VPDU prediction unit partitioning is applied to a coding block, prediction is performed for each sub-block, but transformation and / or quantization may be performed for the coding block. In this case, a prediction mode, such as a prediction encoding mode, an intra-prediction mode, or an inter-prediction mode, may be determined for the coding block.

[0148] When only VPDU transform unit partitioning is applied to a coding block, prediction is performed on a sub-block basis, but transformation and / or quantization may be performed on a per-sub-block basis.

[0149] Figures 7 and 8 are diagrams showing examples in which a coding block is divided into multiple sub-blocks.

[0150] Figure 7 is a diagram showing a partitioning aspect when only square data basic units are allowed, and Figure 8 is a diagram showing a partitioning aspect when square data basic units and non-square data basic units are allowed.

[0151] Assuming that only square data basic units are allowed, in (a) and (b) of Fig. 7, CU0 and CU3 are defined by two different VPDUs, and CU1 is defined by four different VPDUs. Accordingly, CU0 and CU3 can be divided into two sub-blocks, and CU1 can be divided into four sub-blocks.

[0152] Assuming that both square and non-square data basic units are allowed, in (a) and (b) of FIG. 8, CU0 and CU3 can be defined by a single VPDU, while CU1 can be defined using two different VPDUs. Accordingly, CU0 and CU3 are not divided into sub-blocks, while CU1 can be divided into two sub-blocks.

[0153] At this time, CU1 can be divided into square sub-blocks or non-square sub-blocks. For example, CU1 can be divided into two square sub-blocks based on a horizontal line dividing CU1 vertically. Alternatively, CU1 can be divided into two non-square sub-blocks based on a vertical line dividing CU1 horizontally.

[0154] When there are multiple partition type candidates applicable to a coding block, information indicating one of the multiple partition type candidates may be signaled via the bitstream. For example, the information may indicate whether the coding block is partitioned into square sub-blocks or whether the coding block is partitioned into non-square sub-blocks.

[0155] Alternatively, dividing a coding block into square sub-blocks may be set to have a higher priority than dividing the coding block into non-square sub-blocks. For example, dividing a coding block into non-square sub-blocks may be allowed when dividing the coding block into square sub-blocks is not possible.

[0156] Alternatively, the partitioning shape of the coding block can be determined based on the partitioning shape of the coding block, which is the parent node. For example, if the coding block, which is the parent node, is partitioned based on a triple tree, the coding block can be configured to be partitioned into square sub-blocks. Conversely, if the coding block, which is the parent node, is partitioned based on a binary tree or triple tree, the coding block can be configured to be partitioned into non-square sub-blocks.

[0157]

[0158] Inter prediction is a predictive encoding mode that uses information from previous pictures to predict the current block. For example, a block in the same position as the current block within the previous picture (hereinafter referred to as a collocated block) can be set as the prediction block for the current block. Hereinafter, a prediction block generated based on a block in the same position as the current block will be referred to as a collocated prediction block.

[0159] On the other hand, if an object that existed in the previous picture has moved to a different location in the current picture, the object's movement can be effectively used to predict the current block. For example, if the object's movement direction and size can be determined by comparing the previous picture with the current picture, a prediction block (or predicted image) for the current block can be generated by considering the object's motion information. Hereinafter, a prediction block generated using motion information can be referred to as a motion prediction block.

[0160] A residual block can be generated by subtracting the predicted block from the current block. If object motion exists, using a motion prediction block instead of a collocated prediction block can reduce the energy of the residual block, thereby improving its compression performance.

[0161] As described above, generating prediction blocks using motion information can be called motion-compensated prediction. In most inter predictions, prediction blocks can be generated based on motion-compensated prediction.

[0162] The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index. The motion vector indicates the movement direction and size of the object. The reference picture index specifies the reference picture of the current block among the reference pictures included in the reference picture list. The prediction direction indicates any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). Depending on the prediction direction of the current block, at least one of the motion information in the L0 direction or the motion information in the L1 direction may be used. The bidirectional weight index specifies the weight applied to the L0 prediction block and the weight applied to the L1 prediction block.

[0163] Figure 9 is a flowchart of an inter prediction method according to one embodiment of the present invention.

[0164] Referring to FIG. 9, the inter prediction method includes a step of determining an inter prediction mode of a current block (S901), a step of obtaining motion information of the current block according to the determined inter prediction mode (S902), and a step of performing motion compensation prediction for the current block based on the obtained motion information (S903).

[0165] Here, the inter prediction mode represents various techniques for determining motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information may include a merge mode and a motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. The motion information of the current block may be determined based on information parsed from a neighboring block neighboring the current block or a bitstream, depending on the inter prediction mode.

[0166] The motion information of the current block can be derived from the motion information of another block. Here, the other block may be a block that was encoded / decoded using inter prediction before the current block. Setting the motion information of the current block to be identical to the motion information of the other block can be defined as merge mode. Furthermore, setting the motion vector of the other block as a predicted value of the motion vector of the current block can be defined as motion vector prediction mode.

[0167] Figure 10 is a flowchart of the process of deriving motion information of the current block under merge mode.

[0168] A merge candidate for the current block can be derived (S1001). The merge candidate for the current block can be derived from a block encoded / decoded using inter prediction prior to the current block.

[0169] Figure 11 is a diagram illustrating candidate blocks used to derive merge candidates.

[0170] Candidate blocks may include at least one of neighboring blocks containing samples adjacent to the current block or non-neighboring blocks containing samples not adjacent to the current block. Hereinafter, the samples that determine candidate blocks are defined as reference samples. In addition, reference samples adjacent to the current block are referred to as neighboring reference samples, and reference samples not adjacent to the current block are referred to as non-neighboring reference samples.

[0171] A neighboring reference sample may be included in a neighboring column of the leftmost column of the current block or a neighboring row of the top row of the current block. For example, when the coordinates of the upper left sample of the current block are (0, 0), at least one of a block including a reference sample at a position (-1, H-1), a block including a reference sample at a position (W-1, -1), a block including a reference sample at a position (W, -1), a block including a reference sample at a position (-1, H), or a block including a reference sample at a position (-1, -1) may be used as a candidate block. Referring to the drawing, neighboring blocks having indices 0 to 4 may be used as candidate blocks.

[0172] A non-neighboring reference sample represents a sample in which at least one of the x-axis distance or the y-axis distance from a reference sample adjacent to the current block has a predefined value. For example, at least one of a block including a reference sample in which the x-axis distance from the left reference sample is a predefined value, a block including a non-neighboring sample in which the y-axis distance from the upper reference sample is a predefined value, or a block including a non-neighboring sample in which the x-axis distance and the y-axis distance from the upper left reference sample are predefined values ​​may be used as a candidate block. The predefined value may be a natural number such as 4, 8, 12, or 16. Referring to the drawing, at least one of blocks having indices 5 to 26 may be used as a candidate block.

[0173]

[0174] Alternatively, a candidate block that does not belong to the same coding tree unit as the current block may be set as unavailable as a merge candidate. For example, if a reference sample falls outside the upper boundary of the coding tree unit to which the current block belongs, a candidate block containing the reference sample may be set as unavailable as a merge candidate.

[0175] Merge candidates can also be derived from temporal neighboring blocks included in pictures different from the current block. For example, merge candidates can be derived from collocated blocks included in collocated pictures. Any one of the reference pictures included in the reference picture list can be set as the collocated picture. Index information identifying the collocated picture among the reference pictures can be signaled through the bitstream. Alternatively, a reference picture having a predefined index among the reference pictures can be determined as the collocated picture.

[0176] The motion information of the merge candidate may be set to be identical to the motion information of the candidate block. For example, at least one of the motion vector, reference picture index, prediction direction, or bidirectional weight index of the candidate block may be set as the motion information of the merge candidate.

[0177] A merge candidate list including merge candidates can be created (S1002).

[0178] The indexes of merge candidates in the merge candidate list may be assigned in a predetermined order. For example, the indexes may be assigned in the following order: merge candidate derived from the left neighboring block, merge candidate derived from the upper neighboring block, merge candidate derived from the upper right neighboring block, merge candidate derived from the lower left neighboring block, merge candidate derived from the upper left neighboring block, and merge candidate derived from the temporal neighboring block.

[0179] If multiple merge candidates are included in the merge candidate list, at least one of the multiple merge candidates may be selected (S1003). Specifically, information for specifying one of the multiple merge candidates may be signaled via a bitstream. For example, information "merge_idx" indicating the index of one of the merge candidates included in the merge candidate list may be signaled via a bitstream.

[0180]

[0181] If the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidate included in the inter-domain motion information table can be added to the merge candidate list. Here, the threshold can be the maximum number of merge candidates that the merge candidate list can include or a value obtained by subtracting an offset from the maximum number of merge candidates. The offset can be a natural number such as 1 or 2. The inter-domain motion information table can include merge candidates derived based on blocks encoded / decoded before the current block.

[0182] The inter-region motion information table includes merge candidates derived from blocks encoded / decoded based on inter prediction within the current picture. For example, the motion information of the merge candidates included in the inter-region motion information table may be set to be identical to the motion information of blocks encoded / decoded based on inter prediction. Here, the motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index.

[0183]

[0184] A coding block can be divided into multiple prediction units, and prediction can be performed on each of the divided prediction units. Here, the prediction unit represents a basic unit for performing prediction.

[0185] A coding block can be partitioned using at least one of vertical lines, horizontal lines, diagonal lines, or diagonal lines. Information for determining at least one of the number, angle, or position of lines that partition the coding block can be signaled via the bitstream. For example, information indicating any one of the partition type candidates of the coding block can be signaled via the bitstream, or information specifying any one of a plurality of line candidates that partition the coding block can be signaled via the bitstream. Alternatively, information for determining the number or type of line candidates that partition the coding block can be signaled via the bitstream. For example, a 1-bit flag can be used to determine whether diagonal lines having a greater angle than the diagonal and / or diagonal lines having a smaller angle than the diagonal can be used as line candidates.

[0186] Alternatively, at least one of the number, angle, or position of lines dividing the coding block may be adaptively determined based on at least one of the intra prediction mode of the coding block, the inter prediction mode, the position of available merge candidates, or the splitting aspect of the neighboring block.

[0187] When a coding block is divided into multiple prediction units, intra prediction or inter prediction can be performed on each of the divided prediction units.

[0188] Figure 12 is a diagram showing an example of dividing a coding block into multiple prediction units using a diagonal.

[0189] As in the examples shown in (a) and (b) of Fig. 12, a coding block can be divided into two triangular prediction units using a diagonal.

[0190] In (a) and (b) of FIG. 12, a coding block is divided into two prediction units using a diagonal line connecting two vertices of the coding block. However, a coding block can also be divided into two prediction units using a diagonal line where at least one end does not pass through a vertex of the coding block.

[0191] Figure 13 is a diagram illustrating an example of dividing a coding block into two prediction units.

[0192] As in the examples shown in (a) and (b) of Fig. 13, a coding block can be divided into two prediction units using diagonal lines whose ends are tangent to the upper and lower boundaries of the coding block, respectively.

[0193] Alternatively, as in the examples shown in (c) and (d) of FIG. 13, the coding block may be divided into two prediction units using diagonal lines whose ends are tangent to the left and right boundaries of the coding block, respectively.

[0194] Alternatively, a coding block can be split into two prediction blocks of different sizes. For example, a coding block can be split into two prediction units of different sizes by setting the diagonal line splitting the coding block so that it touches two boundaries forming a single vertex.

[0195] Figure 14 shows examples of dividing a coding block into multiple prediction blocks of different sizes.

[0196] As in the examples shown in (a) and (b) of FIG. 14, by setting the diagonal line connecting the upper left and lower right of the coding block to pass through the left boundary, right boundary, upper boundary, or lower boundary of the coding block instead of the upper left corner or lower right corner, the coding block can be divided into two prediction units having different sizes.

[0197] Alternatively, as in the examples shown in (c) and (d) of FIG. 14, the diagonal connecting the upper right and lower left corners of the coding block may be set to pass through the left boundary, right boundary, upper boundary, or lower boundary of the coding block instead of the upper left corner or lower right corner, thereby splitting the coding block into two prediction units having different sizes.

[0198] Each of the prediction units generated by dividing the coding block will be referred to as an 'Nth prediction unit'. For example, in the examples illustrated in FIGS. 12 to 14, PU1 may be defined as a first prediction unit, and PU2 may be defined as a second prediction unit. The first prediction unit may refer to a prediction unit that includes a sample located at the lower left or upper left within the coding block, and the second prediction unit may refer to a prediction unit that includes a sample located at the upper right or lower right within the coding block.

[0199] Conversely, a prediction unit including a sample located at the upper right or lower right within a coding block can be defined as a first prediction unit, and a prediction unit including a sample located at the lower left or upper left within a coding block can be defined as a second prediction unit.

[0200] The embodiments described below will be explained focusing on a partitioning example using a diagonal. In particular, partitioning a coding block into two prediction units using a diagonal will be referred to as diagonal partitioning or triangular partitioning, and a prediction unit generated based on diagonal partitioning will be referred to as a triangular prediction unit. However, it should be understood that the embodiments described below can also be applied to partitioning examples using vertical lines, horizontal lines, or diagonal lines at different angles from the diagonal.

[0201] Whether to apply diagonal partitioning to a coding block may be determined based on at least one of a slice type, a maximum number of merge candidates that a merge candidate list can include, a size of the coding block, a shape of the coding block, a prediction encoding mode of the coding block, or a partitioning aspect of a parent node.

[0202] For example, it may be determined whether to apply diagonal partitioning to a coding block based on whether the current slice is of type B. Diagonal partitioning may be allowed only when the current slice is of type B.

[0203] Alternatively, it may be determined whether to apply diagonal partitioning to a coding block based on whether the maximum number of merge candidates included in the merge candidate list is 2 or more. Diagonal partitioning may be allowed only when the maximum number of merge candidates included in the merge candidate list is 2 or more.

[0204] Alternatively, if at least one of the width or height is greater than 64 in hardware implementation, a data processing unit of size 64x64 is repeatedly accessed. Accordingly, if at least one of the width or height of a coding block is greater than a threshold, the coding block may not be allowed to be divided into multiple prediction blocks. For example, if at least one of the width or height of a coding block is greater than 64 (e.g., at least one of the width or height is 128), diagonal partitioning may not be used.

[0205] Alternatively, diagonal partitioning may not be allowed for coding blocks with a sample count greater than a threshold, taking into account the maximum number of samples that can be processed simultaneously in hardware implementations. For example, diagonal partitioning may not be allowed for coding tree blocks with a sample count greater than 4096.

[0206] Alternatively, diagonal partitioning may not be allowed for coding blocks containing fewer than a threshold number of samples. For example, if the number of samples contained in a coding block is less than 64, diagonal partitioning may not be applied to the coding block.

[0207] Alternatively, it may be determined whether to apply diagonal partitioning to the coding block based on whether the width and height ratio of the coding block is less than a first threshold or whether the width and height ratio of the coding block is greater than a second threshold. Here, the width and height ratio whRatio of the coding block may be determined as a ratio of the width CbW and the height CbH of the coding block, as in the following mathematical expression 1.

[0208]

[0209] The second threshold may be the reciprocal of the first threshold. For example, if the first threshold is k, the second threshold may be 1 / k.

[0210] Diagonal partitioning can be applied to a coding block only if the width and height ratio of the coding block is between the first threshold and the second threshold.

[0211] Alternatively, triangular partitioning may be used only when the width-to-height ratio of the coding block is less than a first threshold or greater than a second threshold. For example, if the first threshold is 16, diagonal partitioning may not be allowed for coding blocks of size 64x4 or 4x64.

[0212] Alternatively, whether diagonal partitioning is allowed can be determined based on the partitioning pattern of the parent node. For example, if the parent node, the coding block, is partitioned based on a quad-tree partitioning, diagonal partitioning can be applied to the leaf node, the coding block. Conversely, if the parent node, the coding block, is partitioned based on a binary tree or triple tree partitioning, diagonal partitioning can be set not to be allowed for the leaf node, the coding block.

[0213] Alternatively, whether to allow diagonal partitioning may be determined based on the prediction encoding mode of the coding block. For example, diagonal partitioning may be allowed only when the coding block is encoded with intra prediction, when the coding block is encoded with inter prediction, or when the coding block is encoded with a predefined inter prediction mode. Here, the predefined inter prediction mode may indicate at least one of a merge mode, a motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0214] Alternatively, diagonal partitioning can be determined based on the size of the parallel processing region. For example, diagonal partitioning may not be used if the size of the coding block is larger than the size of the parallel processing region.

[0215] Considering two or more of the conditions listed above, it may be determined whether to apply diagonal partitioning to a coding block.

[0216] As another example, information indicating whether diagonal partitioning is to be applied to a coding block can be signaled via the bitstream. This information can be signaled at the sequence, picture, slice, or block level. For example, a flag triangle_partition_flag indicating whether triangle partitioning is to be applied to a coding block can be signaled at the coding block level.

[0217] If it is decided to apply diagonal partitioning to a coding block, information indicating the number of lines dividing the coding block or the positions of the lines can be signaled through the bitstream.

[0218] For example, when a coding block is divided by a diagonal, information indicating the direction of the diagonal dividing the coding block can be signaled through the bitstream. For example, a flag triangle_partition_type_flag indicating the direction of the diagonal can be signaled through the bitstream. The flag indicates whether the coding block is divided by the diagonal connecting the upper left and lower right, or by the diagonal connecting the upper right and lower left. Dividing the coding block by the diagonal connecting the upper left and lower right may be referred to as a left triangle partition type, and dividing the coding block by the diagonal connecting the upper right and lower left may be referred to as a right triangle partition type. For example, a value of the flag of 0 may indicate that the partition type of the coding block is a left triangle partition type, and a value of the flag of 1 may indicate that the partition type of the coding block is a right triangle partition type.

[0219] Additionally, information indicating whether the sizes of the prediction units are the same or information indicating the position of the diagonal that divides the coding block may be signaled through the bitstream. For example, if the information indicating the sizes of the prediction units indicates that the sizes of the prediction units are the same, encoding of the information indicating the position of the diagonal may be omitted, and the coding block may be divided into two prediction units using a diagonal that passes through two vertices of the coding block. On the other hand, if the information indicating the sizes of the prediction units indicates that the sizes of the prediction units are not the same, the position of the diagonal that divides the coding block may be determined based on the information indicating the position of the diagonal. For example, if a left triangular partition type is applied to the coding block, the position information may indicate whether the diagonal touches the left boundary and the bottom boundary of the coding block, or whether it touches the top boundary and the right boundary. Alternatively, if a right triangular partition type is applied to the coding block, the position information may indicate whether the diagonal touches the right boundary and the bottom boundary of the coding block, or whether it touches the top boundary and the left boundary.

[0220] Information indicating the partition type of a coding block can be signaled at the coding block level. Accordingly, the partition type can be determined for each coding block to which diagonal partitioning is applied.

[0221] As another example, information indicating a partition type for a sequence, picture, slice, tile, or coding tree unit may be signaled. In this case, the partition types of coding blocks to which diagonal partitioning is applied within the sequence, picture, slice, tile, or coding tree unit may be set to be the same.

[0222] Alternatively, information for determining a partition type for the first coding unit to which diagonal partitioning is applied within a coding tree unit can be encoded and signaled, and the second and subsequent coding units to which diagonal partitioning is applied can be set to use the same partition type as the first coding unit.

[0223] As another example, the partition type of a coding block may be determined based on the partition type of a neighboring block. Here, the neighboring block may include at least one of a neighboring block adjacent to the upper left corner of the coding block, a neighboring block adjacent to the upper right corner, a neighboring block adjacent to the lower left corner, a neighboring block located above, or a neighboring block located to the left. As an example, the partition type of the current block may be set to be the same as the partition type of the neighboring block. Alternatively, the partition type of the current block may be determined based on whether the left triangle partition type is applied to the upper left neighboring block, the upper right neighboring block, or the lower left neighboring block.

[0224] The size of a prediction unit may be set to be no larger than that of a transformation unit. Furthermore, one prediction unit may be configured to be included in one transformation unit. Accordingly, when VPDU transformation unit partitioning is applied to a coding block, diagonal partitioning may be applied to the transformation unit instead of the coding block.

[0225] Figure 15 is a diagram showing an example in which diagonal partitioning is applied to a conversion unit.

[0226] As in the example illustrated in Fig. 15, when a coding block is divided into multiple transformation units, it is possible to determine whether to apply diagonal partitioning for each transformation unit. Alternatively, it is also possible to determine whether to apply diagonal partitioning for each block having a size smaller than the transformation unit.

[0227] Alternatively, when VPDU transform unit partitioning is applied to a coding block, it is possible to determine whether to apply diagonal partitioning to the coding block, and to set all transform units included in the coding block to follow the determination. For example, when it is determined to apply diagonal partitioning to the coding block, all transform units included in the coding block may be split into two prediction units.

[0228] In order to perform motion prediction compensation for the first triangular prediction unit and the second triangular prediction unit, motion information of each of the first triangular prediction unit and the second triangular prediction unit can be derived. At this time, the motion information of the first triangular prediction unit and the second triangular prediction unit can be derived from merge candidates included in a merge candidate list. In order to distinguish between a general merge candidate list and a merge candidate list used to derive motion information of triangular prediction units, the merge candidate list for deriving motion information of triangular prediction units will be referred to as a triangle merge candidate list, and the merge candidates included in the triangle merge candidate list will be referred to as triangle merge candidates. However, using the above-described merge candidate derivation method and merge candidate list construction method in the triangle merge candidate and triangle merge candidate list construction method is also included in the spirit of the present invention.

[0229] Information for determining the maximum number of triangle merge candidates that the triangle merge candidate list can contain may be signaled via a bitstream. The information may indicate a difference between the maximum number of merge candidates that the triangle merge candidate list can contain and the maximum number of triangle merge candidates that the triangle merge candidate list can contain.

[0230] Triangular merge candidates can be derived from spatial neighboring blocks and temporal neighboring blocks of a coding block.

[0231] Figure 16 is a diagram showing neighboring blocks used to derive triangle merge candidates.

[0232] A triangle merge candidate can be derived using at least one of a neighboring block located above the coding block, a neighboring block located to the left of the coding block, or a collocated block included in a different picture from the coding block. The top neighboring block can include at least one of a block including a sample (xCb+CbW-1, yCb-1) located above the coding block, a block including a sample (xCb+CbW, yCb-1) located above the coding block, or a block including a sample (xCb-1, yCb-1) located above the coding block. The left neighboring block can include at least one of a block including a sample (xCb-1, yCb+CbH-1) located to the left of the coding block, or a block including a sample (xCb-1, yCb+CbH) located to the left of the coding block. A collocated block can be determined as either a block containing samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of a coding block within a collocated picture or a block containing samples (xCb / 2, yCb / 2) located at the center of the coding block.

[0233] Neighboring blocks can be explored in a predefined order, and a triangle merge candidate list can be constructed based on the triangle merge candidates in the predefined order. For example, a triangle merge candidate list can be constructed by exploring triangle merge candidates in the order of B1, A1, B0, A0, C0, B2, and C1.

[0234] The motion information of the triangle prediction units can be derived based on the triangle merge candidate list. That is, the triangle prediction units can share a single triangle merge candidate list.

[0235] In order to derive motion information of a triangle merge unit, information for specifying at least one of the triangle merge candidates included in the triangle merge candidate list may be signaled via a bitstream. For example, index information merge_triangle_idx for specifying at least one of the triangle merge candidates may be signaled via a bitstream.

[0236] Index information can specify a combination of merge candidates of the first triangle prediction unit and the second triangle prediction unit. As an example, Table 1 below shows an example of a combination of merge candidates according to the index information merge_triangle_idx.

[0237] merge_triangle_idx012345678First prediction unit100020013Second prediction unit012103400merge_triangle_idx91011121314151617First prediction unit401100111Second prediction unit022243344merge_triangle_idx181920212223242526First triangle prediction unit122243334Second triangle prediction unit310130240merge_triangle_idx272829303132333435First triangle prediction unit324424343Second triangle prediction unit131132231merge_triangle_idx36373839First triangle prediction unit2243Second triangle prediction unit4424

[0238] The value of the index information merge_triangle_idx being 1 indicates that the motion information of the first triangle prediction unit is derived from a merge candidate with an index of 1, and the motion information of the second triangle prediction unit is derived from a merge candidate with an index of 0. Through the index information merge_triangle_idx, a triangle merge candidate for deriving the motion information of the first triangle prediction unit and a triangle merge candidate for deriving the motion information of the second triangle prediction unit can be determined.

[0239] The partition type of the coding block to which diagonal partitioning is applied can also be determined by the index information. That is, the index information can specify a combination of the merge candidate of the first triangular prediction unit, the merge candidate of the second triangular prediction unit, and the splitting direction of the coding block. When the partition type of the coding block is determined by the index information, the information triangle_partition_type_flag indicating the direction of the diagonal that splits the coding block may not be encoded. Table 2 shows the partition type of the coding block in the index information merge_triangle_idx.

[0240] merge_triangle_idx012345678TriangleDir011001110merge_triangle_idx91011121314151617TriangleDir000100001merge_triangle_idx1819202 12223242526TriangleDir111001111merge_triangle_idx272829303132333435TriangleDir111001010merge_triangle_idx36373839TriangleDir0100

[0241] A variable TriangleDir of 0 indicates that a left triangle partition type is applied to the coding block, and a variable TriangleDir of 1 indicates that a right triangle partition type is applied to the coding block. By combining Tables 1 and 2, the index information merge_triangle_idx can be set to specify a combination of a merge candidate of the first triangle prediction unit, a merge candidate of the second triangle prediction unit, and a splitting direction of the coding block.

[0242] As another example, only index information for one of the first triangle prediction unit and the second triangle prediction unit may be signaled, and an index of a triangle merge candidate for the other of the first triangle prediction unit and the second triangle prediction unit may be determined based on the index information. For example, a triangle merge candidate of the first triangle prediction unit may be determined based on index information merge_triangle_idx indicating an index of one of the triangle merge candidates. Then, a triangle merge candidate of the second triangle prediction unit may be specified based on the merge_triangle_idx. For example, the triangle merge candidate of the second triangle prediction unit may be derived by adding or subtracting an offset to the index information merge_triangle_idx. The offset may be an integer such as 1 or 2. For example, the triangle merge candidate of the second triangle prediction unit may be determined as a triangle merge candidate having an index value obtained by adding 1 to merge_traingle_idx. If merge_triangle_idx points to a triangle merge candidate with the largest index value among triangle merge candidates, the motion information of the second triangle prediction unit can be derived from a triangle merge candidate with an index of 0 or a triangle merge candidate with an index that is a difference of 1 in merge_triangle_idx.

[0243] Alternatively, the motion information of the second triangular prediction unit can be derived from a triangular merge candidate having the same reference picture as the triangular merge candidate of the first triangular prediction unit specified by the index information. Here, the triangular merge candidate having the same reference picture as the triangular merge candidate of the first triangular prediction unit can refer to a triangular merge candidate having at least one of the L0 reference picture or the L1 reference picture and the triangular merge candidate of the first triangular prediction unit. When there are multiple triangular merge candidates having the same reference picture as the triangular merge candidate of the first triangular prediction unit, one of the merge candidates can be selected based on at least one of whether the merge candidate includes bidirectional motion information or a difference value between the index of the merge candidate and the index information.

[0244] As another example, index information may be signaled for each of the first triangle prediction unit and the second triangle prediction unit. For example, first index information 1st_merge_idx for determining a triangle merge candidate of the first triangle prediction unit and second index information 2nd_merge_idx for determining a triangle merge candidate of the second triangle prediction unit may be signaled via a bitstream. Motion information of the first triangle prediction unit may be derived from a triangle merge candidate determined based on the first index information 1st_merge_idx, and motion information of the second triangle prediction unit may be derived from a triangle merge candidate determined based on the second index information 2nd_merge_idx.

[0245] The first index information 1st_merge_idx may indicate the index of any one of the triangle merge candidates included in the triangle merge candidate list. The triangle merge candidate of the first triangle prediction unit may be determined as the triangle merge candidate indicated by the first index information 1st_merge_idx.

[0246] The triangle merge candidate pointed to by the first index information 1st_merge_idx may be set not to be available as a triangle merge candidate of the second triangle prediction unit. Accordingly, the second index information 2nd_merge_idx of the second triangle prediction unit may indicate an index of any one of the remaining triangle merge candidates excluding the triangle merge candidate pointed to by the first index information. When the value of the second index information 2nd_merge_idx is smaller than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangle prediction unit may be determined as a triangle merge candidate having the index information indicated by the second index information 2nd_merge_idx. On the other hand, if the value of the second index information 2nd_merge_idx is equal to or greater than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangle prediction unit can be determined as a triangle merge candidate having an index that is a value obtained by adding 1 to the value of the second index information 2nd_merge_idx.

[0247] Alternatively, whether to signal the second index information may be determined based on the number of triangle merge candidates included in the triangle merge candidate list. For example, if the maximum number of triangle merge candidates that the triangle merge candidate list can include does not exceed 2, the signaling of the second index information may be omitted. If the signaling of the second index information is omitted, the second triangle merge candidate may be derived by adding or subtracting an offset to the first index information. For example, if the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information points to index 0, the second triangle merge candidate may be derived by adding 1 to the first index information. Alternatively, if the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information points to 1, the second triangle merge candidate may be derived by subtracting 1 from the first index information.

[0248] Alternatively, if the signaling of the second index information is omitted, the second index information may be set to a default value. Here, the default value may be 0. The first index information and the second index information may be compared to derive a second triangle merge candidate. For example, if the second index information is smaller than the first index information, the merge candidate with index 0 may be set as the second triangle merge candidate, and if the second index information is equal to or greater than the first index information, the merge candidate with index 1 may be set as the second triangle merge candidate.

[0249] If the triangle merge candidate has unidirectional motion information, the unidirectional motion information of the triangle merge candidate can be set as the motion information of the triangle prediction unit. On the other hand, if the triangle merge candidate has bidirectional motion information, only one of the L0 motion information or the L1 motion information can be set as the motion information of the triangle prediction unit. Whether to use the L0 motion information or the L1 motion information can be determined based on the index of the triangle merge candidate or the motion information of the other triangle prediction unit.

[0250] For example, when the index of the triangle merge candidate is even, the L0 motion information of the triangle prediction unit may be set to 0, and the L1 motion information of the triangle merge candidate may be set to the L1 motion information of the triangle prediction unit. On the other hand, when the index of the triangle merge candidate is odd, the L1 motion information of the triangle prediction unit may be set to 0, and the L0 motion information of the triangle merge candidate may be set to 0. Conversely, when the index of the triangle merge candidate is even, the L0 motion information of the triangle merge candidate may be set to the L0 motion information of the triangle prediction unit, and when the index of the triangle merge candidate is odd, the L1 motion information of the triangle merge candidate may be set to the L1 motion information of the triangle prediction unit. Alternatively, for the first triangle prediction unit, if the number of triangle merge candidates is even, the L0 motion information of the triangle merge candidate may be set to the L0 motion information of the first triangle prediction unit, whereas for the second triangle prediction unit, if the number of triangle merge candidates is odd, the L1 motion information of the triangle merge candidate may be set to the L1 motion information of the second triangle prediction unit.

[0251] Alternatively, if the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit may be set to 0, and the L1 motion information of the triangle merge candidate may be set to the L1 information of the second triangle prediction unit. On the other hand, if the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit may be set to 0, and the L0 motion information of the triangle merge candidate may be set to the L0 positive motion information of the second triangle prediction unit.

[0252] The triangle merge candidate list for deriving motion information of the first triangle prediction unit and the triangle merge candidate list for deriving motion information of the second triangle prediction unit may be set differently.

[0253] For example, if a triangle merge candidate for deriving motion information of the first triangle prediction unit in the triangle merge candidate list is specified based on index information about the first triangle prediction unit, motion information of the second triangle prediction unit can be derived using a triangle merge list including the remaining triangle merge candidates excluding the triangle merge candidate indicated by the index information. Specifically, the motion information of the second triangle prediction unit can be derived from any one of the remaining triangle merge candidates.

[0254] Accordingly, the maximum number of triangle merge candidates included in the triangle merge candidate list of the first triangle prediction unit and the maximum number of triangle merge candidates included in the triangle merge candidate list of the second triangle prediction unit may be different. For example, when the triangle merge candidate list of the first triangle prediction unit includes M merge candidates, the triangle merge candidate list of the second triangle prediction unit may include M-1 merge candidates excluding the triangle merge candidates indicated by the index information of the first triangle prediction unit.

[0255] As another example, the merge candidate of each triangular prediction unit may be derived based on neighboring blocks adjacent to the coding block, and the availability of the neighboring block may be determined by considering the shape or position of the triangular prediction unit.

[0256] Figure 17 is a diagram illustrating an example of determining the availability of neighboring blocks for each triangle prediction unit.

[0257] A neighboring block that is not adjacent to the first triangular prediction unit may be set as unavailable for the first triangular prediction unit, and a neighboring block that is not adjacent to the second triangular prediction unit may be set as unavailable for the second triangular prediction unit.

[0258] For example, as in the example illustrated in (a) of FIG. 17, when a left triangle partition type is applied to a coding block, among the neighboring blocks adjacent to the coding block, blocks A1, A0, and A2 adjacent to the first triangle prediction unit may be determined to be available to the first triangle prediction unit, while blocks B0 and B1 may be determined to be unavailable to the first triangle prediction unit. Accordingly, the triangle merge candidate list for the first triangle prediction unit may include triangle merge candidates derived from blocks A1, A0, and A2, while not including triangle merge candidates derived from blocks B0 and B1.

[0259] As in the example illustrated in (b) of Fig. 17, when a left triangle partition type is applied to a coding block, blocks B0 and B1 adjacent to the second triangle prediction unit may be determined to be available to the second triangle prediction unit, while blocks A1, A0, and A2 may be determined to be unavailable to the second triangle prediction unit. Accordingly, the triangle merge candidate list for the second triangle prediction unit may include triangle merge candidates derived from blocks B0 and B1, while not including triangle merge candidates derived from blocks A1, A0, and A2.

[0260] Accordingly, the number of triangle merge candidates or the range of triangle merge candidates available to the triangle prediction unit may be determined based on at least one of the position of the triangle prediction unit or the partition type of the coding block.

[0261] As another example, the merge mode may be applied to only one of the first triangle prediction unit and the second triangle prediction unit. Furthermore, the motion information of the other of the first triangle prediction unit and the second triangle prediction unit may be set to be identical to the motion information of the triangle prediction unit to which the merge mode is applied, or may be derived by refining the motion information of the triangle prediction unit to which the merge mode is applied.

[0262] For example, based on the triangle merge candidate, the motion vector and reference picture index of the first triangle prediction unit can be derived, and the motion vector of the first triangle prediction unit can be refined to derive the motion vector of the second triangle prediction unit. For example, the motion vector of the second triangle prediction unit can be derived by adding or subtracting the refined motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangle prediction unit. The reference picture index of the second triangle prediction unit can be set to be the same as the reference picture index of the first triangle prediction unit.

[0263] Information for determining a refined motion vector representing the difference between the motion vector of the first triangular prediction unit and the motion vector of the second triangular prediction unit may be signaled via the bitstream. The information may include at least one of information representing the magnitude of the refined motion vector or information representing the sign of the refined motion vector.

[0264] Alternatively, the sign of the refined motion vector can be derived based on at least one of the location, index, or partition type applied to the coding block of the triangular prediction unit.

[0265] As another example, the motion vector and reference picture index of one of the first triangular prediction unit and the second triangular prediction unit can be signaled. The motion vector of the other of the first triangular prediction unit and the second triangular prediction unit can be derived by refining the signaled motion vector.

[0266] For example, a motion vector and a reference picture index of a first triangular prediction unit can be determined based on information signaled from a bitstream. Then, a motion vector of a second triangular prediction unit can be derived by refining the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit can be derived by adding or subtracting a refined motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit can be set to be the same as the reference picture index of the first triangular prediction unit.

[0267] Motion prediction compensation prediction can be performed for each coding block based on the motion information of the first triangular prediction unit and the motion information of the second triangular prediction unit. At this time, image quality deterioration may occur at the boundary portion of the first triangular prediction unit and the second triangular prediction unit. For example, image quality continuity may deteriorate around the edge existing at the boundary portion of the first triangular prediction unit and the second triangular prediction unit. To reduce image quality deterioration at the boundary portion, prediction samples can be derived through a smoothing filter or weighted prediction.

[0268] Prediction samples within a coding block to which diagonal partitioning is applied can be derived based on a weighted sum operation of a first prediction sample obtained based on motion information of a first triangular prediction unit and a second prediction sample obtained based on motion information of a second triangular prediction unit. Alternatively, the prediction sample of the first triangular prediction unit is derived from a first prediction block determined based on motion information of the first triangular prediction unit, and the prediction sample of the second triangular prediction unit is derived from a second prediction block determined based on motion information of the second triangular prediction unit, wherein prediction samples located in a boundary area of ​​the first triangular prediction unit and the second triangular prediction unit can be derived based on a weighted sum operation of the first prediction sample included in the first prediction block and the second prediction sample included in the second prediction block. As an example, the following Mathematical Expression 2 shows an example of deriving prediction samples of the first triangular prediction unit and the second triangular prediction unit.

[0269]

[0270] In the above mathematical expression 2, P1 represents the first prediction sample, and P2 represents the second prediction sample. w1 represents the weight applied to the first prediction sample, and (1-w1) represents the weight applied to the second prediction sample. As in the example shown in mathematical expression 2, the weight applied to the second prediction sample can be derived by differentiating the weight applied to the first prediction sample from a constant value.

[0271] When the left triangular partition type is applied to the coding block, the boundary region may include prediction samples whose x-axis coordinates and y-axis coordinates are the same. On the other hand, when the right triangular partition type is applied to the coding block, the boundary region may include prediction samples whose sum of the x-axis coordinates and y-axis coordinates is greater than or equal to the first threshold value and less than or equal to the second threshold value.

[0272] The size of the boundary region may be determined based on at least one of the size of the coding block, the shape of the coding block, the motion information of the triangular prediction units, the motion vector differential values ​​of the triangular prediction units, the output order of the reference picture, or the differential values ​​of the first prediction sample and the second prediction sample at the diagonal boundary.

[0273] Figures 18 and 19 are diagrams illustrating examples of deriving prediction samples based on a weighted sum operation of the first prediction sample and the second prediction sample. Figure 18 illustrates a case where a left triangular partition type is applied to a coding block, and Figure 19 illustrates a case where a right triangular partition type is applied to a coding block. In addition, Figures 18 (a) and 19 (a) are diagrams illustrating prediction aspects for a luma component, and Figures 18 (b) and 19 (b) are diagrams illustrating prediction aspects for a chroma component.

[0274] In the illustrated drawing, the numbers written in the prediction samples located near the boundaries of the first prediction unit and the second prediction unit represent weights applied to the first prediction sample. For example, if the number written in the prediction sample is N, the prediction sample can be derived by applying a weight of N / 8 to the first prediction sample and a weight of (1-(N / 8)) to the second prediction sample.

[0275] In a non-boundary region, either the first prediction sample or the second prediction sample can be determined as the prediction sample. Referring to the example of Fig. 18, in a region belonging to the first triangular prediction unit among regions where the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is greater than the threshold, the first prediction sample derived based on the motion information of the first triangular prediction unit can be determined as the prediction sample. On the other hand, in a region belonging to the second triangular prediction unit among regions where the difference between the x-axis coordinate and the y-axis coordinate is greater than the threshold, the second prediction sample derived based on the motion information of the second triangular prediction unit can be determined as the prediction sample.

[0276] Referring to the example of Fig. 19, in a region where the sum of the x-axis coordinate and the y-axis coordinate is less than the first threshold, the first prediction sample derived based on the motion information of the first triangular prediction unit may be determined as the prediction sample. On the other hand, in a region where the sum of the x-axis coordinate and the y-axis coordinate is greater than the second threshold, the second prediction sample derived based on the motion information of the second triangular prediction unit may be determined as the prediction sample.

[0277] The threshold for determining the non-boundary region may be determined based on at least one of the size of the coding block, the shape of the coding block, or the color component. For example, if the threshold for the luma component is set to N, the threshold for the chroma component may be set to N / 2.

[0278] Prediction samples included in the boundary region can be derived based on a weighted sum operation of the first prediction sample and the second prediction sample. At this time, the weights applied to the first prediction sample and the second prediction sample can be determined based on at least one of the position of the prediction sample, the size of the coding block, the shape of the coding block, or the color component.

[0279] For example, as in the example illustrated in (a) of FIG. 18, prediction samples at positions where the x-axis coordinate and the y-axis coordinate are the same can be derived by applying the same weight to the first prediction sample and the second prediction sample. Prediction samples in which the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (3:1) or (1:3). In addition, prediction samples in which the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 2 can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0280] Alternatively, as in the example illustrated in (b) of FIG. 18, prediction samples at positions where the x-axis coordinate and the y-axis coordinate are the same can be derived by applying the same weight to the first prediction sample and the second prediction sample, and prediction samples where the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0281] For example, as in the example illustrated in (a) of FIG. 19, prediction samples whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived by applying the same weight to the first prediction sample and the second prediction sample. Prediction samples whose sum of the x-axis coordinate and the y-axis coordinate is equal to or 2 less than the width or height of the coding block can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (3:1) or (1:3). Prediction samples whose sum of the x-axis coordinate and the y-axis coordinate is 1 greater than or 3 less than the width or height of the coding block can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0282] Alternatively, as in the example illustrated in (b) of FIG. 19, prediction samples whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived by applying the same weight to the first prediction sample and the second prediction sample. Prediction samples whose sum of the x-axis coordinate and the y-axis coordinate is equal to or 2 less than the width or height of the coding block can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0283] As another example, weights can be determined by considering the location of the prediction sample or the shape of the coding block. Equations 3 to 5 illustrate examples of deriving weights when a left triangular partition type is applied to the coding block. Equation 3 illustrates an example of deriving weights applied to the first prediction sample when the coding block is square.

[0284]

[0285] In Equation 3, x and y represent the positions of the prediction samples. When the coding block is non-square, the weight applied to the first prediction sample can be derived as in Equation 4 or Equation 5 below. Equation 4 represents the case where the width of the coding block is greater than the height, and Equation 5 represents the case where the width of the coding block is less than the height.

[0286]

[0287]

[0288] When a right-triangular partition type is applied to a coding block, the weight applied to the first prediction sample can be determined as in Equations 6 to 8. Equation 6 shows an example of deriving the weight applied to the first prediction sample when the coding block is square.

[0289]

[0290] In Equation 6, CbW represents the width of the coding block. If the coding block is non-square, the weight applied to the first prediction sample can be derived as in Equation 7 or Equation 8 below. Equation 7 represents the case where the width of the coding block is greater than the height, and Equation 8 represents the case where the width of the coding block is less than the height.

[0291]

[0292]

[0293] In mathematical expression 7, CbH represents the height of the coding block.

[0294] As in the illustrated example, among the prediction samples within the boundary region, those included in the first triangular prediction unit can be derived by giving a greater weight to the first prediction sample than to the second prediction sample, and those included in the second triangular prediction unit can be derived by giving a greater weight to the second prediction sample than to the first prediction sample.

[0295] When diagonal partitioning is applied to a coding block, the coding block may be set not to apply a combined prediction mode that combines an intra prediction mode and a merge mode.

[0296]

[0297] Intra prediction is the process of predicting the current block using reconstructed samples that have been encoded / decoded around the current block. In this case, reconstructed samples from before the in-loop filter is applied can be used for intra prediction of the current block.

[0298] Intra prediction techniques include matrix-based intra prediction and general intra prediction that considers directionality with respect to surrounding reconstruction samples. Information indicating the intra prediction technique of the current block may be signaled through the bitstream. The information may be a 1-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, shape of the current block, or the intra prediction technique of a neighboring block. For example, if the current block straddles a picture boundary, matrix-based intra prediction may not be applied to the current block.

[0299] Matrix-based intra prediction is a method for obtaining a prediction block of a current block based on a matrix product between a previously stored matrix and reconstructed samples surrounding the current block in an encoder and decoder. Information for specifying one of a plurality of previously stored matrices can be signaled through the bitstream. The decoder can determine a matrix for intra prediction of the current block based on the information and the size of the current block.

[0300] General intra prediction is a method of obtaining a predicted block for the current block based on either a non-directional intra prediction mode or a directional intra prediction mode. Below, with reference to the drawings, the intra prediction process based on general intra prediction will be examined in more detail.

[0301] FIG. 20 is a flowchart of an intra prediction method according to one embodiment of the present invention.

[0302] A reference sample line of the current block can be determined (S2001). The reference sample line refers to a set of reference samples included in the k-th line from the top and / or left of the current block. The reference samples can be derived from restored samples whose encoding / decoding has been completed around the current block.

[0303] Index information identifying a reference sample line of the current block among multiple reference sample lines may be signaled via the bitstream. For example, index information intra_luma_ref_idx for specifying a reference sample line of the current block may be signaled via the bitstream. The index information may be signaled on a coding block basis.

[0304] The plurality of reference sample lines may include at least one of the first line, the second line, the third line, or the fourth line from the top and / or the left of the current block. Among the plurality of reference sample lines, a reference sample line composed of a row adjacent to the top of the current block and a column adjacent to the left of the current block may be referred to as an adjacent reference sample line, and other reference sample lines may be referred to as a non-adjacent reference sample line.

[0305] Only some of the multiple reference sample lines may be selected as the reference sample lines of the current block. For example, the remaining reference sample lines, excluding the third non-adjacent reference sample line, may be set as candidate reference sample lines. Table 3 shows the index assigned to each of the candidate reference sample lines.

[0306] Index (intra_luma_ref_idx)Reference sample line 0Contiguous reference sample line 1First non-contiguous reference sample line 2Second non-contiguous reference sample line

[0307] A greater number of candidate reference sample lines may be set than described, or a less number of candidate reference sample lines may be set. In addition, the number or positions of non-adjacent reference sample lines set as candidate reference sample lines are not limited to the described examples. For example, the first non-adjacent reference sample line and the third non-adjacent reference sample line may be set as candidate reference sample lines, or the second non-adjacent reference sample line and the third non-adjacent reference sample line may be set as candidate reference sample lines. Alternatively, the first non-adjacent reference sample line, the second non-adjacent reference sample line, and the third non-adjacent reference sample line may all be set as candidate reference sample lines.

[0308] The number or type of candidate reference sample lines may be determined based on at least one of the size, shape, position, division into sub-blocks, or intra prediction mode of the current block.

[0309] A reference sample line of the current block may be determined based on at least one of the position, size, shape, or predictive encoding mode of the current block or the neighboring block. For example, if the current block borders a picture, tile, slice, or coding tree unit, the neighboring reference sample line may be determined as the reference sample line of the current block.

[0310] Alternatively, if the current block is non-square, the adjacent reference sample line may be determined as the reference sample line of the current block. Alternatively, if the width and height ratio of the current block is greater than or equal to a threshold value, the adjacent reference sample line may be determined as the reference sample line of the current block.

[0311] The reference sample line may include upper reference samples located above the current block and left reference samples located to the left of the current block. The upper reference samples and left reference samples may be derived from reconstructed samples surrounding the current block. The reconstructed samples may be from a state before the in-loop filter is applied.

[0312] Next, the intra prediction mode of the current block can be determined (S2002). The intra prediction mode of the current block can be determined as at least one of a non-directional intra prediction mode or a directional intra prediction mode. The non-directional intra prediction mode includes a planar and a DC, and the directional intra prediction mode includes 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

[0313] Figure 21 is a diagram showing intra prediction modes.

[0314] Figure 21 (a) shows 35 intra prediction modes, and Figure 21 (b) shows 67 intra prediction modes.

[0315] More or fewer intra prediction modes may be defined than those shown in Fig. 21.

[0316] A Most Probable Mode (MPM) can be set based on the intra prediction mode of a neighboring block adjacent to the current block. Here, the neighboring block may include a left neighboring block adjacent to the left of the current block and an upper neighboring block adjacent to the top of the current block.

[0317] The number of MPMs included in the MPM list can be preset in the encoder and decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs can be signaled through the bitstream. Alternatively, the number of MPMs can be determined based on at least one of a prediction coding mode of a neighboring block, the size, shape, or reference sample line index of the current block. For example, when an adjacent reference sample line is determined to be a reference sample line of the current block, N MPMs can be used, whereas when a non-adjacent reference sample line is determined to be a reference sample line of the current block, M MPMs can be used. M is a natural number less than N. For example, N can be 6, and M can be 5, 4, or 3. Accordingly, if the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra prediction mode of the current block is determined as one of the six candidate intra prediction modes, whereas if the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra prediction mode of the current block can be determined as one of the five candidate intra prediction modes.

[0318] Alternatively, a fixed number of MPM candidates (e.g., 6 or 5) can be used, regardless of the index of the reference sample line of the current block.

[0319] An MPM list including multiple MPMs can be generated, and information indicating whether an MPM identical to an intra prediction mode of a current block is included in the MPM list can be signaled via a bitstream. The information can be a 1-bit flag called an MPM flag. If the MPM flag indicates that an MPM identical to the current block is included in the MPM list, index information identifying one of the MPMs can be signaled via the bitstream. For example, index information mpm_idx specifying one of the multiple MPMs can be signaled via the bitstream. The MPM specified by the index information can be set as the intra prediction mode of the current block. If the MPM flag indicates that an MPM identical to the current block is not included in the MPM list, residual mode information indicating one of the remaining intra prediction modes excluding the MPMs can be signaled via the bitstream. Residual mode information refers to the index value corresponding to the intra prediction mode of the current block when the indices are reallocated to the remaining intra prediction modes excluding MPMs. The decoder can sort the MPMs in ascending order and compare the residual mode information with the MPMs to determine the intra prediction mode of the current block. For example, if the residual mode information is equal to or less than the MPM, the intra prediction mode of the current block can be derived by adding 1 to the residual mode information.

[0320] When deriving the intra prediction mode of the current block, comparison of some of the MPMs with the residual mode information may be omitted. For example, MPMs that are non-directional intra prediction modes among the MPMs may be excluded from the comparison. When non-directional intra prediction modes are set as MPMs, since the residual mode information clearly indicates a directional intra prediction mode, the intra prediction mode of the current block can be derived by comparing the residual MPMs excluding the non-directional intra prediction modes with the residual mode information. Instead of excluding the non-directional intra prediction modes from the comparison, the number of non-directional intra prediction modes can be added to the residual mode information, and the result can be compared with the residual MPMs.

[0321] Instead of setting the default mode to MPM, information indicating whether the intra prediction mode of the current block is the default mode can be signaled via the bitstream. The information is a 1-bit flag, and the flag can be referred to as a default mode flag. The default mode flag can be signaled only when the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode can include at least one of a planar, a DC, a vertical mode, or a horizontal mode. For example, when the planar is set to the default mode, the default mode flag can indicate whether the intra prediction mode of the current block is the planar. When the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information can be set as the intra prediction mode of the current block.

[0322] When the default mode flag is used, an intra prediction mode identical to the default mode may be configured not to be set as an MPM. For example, when the default mode flag indicates whether the intra prediction mode of the current block is a planar, the intra prediction mode of the current block may be derived using five MPMs excluding the MPM corresponding to the planar.

[0323] When multiple intra prediction modes are set as default modes, index information indicating one of the default modes may be further signaled. The intra prediction mode of the current block may be set to the default mode indicated by the index information.

[0324] If the index of the reference sample line of the current block is not 0, the default mode can be set not to be used. For example, if a non-adjacent reference sample line is determined to be the reference sample line of the current block, the non-directional intra prediction mode such as DC mode or planar mode can be set not to be used. Accordingly, if the index of the reference sample line is not 0, the default mode flag can be set to a predefined value (i.e., false) without signaling the default mode flag.

[0325] Once the intra prediction mode of the current block is determined, prediction samples for the current block can be obtained based on the determined intra prediction mode (S2003).

[0326] When DC mode is selected, prediction samples for the current block are generated based on the average value of the reference samples. Specifically, the values ​​of all samples within the prediction block can be generated based on the average value of the reference samples. The average value can be derived using at least one of the upper reference samples located at the top of the current block and the left reference samples located to the left of the current block.

[0327] Depending on the shape of the current block, the number or range of reference samples used to derive the average may vary. For example, if the current block is a non-square block whose width is greater than its height, the average may be calculated using only the top reference samples. Conversely, if the current block is a non-square block whose width is smaller than its height, the average may be calculated using only the left reference samples. In other words, if the width and height of the current block are different, the average may be calculated using only the reference samples adjacent to the longer side. Alternatively, based on the width and height ratio of the current block, it may be determined whether to calculate the average using only the top reference samples or only the left reference samples.

[0328] When the planar mode is selected, prediction samples can be obtained using horizontal direction prediction samples and vertical direction prediction samples. Here, the horizontal direction prediction sample is obtained based on a left reference sample and a right reference sample located on the same horizontal line as the prediction sample, and the vertical direction prediction sample is obtained based on an upper reference sample and a lower reference sample located on the same vertical line as the prediction sample. Here, the right reference sample can be generated by copying a reference sample adjacent to the upper right corner of the current block, and the lower reference sample can be generated by copying a reference sample adjacent to the lower left corner of the current block. The horizontal direction prediction sample can be obtained based on a weighted sum operation of the left reference sample and the right reference sample, and the vertical direction prediction sample can be obtained based on a weighted sum operation of the upper reference sample and the lower reference sample. At this time, the weight assigned to each reference sample can be determined according to the position of the prediction sample. The prediction sample can be obtained based on an average operation or a weighted sum operation of the horizontal direction prediction sample and the vertical direction prediction sample. When a weighted sum operation is performed, the weights assigned to the horizontal direction prediction samples and the vertical direction prediction samples can be determined based on the positions of the prediction samples.

[0329] When a directional prediction mode is selected, a parameter indicating the prediction direction (or prediction angle) of the selected directional prediction mode can be determined. Table 4 below shows the intra-directional parameter intraPredAng for each intra-prediction mode.

[0330] PredModeIntraIntraPredAng1-23232642151761379PredModeIntraIntraPredAng859210011-212-513-914-13PredModeIntraIntraPredAng15-1716- 2117-2618-3219-2620-2121-17PredModeIntraIntraPredAng22-1323-924-525-2260272285PredModeIntraIntraPredAng29930133117322133263432

[0331] Table 4 shows the intra directional parameters of each intra prediction mode whose index is between 2 and 34 when 35 intra prediction modes are defined. If more than 33 directional intra prediction modes are defined, Table 4 can be further refined to set the intra directional parameters of each directional intra prediction mode.

[0332] After arranging the upper reference samples and the left reference samples of the current block in a row, a prediction sample can be obtained based on the value of the intra direction parameter. At this time, if the value of the intra direction parameter is negative, the left reference samples and the upper reference samples can be arranged in a row.

[0333] Figures 22 and 23 are diagrams showing examples of one-dimensional arrays that arrange reference samples in a row.

[0334] Fig. 22 illustrates an example of a vertical one-dimensional array that arranges reference samples vertically, and Fig. 23 illustrates an example of a horizontal one-dimensional array that arranges reference samples horizontally. The embodiments of Figs. 22 and 23 are described assuming that 35 intra prediction modes are defined.

[0335] When the intra prediction mode index is any one of 11 to 18, a horizontal one-dimensional array with the upper reference samples rotated counterclockwise may be applied, and when the intra prediction mode index is any one of 19 to 25, a vertical one-dimensional array with the left reference samples rotated clockwise may be applied. When arranging the reference samples in a row, the intra prediction mode angle may be taken into consideration.

[0336] Based on the intra-directional parameters, reference sample decision parameters can be determined. The reference sample decision parameters can include a reference sample index for specifying a reference sample and a weight parameter for determining a weight applied to the reference sample.

[0337] The reference sample index iIdx and the weight parameter ifact can be obtained through the following mathematical expressions 9 and 10, respectively.

[0338]

[0339]

[0340] P in equations 9 and 10 ang represents the intra-directional parameter. The reference sample specified by the reference sample index iIdx corresponds to an integer pel.

[0341] To derive a prediction sample, at least one reference sample can be specified. Specifically, the location of the reference sample used to derive the prediction sample can be specified by considering the slope of the prediction mode. For example, the reference sample index iIdx can be used to specify the reference sample used to derive the prediction sample.

[0342] At this time, if the slope of the intra prediction mode is not expressed by a single reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, if the slope of the intra prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, a prediction sample can be obtained by interpolating the first reference sample and the second reference sample. That is, if an angular line following the intra prediction angle does not pass through a reference sample located at an integer pel, a prediction sample can be obtained by interpolating reference samples located adjacent to the left, right, above, and below the position where the angular line passes.

[0343] The following mathematical expression 11 shows an example of obtaining a prediction sample based on reference samples.

[0344]

[0345] In Equation 11, P represents a prediction sample, and Ref_1D represents one of the one-dimensionally arranged reference samples. At this time, the location of the reference sample can be determined by the location (x, y) of the prediction sample and the reference sample index iIdx.

[0346] If the gradient of the intra prediction mode can be expressed by one reference sample, the weight parameter i fact is set to 0. Accordingly, mathematical expression 11 can be simplified as mathematical expression 12 below.

[0347]

[0348] Intra prediction for the current block can also be performed based on multiple intra prediction modes. For example, an intra prediction mode can be derived for each prediction sample, and each prediction sample can be derived based on the intra prediction mode assigned to each prediction sample.

[0349] Alternatively, an intra prediction mode may be derived for each region, and intra prediction may be performed for each region based on the intra prediction mode assigned to each region. Here, the region may include at least one sample. At least one of the size or shape of the region may be adaptively determined based on at least one of the size, shape, or intra prediction mode of the current block. Alternatively, in the encoder and decoder, at least one of the size or shape of the region may be predefined independently of the size or shape of the current block.

[0350] Alternatively, intra prediction may be performed based on each of a plurality of intra predictions, and a final prediction sample may be derived based on an average operation or a weighted sum operation of a plurality of prediction samples obtained through a plurality of intra predictions. For example, intra prediction may be performed based on a first intra prediction mode to obtain a first prediction sample, and intra prediction may be performed based on a second intra prediction mode to obtain a second prediction sample. Thereafter, a final prediction sample may be obtained based on an average operation or a weighted sum operation between the first prediction sample and the second prediction sample. At this time, the weights assigned to each of the first prediction sample and the second prediction sample may be determined by considering at least one of whether the first intra prediction mode is a non-directional / directional prediction mode, whether the second intra prediction mode is a non-directional / directional prediction mode, or an intra prediction mode of a neighboring block.

[0351] The multiple intra prediction modes may be a combination of a non-directional intra prediction mode and a directional prediction mode, a combination of directional prediction modes, or a combination of non-directional prediction modes.

[0352] Figure 24 is a diagram illustrating the angles formed by directional intra prediction modes with a straight line parallel to the x-axis.

[0353] As shown in the example in Fig. 24, the directional prediction modes can exist between the lower left diagonal direction and the upper right diagonal direction. In terms of the angle formed by the x-axis and the directional prediction modes, the directional prediction modes can exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0354] If the current block is non-square, depending on the intra prediction mode of the current block, there may be cases where a prediction sample is derived using a reference sample that is further from the prediction sample instead of a reference sample that is closer to the prediction sample among the reference samples located on the angular line along the intra prediction angle.

[0355] Figure 25 is a diagram showing how prediction samples are obtained when the current block is a non-square shape.

[0356] For example, as in the example illustrated in (a) of Fig. 25, it is assumed that the current block is a non-square shape with a width greater than a height, and the intra prediction mode of the current block is a directional intra prediction mode having an angle between 0 and 45 degrees. In the above case, when deriving a prediction sample A near the right column of the current block, there may be a case where, among the reference samples located on the angular mode along the angle, the left reference sample L far from the prediction sample is used instead of the upper reference sample T close to the prediction sample.

[0357] As another example, as in the example illustrated in (b) of Fig. 25, it is assumed that the current block is a non-square shape with a height greater than a width, and that the intra prediction mode of the current block is a directional intra prediction mode between -90 degrees and -135 degrees. In the above case, when deriving a prediction sample A near the bottom row of the current block, there may be a case where, among the reference samples located on the angular mode along the angle, the upper reference sample T far from the prediction sample is used instead of the left reference sample L close to the prediction sample.

[0358] To solve the above problem, if the current block is non-square, the intra prediction mode of the current block can be replaced with an intra prediction mode in the opposite direction. Accordingly, for non-square blocks, directional prediction modes with larger or smaller angles than the directional prediction modes illustrated in FIG. 21 can be used. Such directional intra prediction modes can be defined as wide-angle intra prediction modes. Wide-angle intra prediction modes represent directional intra prediction modes that do not fall within the range of 45 degrees to -135 degrees.

[0359] Figure 26 is a diagram showing wide-angle intra prediction modes.

[0360] In the example illustrated in FIG. 26, intra prediction modes with indices of -1 to -14 and intra prediction modes with indices of 67 to 80 represent wide-angle intra prediction modes.

[0361] In FIG. 26, 14 wide-angle intra prediction modes (-1 to -14) having angles greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) having angles less than -135 degrees are exemplified, but more or fewer wide-angle intra prediction modes may be defined.

[0362] When wide-angle intra prediction mode is used, the length of the upper reference samples can be set to 2W+1, and the length of the left reference samples can be set to 2H+1.

[0363] By using the wide-angle intra prediction mode, sample A shown in (a) of FIG. 25 can be predicted using reference sample T, and sample A shown in (b) of FIG. 25 can be predicted using reference sample L.

[0364] By adding the existing intra prediction modes and N wide-angle intra prediction modes, a total of 67 + N intra prediction modes can be used. As an example, Table 5 shows the intra direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0365] PredModeIntra-10-9-8-7-6-5-4-3-2intraPredAngle1149379686054494539PredModeIntra-123456789intraPredAngle353229262321191715PredModeIntra101112131415161718i ntraPredAngle13119753210PredModeIntra192021222324252627intraPredAngle-1-2-3-5-7-9-11-13-15PredModeIntra282930313233343536intraPredAngle-17-19-21-23-26-29 -32-29-26PredModeIntra373839404142434445intraPredAngle-23-21-19-17-15-13-11-9-7PredModeIntra464748495051525354intraPredAngle-5-3-2-101235PredModeIntra55 5657585960616263intraPredAngle7911131517192123PredModeIntra646566676869707172intraPredAngle262932353945495460PredModeIntra73747576intraPredAngle687993114

[0366] If the current block is non-square and the intra prediction mode of the current block obtained in step S2002 falls within a transformation range, the intra prediction mode of the current block may be transformed into a wide-angle intra prediction mode. The transformation range may be determined based on at least one of the size, shape, or ratio of the current block. Here, the ratio may represent the ratio between the width and height of the current block.

[0367] If the current block is a non-square whose width is greater than its height, the transformation range may be set from the intra prediction mode index in the upper right diagonal direction (e.g., 66) to (the index of the intra prediction mode in the upper right diagonal direction - N). Here, N may be determined based on the ratio of the current block. If the intra prediction mode of the current block belongs to the transformation range, the intra prediction mode may be transformed into a wide-angle intra prediction mode. The transformation may be to subtract a predefined value from the intra prediction mode, and the predefined value may be the total number of intra prediction modes excluding the wide-angle intra prediction modes (e.g., 67).

[0368] By the above embodiment, intra prediction modes from 66 to 53 can be converted into wide-angle intra prediction modes from -1 to -14, respectively.

[0369] If the current block is a non-square shape with a height greater than a width, the transformation range may be set from the left lower diagonal intra prediction mode index (e.g., 2) to (the left lower diagonal intra prediction mode index + M). Here, M may be determined based on a ratio of the current block. If the intra prediction mode of the current block falls within the transformation range, the intra prediction mode may be transformed into a wide-angle intra prediction mode. The transformation may be adding a predefined value to the intra prediction mode, and the predefined value may be the total number of directional intra prediction modes excluding wide-angle intra prediction modes (e.g., 65).

[0370] By the above embodiment, each of the intra prediction modes from 2 to 15 can be converted into wide-angle intra prediction modes from 67 to 80.

[0371] Hereinafter, intra prediction modes belonging to the transformation range are referred to as wide-angle intra alternative prediction modes.

[0372] The transformation range can be determined based on the ratio of the current block. As an example, Tables 6 and 7 show the transformation range when 35 intra prediction modes are defined, excluding the wide-angle intra prediction mode, and when 67 intra prediction modes are defined, respectively.

[0373] ConditionReplaced Intra Prediction ModesW / H = 2Modes 2, 3, 4W / H > 2Modes 2, 3, 4, 5, 6W / H = 1NoneH / W = 1 / 2Modes 32, 33, 34H / W < 1 / 2Modes 30, 31, 32, 33, 34

[0374] ConditionReplaced Intra Prediction ModesW / H = 2Modes 2, 3, 4, 5, 6, 7W / H > 2Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11W / H = 1NoneH / W = 1 / 2Modes 61, 62, 63, 64, 65, 66H / W < 1 / 2Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0375] As shown in the examples in Tables 6 and 7, the number of wide-angle intra alternative prediction modes included in the transformation range may vary depending on the ratio of the current block.

[0376] By further dividing the ratio of the current block, the conversion range can be set as shown in Table 8 below.

[0377] ConditionReplaced Intra Prediction ModesW / H = 16Modes 12, 13, 14, 15W / H = 8Modes 12, 13W / H = 4Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11H / W = 2Modes 2, 3, 4, 5, 6, 7H / W = 1NoneW / H = 1 / 2Modes 61, 62, 63, 64, 65, 66W / H = 1 / 4Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66W / H = 1 / 8Modes 55, 56H / W = 1 / 16Modes 53, 54, 55, 56

[0378] When a non-adjacent reference sample line is determined as the reference sample line of the current block, or when a multi-line intra prediction encoding method is used that selects any one of a plurality of reference sample lines, the wide-angle intra prediction mode may be set not to be used. That is, even if the current block is non-square and the intra prediction mode of the current block falls within the transformation range, the intra prediction mode of the current block may not be converted to the wide-angle intra prediction mode.

[0379] Alternatively, if the intra prediction mode of the current block is determined to be a wide-angle intra prediction mode, non-adjacent reference sample lines may be set to not be available as reference sample lines of the current block, or a multi-line intra prediction encoding method for selecting any one of a plurality of reference sample lines may be set not to be used. If the multi-line intra prediction encoding method is not used, an adjacent reference sample line may be determined as the reference sample line of the current block.

[0380] When the wide-angle intra prediction mode is not used, refW and refH can be set to the sum of nTbW and nTbH. Accordingly, excluding the upper left reference sample, the non-adjacent reference sample whose distance from the current block is i can include (nTbW + nTbH + offsetX[i]) upper reference samples and (nTbW + nTbH + offsetY[i]) left reference samples. That is, the non-adjacent reference sample whose distance from the current block is i can include (2nTbW + 2nTbH + offsetX[i] + offsetY[i] + 1) reference samples. For example, when the value of whRatio is greater than 1, the value of offsetX can be set to be greater than the value of offsetY. For example, the value of offsetX can be set to 1, and the value of offsetY can be set to 0. On the other hand, when the value of whRatio is less than 1, the value of offsetY can be set to be greater than the value of offsetX. For example, the value of offsetX may be set to 0, and the value of offsetY may be set to 1.

[0381] As wide-angle intra prediction modes are used in addition to existing intra prediction modes, the resources required to encode the wide-angle intra prediction modes increase, which may lower encoding efficiency. Accordingly, instead of encoding the wide-angle intra prediction modes as is, encoding alternative intra prediction modes for the wide-angle intra prediction modes can be performed to improve encoding efficiency.

[0382] For example, if the current block is encoded using the wide-angle intra prediction mode 67, then the wide-angle alternative intra prediction mode 2 of the 67 can be encoded as the intra prediction mode of the current block. Also, if the current block is encoded using the wide-angle intra prediction mode -1, then the wide-angle alternative intra prediction mode 66 of the -1 can be encoded as the intra prediction mode of the current block.

[0383] The decoder can decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode falls within the transformation range. If the decoded intra prediction mode is a wide-angle alternative intra prediction mode, the intra prediction mode can be converted to a wide-angle intra prediction mode.

[0384] Alternatively, if the current block is encoded in wide-angle intra prediction mode, the wide-angle intra prediction mode may be encoded as is.

[0385] Encoding of the intra prediction mode can be performed based on the above-described MPM list. Specifically, if a neighboring block is encoded in a wide-angle intra prediction mode, the MPM can be set based on a wide-angle alternative intra prediction mode corresponding to the wide-angle intra prediction mode.

[0386]

[0387] Once a prediction block is generated as a result of performing intra prediction, the prediction samples can be updated based on the positions of each prediction sample included in the prediction block. This update method can be called a sample position-based intra-weighted prediction method (or Position Dependent Prediction Combination, PDPC).

[0388] Whether to use PDPC can be determined by considering the intra prediction mode of the current block, the reference sample line of the current block, the size of the current block, or the color component. For example, PDPC can be used when the intra prediction mode of the current block is at least one of planar, DC, vertical, horizontal, a mode with an index value smaller than the vertical direction, or a mode with an index value larger than the horizontal direction. Alternatively, PDPC can be used only when at least one of the width or height of the current block is greater than 4. Alternatively, PDPC can be used only when the index of the reference picture line of the current block is 0. Alternatively, PDPC can be used only when the index of the reference picture line of the current block is greater than or equal to a predefined value. Alternatively, PDPC can be used only for the luminance component. Alternatively, whether to use PDPC can be determined based on whether two or more of the conditions listed above are satisfied.

[0389] As another example, information indicating whether PDPC is applied can be signaled via the bitstream.

[0390] When a prediction sample is obtained through an intra prediction sample, a reference sample used to correct the prediction sample can be determined based on the position of the obtained prediction sample. For convenience of explanation, in the embodiments described below, the reference sample used to correct the prediction sample will be referred to as a PDPC reference sample. In addition, the prediction sample obtained through intra prediction will be referred to as a first prediction sample, and the prediction sample obtained by correcting the first prediction sample will be referred to as a second prediction sample.

[0391] Figure 27 is a drawing showing the application aspect of PDPC.

[0392] The first prediction sample may be corrected using at least one PDPC reference sample. The PDPC reference sample may include at least one of a reference sample adjacent to the upper left corner of the current block, an upper reference sample located at the upper end of the current block, or a left reference sample located at the left end of the current block.

[0393] At least one of the reference samples belonging to the reference sample line of the current block may be set as a PDPC reference sample. Alternatively, regardless of the reference sample line of the current block, at least one of the reference samples belonging to the reference sample line with index 0 may be set as a PDPC reference sample. For example, even if the first prediction sample is obtained using a reference sample included in the reference sample line with index 1 or 2, the second prediction sample may be obtained using a reference sample included in the reference sample line with index 0.

[0394] The number or position of PDPC reference samples used to correct the first prediction sample may be determined by considering at least one of the intra prediction mode of the current block, the size of the current block, the shape of the current block, or the position of the first prediction sample.

[0395] For example, if the intra prediction mode of the current block is planar or DC mode, the second prediction sample can be obtained using the upper reference sample and the left reference sample. At this time, the upper reference sample can be a reference sample that is perpendicular to the first prediction sample (e.g., a reference sample with the same x-coordinate), and the left reference sample can be a reference sample that is horizontal to the first prediction sample (e.g., a reference sample with the same y-coordinate).

[0396] If the intra prediction mode of the current block is a horizontal intra prediction mode, a second prediction sample can be obtained using an upper reference sample. In this case, the upper reference sample may be a reference sample perpendicular to the first prediction sample.

[0397] If the intra prediction mode of the current block is a vertical intra prediction mode, a second prediction sample can be obtained using a left reference sample. In this case, the left reference sample may be a reference sample that is horizontal to the first prediction sample.

[0398] If the intra prediction mode of the current block is the lower left diagonal or upper right diagonal intra prediction mode, the second prediction sample can be obtained based on the upper left reference sample, the upper reference sample, and the left reference sample. The upper left reference sample may be a reference sample adjacent to the upper left corner of the current block (e.g., a reference sample at position (-1, -1)). The upper reference sample may be a reference sample located in the upper right diagonal direction of the first prediction sample, and the left reference sample may be a reference sample located in the lower left diagonal direction of the first prediction sample.

[0399] In summary, when the location of the first prediction sample is (x, y), R(-1, -1) is set as the upper left reference sample, and R(x+y+1, -1) or R(x, -1) can be set as the upper reference sample. In addition, R(-1, x+y+1) or R(-1, y) can be set as the left reference sample.

[0400]

[0401] A single prediction mode can be applied multiple times to the current block, or multiple prediction modes can be applied in a nested manner. This type of prediction method utilizing homogeneous or heterogeneous prediction modes can be called a combined prediction mode (or multi-hypothesis prediction mode).

[0402] The combined prediction mode may include at least one of a merge mode and a combined merge mode, a combined inter prediction and an intra prediction, a combined merge mode and a motion vector prediction mode, or a combined merge mode and an intra prediction.

[0403] In the combined prediction mode, a first prediction block may be generated based on a first prediction mode, and a second prediction block may be generated based on a second prediction mode. Then, a third prediction block may be generated based on a weighted sum operation of the first and second prediction blocks. The third prediction block may be set as the final prediction block of the current block.

[0404] Whether to use the combined prediction mode may be determined based on the size or shape of the current block. For example, whether to use the combined prediction mode may be determined based on at least one of the size of the coding block, the number of samples contained in the coding block, the width of the coding block, or the height of the coding block. For example, if at least one of the width or height of the coding block is 128 or greater, or if the number of samples contained in the coding block is 64 or less, the combined prediction mode may not be applied to the coding block.

[0405] Alternatively, information indicating whether the combined prediction mode is applied to the current block can be signaled via the bitstream. For example, the information can be a 1-bit flag. For example, a flag mh_intra_flag indicating whether the combined prediction mode combining the merge mode and intra prediction is used can be signaled via the bitstream. mh_intra_flag being 1 indicates that the combined prediction mode combining the merge mode and intra prediction is used, and mh_intra_flag being 0 indicates that the combined prediction mode combining the merge mode and intra prediction is not used.

[0406] A combined prediction mode, which combines the merge mode and intra prediction, can be applied to the current block only when the prediction encoding mode of the current block is the inter prediction mode and the merge mode is applied to the current block. That is, mh_intra_flag can be signaled when the value of the flag merge_flag, which indicates whether the merge mode is applied to the current block, is 1.

[0407] In a prediction mode that combines merge mode and intra prediction, the intra prediction mode of the current block may be set to a predefined intra prediction mode. For example, when the combined prediction mode is used, the intra prediction mode of the current block may be set to planar mode.

[0408] As another example, when a combined prediction mode that combines merge mode and intra prediction is used, the intra prediction mode of the current block may be determined as one of the candidate intra prediction modes. Here, the candidate intra prediction modes may include at least one of a non-directional intra prediction mode or a directional intra prediction mode in a specific direction. Here, the non-directional intra prediction mode may include at least one of a DC or a planar, and the directional intra prediction mode may include at least one of a horizontal intra prediction mode, a vertical intra prediction mode, or a diagonal intra prediction mode. For example, when the combined prediction mode is used, only the DC, the planar, the horizontal intra prediction mode, or the vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when the combined prediction mode is used, only the planar, the horizontal intra prediction mode, or the vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when the combined prediction mode is used, only the DC, planar, or vertical intra prediction modes may be set as the intra prediction mode of the current block. Alternatively, when the combined prediction mode is used, only the DC, planar, or horizontal intra prediction modes may be set as the intra prediction mode of the current block.

[0409] Index information for specifying one of the candidate intra prediction modes can be signaled through the bitstream. For example, an index mh_intra_idx specifying one of the candidate intra prediction modes can be signaled through the bitstream. Tables 9 and 10 show intra prediction modes according to the value of mh_intra_idx. The intra prediction mode of the current block can be determined by the intra prediction mode indicated by mh_intra_idx.

[0410] mh_intra_idx0123intra modePLANARDCVERTICALHORIZONTAL

[0411] mh_intra_idx012intra modePLANARVERTICALHORIZONTAL

[0412] The number of candidate intra prediction modes may be fixed in the encoder and decoder. Alternatively, the number or type of candidate intra prediction modes may vary depending on at least one of the size and shape of the current block or whether neighboring blocks are encoded using intra prediction.

[0413] As another example, when a combined prediction mode that combines merge mode and intra prediction is applied, the intra prediction mode of the current block can be determined based on the MPMs. At this time, the number of MPMs included in the MPM list when the general intra prediction is applied may be different from the number of MPMs included in the MPM list when the combined prediction mode is applied. For example, when the combined prediction mode is not applied, the MPM list may include 6 or 5 MPMs, whereas when the combined prediction mode is applied, the MPM list may include 4 or 3 MPMs. In other words, when performing general intra prediction, N MPMs are used, whereas when the combined prediction mode is applied, fewer than N MPMs may be used.

[0414] For example, when a general intra prediction mode is applied, it is assumed that the MPMs derived based on the intra prediction modes of neighboring blocks adjacent to the current block are Planar, DC, INTRA_MODE32, INTRA_MODE31, INTRA_MODE33, and INTRA_MODE30. When a combined prediction mode is applied to the current block, the intra prediction mode of the current block can be determined using only some of the six MPMs. For example, the intra prediction mode of the current block can be determined using an MPM list including the three MPMs with the smallest indices among the MPMs, Planar, DC, and INTRA_MODE32. Alternatively, any one of the MPMs having a predefined index value can be determined as the intra prediction mode of the current block.

[0415] As another example, the number of MPMs used to determine the intra prediction mode of the current block when general intra prediction is applied and the number of MPMs used to determine the intra prediction mode of the current block when combined prediction mode is applied may be set to be the same.

[0416] When the combined prediction mode is applied, the encoding of the MPM flag can be omitted and the MPM flag can be considered as true. That is, when the combined prediction mode is applied, any one of the MPMs can be determined as the intra prediction mode of the current block.

[0417] If the current block uses a combined prediction mode that combines merge mode and intra prediction, the multi-line intra prediction encoding method can be disabled. If the multi-line intra prediction encoding method is not used, adjacent reference sample lines can be used for intra prediction of the current block.

[0418] Alternatively, when a combined prediction mode that combines merge mode and intra prediction is applied to the current block, a multi-line intra prediction encoding method may be configured to be used. Information for determining a reference sample line of the current block among a plurality of reference sample lines may be signaled through the bitstream. Alternatively, the reference sample line of the current block may be determined by considering at least one of the size of the current block, the shape of the current block, the intra prediction mode of the current block, or the reference sample line of a neighboring block.

[0419] Depending on the reference sample line of the current block, the number of available candidate intra prediction modes may be set differently. For example, when adjacent reference sample lines are used, the candidate intra prediction modes may include at least one of a planar, DC, a horizontal intra prediction mode, a vertical intra prediction mode, a lower-left diagonal intra prediction mode, or an upper-right diagonal intra prediction mode. On the other hand, when non-adjacent reference sample lines are used, the candidate intra prediction modes may include at least one of a horizontal intra prediction mode, a vertical intra prediction mode, a lower-left diagonal intra prediction mode, or an upper-right diagonal intra prediction mode.

[0420] A first prediction block can be generated using a merge candidate selected under merge mode, and a second prediction block can be generated using intra prediction mode. At this time, when generating the first prediction block, it can be set so that triangular partitioning is not applied to the current block. That is, in a combined prediction mode that combines merge mode and intra prediction, triangular partitioning may not be allowed. In addition, when generating the second prediction block, it can be set so that PDPC is not applied to the current block. That is, in a combined prediction mode that combines merge mode and intra prediction, PDPC may not be allowed.

[0421] A third prediction block can be generated based on the weighted sum operation of the first and second prediction blocks. As an example, Equation 13 is a diagram illustrating an example of generating a third prediction block based on the weighted sum operation of the first and second prediction blocks.

[0422]

[0423] In the above mathematical expression 13, P merge represents the first prediction block obtained based on the merge mode, and P intra represents the second prediction block obtained based on intra prediction. P comb represents a third prediction block obtained by combining the first prediction block and the second prediction block. w represents a first weight applied to the first prediction block. The second weight applied to the second prediction block can be derived by differentiating the first weight w from a constant N. Here, N can have a value predefined in the encoder and decoder. For example, N can be 4 or 8. Alternatively, the constant N can be derived based on at least one of the size, shape, or intra prediction mode of the current block.

[0424] Unlike the example shown in Equation 13, w may be set as the second weight, and a value obtained by subtracting the second weight from a predetermined constant N may be set as the first weight.

[0425] The weights applied to the first prediction block and the second prediction block may be determined based on at least one of whether the intra prediction mode or the merge mode of the current block has bidirectional motion information. For example, when the intra prediction mode of the current block is a directional prediction mode, the value of the first weight w may have a value greater than the value of the first weight w when the intra prediction mode of the current block is a non-directional mode. Alternatively, when bidirectional prediction is performed by a merge candidate, the value of the first weight w may have a value greater than the value of the first weight w when unidirectional prediction is performed by the merge candidate.

[0426] As another example, weights may be determined by considering the predictive encoding mode of neighboring blocks adjacent to the current block. Here, the neighboring block may include at least one of an upper neighboring block adjacent to the upper side of the current block, a left neighboring block adjacent to the left side of the current block, or an upper left neighboring block adjacent to the upper left corner of the current block. The weights may be determined based on the number of neighboring blocks encoded by intra prediction among the neighboring blocks adjacent to the current block. For example, as the number of neighboring blocks encoded by intra prediction among the neighboring blocks adjacent to the current block increases, the first weight may be set to have a smaller value and the second weight may be set to have a larger value. On the other hand, as the number of neighboring blocks encoded by non-intra prediction among the neighboring blocks adjacent to the current block decreases, the first weight may be set to have a larger value and the second weight may be set to have a smaller value. For example, if both the left neighboring block and the upper neighboring block are encoded by intra prediction, the second weight may be set to n. If only one of the left neighboring block and the upper neighboring block is encoded using intra prediction, the second weight may be set to n / 2. If neither the left neighboring block nor the upper neighboring block is encoded using intra prediction, the second weight may be set to n / 4, where n represents an integer less than or equal to N.

[0427] Weights can be determined on a sample-by-sample or sub-block basis within the current block. For example, the current block can be divided into multiple sub-blocks, and the weights applied to each sub-block can be set to have different values. In this case, the number of sub-blocks can have a fixed value in the encoder and decoder. Alternatively, the number of sub-blocks can be adaptively determined based on at least one of the size, shape, intra-prediction mode of the current block, or the prediction encoding mode of a neighboring block.

[0428] Figure 28 is a diagram showing the application of weights.

[0429] The current block can be divided into multiple sub-blocks, and the weights applied to each sub-block can be set to different values. For example, as shown in the examples in (a) and (b) of FIG. 28, the second weight can be set to have a value greater than the first weight for a sub-block containing a sample located at the upper left corner within the current block, while the first weight can be set to have a value greater than the second weight for a sub-block containing a sample located at the lower right corner within the current block.

[0430] A third prediction sub-block for a sub-block can be obtained by weighting the first prediction sub-block and the second prediction sub-block corresponding to the sub-block. At this time, as in the example illustrated in FIG. 28, the weights used for weighted prediction can be set differently for each sub-block.

[0431] Depending on whether the merge candidate has bidirectional motion information, the weights assigned to the sub-blocks may be determined differently.

[0432] Figure 29 illustrates an example in which different weights are assigned to sub-blocks depending on whether the merge candidate has bidirectional motion information.

[0433] When the merge candidate has bidirectional motion information, the first weight w assigned to at least one of the sub-blocks can be set to a larger value compared to when the merge candidate has unidirectional motion information. For example, in (a) and (b) of FIG. 29, it is illustrated that, for all sub-blocks, the weight w applied when bidirectional prediction is applied has a larger value than the weight w applied when unidirectional prediction is applied.

[0434] Alternatively, the weights to be applied to the sub-blocks can be determined based on the intra prediction mode of the current block.

[0435] Figure 30 shows an example in which different weights are assigned to sub-blocks depending on the intra prediction mode of the current block.

[0436] If the intra prediction mode of the current block is vertical, the second weight of the sub-block located at the top of the current block can be set to have a value greater than the second weight of the sub-block located at the bottom of the current block.

[0437] For example, for a sub-block located at the top of the current block, the second weight may be set to have a larger value than the first weight, while for a sub-block located at the bottom of the current block, the first and second weights may be set to be the same, or the first weight may be set to have a larger value than the second weight.

[0438] If the intra prediction mode of the current block is horizontal, the second weight of the sub-block located on the left side of the current block can be set to have a value greater than the second weight of the sub-block located on the right side of the current block.

[0439] For example, for a sub-block located on the left side of the current block, the second weight may be set to have a larger value than the first weight, while for a sub-block located on the right side of the current block, the first weight and the second weight may be set to be the same, or the first weight may be set to have a larger value than the second weight.

[0440] As another example, when the intra prediction mode of the current block is vertical, the first weight of the sub-block located at the top of the current block may be set to a value greater than the first weight of the sub-block located at the bottom of the current block, or when the intra prediction mode of the current block is horizontal, the first weight of the sub-block located at the left of the current block may be set to a value greater than the first weight of the sub-block located at the right of the current block.

[0441] As another example, the weights assigned to each sub-block may be determined based on the predictive encoding modes of the neighboring blocks adjacent to the current block. For example, if the upper neighboring block of the current block is encoded with intra prediction, while the left neighboring block of the current block is not encoded with intra prediction, the second weight of the sub-block located at the upper side of the current block may be set to have a value greater than the second weight of the sub-block located at the lower side of the current block. On the other hand, if the upper neighboring block of the current block is not encoded with intra prediction, while the left neighboring block of the current block is encoded with intra prediction, the second weight of the sub-block located at the left side of the current block may be set to have a value greater than the second weight of the sub-block located at the right side of the current block.

[0442] For at least one of the plurality of sub-blocks, the first weight or the second weight may be set to 0. That is, for at least one of the plurality of sub-blocks, the first prediction sub-block or the second prediction sub-block may be set to the third prediction sub-block. Accordingly, joint prediction may not be performed for at least one of the plurality of sub-blocks. For example, the first weight w may be set to 0 for at least one of the plurality of sub-blocks, or the first weight w may be set to a constant N.

[0443] A block to which a combined prediction mode, which combines merge mode and intra prediction, is applied can be considered encoded using inter prediction. Accordingly, the intra prediction mode of a block encoded using the combined prediction mode can be set as unavailable for deriving the intra prediction mode of a block to be subsequently encoded / decoded. When deriving MPM, the intra prediction mode of an unavailable block can be considered as planar.

[0444] On the other hand, the motion information of a block encoded in the joint prediction mode can be set to be available for deriving motion information of a block to be encoded / decoded later.

[0445] In a combined merge mode and merge mode, motion-compensated prediction can be performed using multiple merge candidates. Specifically, a first prediction block can be generated using a first merge candidate, and a second prediction block can be generated using a second merge candidate. A third prediction block can be generated based on a weighted sum operation of the first and second prediction blocks.

[0446] Information for specifying the first merge candidate and the second merge candidate may be signaled via a bitstream, respectively. For example, index information merge_idx for specifying the first merge candidate and index information merge_2nd_idx for specifying the second merge candidate may be signaled via a bitstream. The second merge candidate may be determined based on the index information merge_2nd_idx and the index information merge_idx.

[0447] The index information merge_idx specifies one of the merge candidates included in the merge candidate list.

[0448] The index information merge_2nd_idx can specify any one of the remaining merge candidates excluding the merge candidate specified by merge_idx. Accordingly, if the value of merge_2nd_idx is smaller than merge_idx, the merge candidate whose index is the value of merge_2nd_idx can be set as the second merge candidate. If the value of merge_2nd_idx is equal to or greater than the value of merge_idx, the merge candidate whose index is the value of merge_2nd_idx plus 1 can be set as the second merge candidate.

[0449] Alternatively, the second merge candidate can be identified by considering the search order of the candidate blocks.

[0450] Figure 31 shows an example in which a second merge candidate is specified by considering the search order of candidate blocks.

[0451] In the example illustrated in Figure 31, the indices indicated for neighboring samples and non-neighboring samples indicate the search order of candidate blocks. For example, candidate blocks may be searched sequentially from position A0 to position A14.

[0452] If block A4 is selected as the first merge candidate, a merge candidate derived from a candidate block with a search order following A4 can be designated as the second merge candidate. For example, a merge candidate derived from block A5 can be selected as the second merge candidate. If the candidate block at position A5 is not available as a merge candidate, a merge candidate derived from a next-ranked candidate block can be selected as the second merge candidate.

[0453] A first merge candidate and a second merge candidate may be selected from among the merge candidates derived from non-neighboring blocks.

[0454] Figure 32 shows an example in which a first merge candidate and a second merge candidate are selected from among merge candidates derived from non-neighboring blocks.

[0455] As in the example illustrated in Fig. 32, merge candidates derived from the first and second candidate blocks that are not adjacent to the current block can be selected as the first merge candidate and the second merge candidate, respectively. At this time, the block line to which the first candidate block belongs and the block line to which the second candidate block belongs may be different. For example, the first merge candidate may be derived from any one of candidate blocks A5 to A10, and the second merge candidate may be derived from any one of candidate blocks A11 to A15.

[0456] Alternatively, the first candidate block and the second candidate block may be set not to be included in the same line (e.g., row or column).

[0457] As another example, a second merge candidate may be specified based on a first merge candidate. At this time, the first merge candidate may be specified by index information merge_idx signaled from the bitstream. As an example, a merge candidate neighboring the first merge candidate may be specified as a second merge candidate. Here, a merge candidate neighboring the first merge candidate may mean a merge candidate whose index difference value with the first merge candidate is 1. As an example, a merge candidate whose index value is merge_idx+1 may be set as the second merge candidate. At this time, if merge_idx+1 is greater than the maximum index value (or if the index value of the first merge candidate is the maximum index), a merge candidate whose index value is merge_idx-1 or a merge candidate whose index value is a predefined value (e.g., 0) may be set as the second merge candidate.

[0458] Alternatively, a merge candidate adjacent to a first merge candidate may refer to a merge candidate derived from a candidate block that is spatially adjacent to the candidate block used to derive the first merge candidate. Here, a neighboring candidate block of a candidate block may refer to a block adjacent to the left, right, above, below, or diagonally of the candidate block.

[0459] As another example, a second merge candidate can be identified based on the motion information of the first merge candidate. For example, a merge candidate having the same reference picture as the first merge candidate can be selected as the second merge candidate. If there are multiple merge candidates having the same reference picture as the first merge candidate, among the multiple merge candidates, the merge candidate with the smallest index, the merge candidate with the smallest index difference from the first merge candidate, or the merge candidate with the smallest motion vector difference from the first merge candidate can be selected as the second merge candidate. Alternatively, the second merge candidate can be selected based on index information identifying any one of the multiple merge candidates.

[0460] Alternatively, if the first merge candidate is a unidirectional prediction in the first direction, a merge candidate including motion information for the second direction may be set as the second merge candidate. For example, if the first merge candidate has motion information in the L0 direction, a merge candidate having motion information in the L1 direction may be selected as the second merge candidate. If there are multiple merge candidates having motion information in the L1 direction, a merge candidate having the smallest index among the multiple merge candidates or a merge candidate having the smallest index difference from the first merge candidate may be set as the second merge candidate. Alternatively, the second merge candidate may be selected based on index information specifying any one of the multiple merge candidates.

[0461] As another example, the first merge candidate may be set to one of the merge candidates derived from neighboring blocks adjacent to the current block, and the second merge candidate may be set to one of the merge candidates derived from non-neighboring blocks not adjacent to the current block.

[0462] As another example, the first merge candidate may be set to one of the merge candidates derived from candidate blocks located on top of the current block, and the second merge candidate may be set to one of the merge candidates derived from candidate blocks located on the left.

[0463] In the combined prediction mode in which the above-described merge mode and merge mode are combined, the merge mode may mean a merge mode based on a translational motion model (hereinafter referred to as a translational merge mode) or a merge mode based on an affine motion model (hereinafter referred to as an affine merge mode). In other words, motion compensation prediction can be performed by combining a translational merge mode and a translational merge mode, or by combining an affine merge mode and an affine merge mode.

[0464] As another example, if a neighboring block used to derive the first merge candidate is encoded based on affine motion information, a merge candidate derived from a neighboring block encoded based on the affine motion information may be set as the second merge candidate. For example, if the first merge candidate is an affine merge candidate, or if the first merge candidate is derived based on motion information of a sub-block within a coding block encoded based on affine motion information, the affine merge candidate or the merge candidate derived based on motion information of a sub-block within a coding block encoded based on affine motion information may be selected as the second merge candidate. Here, the second merge candidate may include at least one of a merge candidate having a search order closest to the first merge candidate, a merge candidate having a smallest index difference from the first merge candidate, a merge candidate having a smallest index, or a merge candidate having a smallest motion vector difference from the first merge candidate among the merge candidates satisfying the above conditions.

[0465] In contrast to the above example, if the neighboring block used to derive the first merge candidate is encoded based on translational motion information (i.e., non-affine motion information), the merge candidate derived from the neighboring block encoded based on translational motion information can be set as the second merge candidate. For example, if the first merge candidate is a non-affine merge candidate, the non-affine merge candidate can be selected as the second merge candidate. Here, the second merge candidate can include at least one of a merge candidate having a search order closest to the first merge candidate among the non-affine merge candidates, a merge candidate having a smallest index difference from the first merge candidate, a merge candidate having a smallest index, or a merge candidate having a smallest motion vector difference from the first merge candidate.

[0466] Alternatively, a second merge candidate may be derived by adding or subtracting an offset to the index of the first merge candidate, but if the encoding methods of the neighboring blocks used to derive the first merge candidate and the second merge candidate are different, the second merge candidate may be reset to a different merge candidate.

[0467] Figure 33 is a drawing for explaining an example of determining a second merge candidate.

[0468] Once the first merge candidate is selected, a merge candidate having an index derived by adding or subtracting an offset to the index of the first merge candidate can be selected as the second merge candidate. For example, a merge candidate having an index corresponding to the value indicated by merge_idx signaled from the bitstream can be selected as the first merge candidate, and a merge candidate having an index of merge_idx + 1 can be selected as the second merge candidate.

[0469] At this time, if the first merge candidate is derived from an affine merge candidate or a sub-block belonging to a coding block encoded with an affine motion model, the second merge candidate must also be set as an affine merge candidate or a merge candidate derived from a sub-block belonging to a coding block encoded with an affine motion model. If the merge candidate having merge_idx + 1 as an index is not an affine merge candidate or is not derived from a sub-block belonging to a coding block encoded with an affine motion model, another merge candidate may be set as the second merge candidate. Here, the other merge candidate may be a merge candidate having the smallest difference from the first merge candidate or merge_idx + 1 among the affine merge candidates or the merge candidates derived from sub-blocks belonging to a coding block encoded with an affine motion model.

[0470] Conversely, if the first merge candidate is a non-affine merge candidate, the second merge candidate must also be set as a non-affine merge candidate. If the merge candidate having merge_idx + 1 as an index is derived from an affine merge candidate or a sub-block belonging to a coding block encoded with an affine motion model, the other merge candidate may be set as the second merge candidate. Here, the other merge candidate may be the merge candidate having the smallest difference from the first merge candidate or merge_idx + 1 among the non-affine merge candidates. For example, in the example illustrated in FIG. 33, merge candidate A3 having merge_idx + 2 as an index is exemplified as being set as the second merge candidate.

[0471] As another example, motion compensation prediction can be performed by combining the translational merge mode and the affine merge mode. That is, either the first merge candidate or the second merge candidate can be an affine merge candidate, and the other can be a non-affine merge candidate.

[0472]

[0473] A combined prediction block can be obtained through a weighted sum operation of a first prediction block derived from a first merge candidate and a second prediction block derived based on a second merge candidate. At this time, the weight applied to the first prediction block can be set to a value greater than the weight applied to the second prediction block.

[0474] Alternatively, weights may be determined based on the motion information of the first merge candidate and the motion information of the second merge candidate. For example, the weights applied to the first prediction block and the second prediction block may be determined based on the difference in output order between the reference picture and the current picture. Specifically, the greater the difference in output order between the reference picture and the current picture, the smaller the weight applied to the prediction block may be set.

[0475] Alternatively, the weights applied to the first prediction block and the second prediction block may be determined by considering the size or shape of the candidate block used to derive the first merge candidate (hereinafter referred to as the first candidate block) and the candidate block used to derive the second merge candidate (hereinafter referred to as the second candidate block). For example, the weight applied to the prediction block derived from one of the first and second candidate blocks having a similar shape to the current block may be set to a large value. On the other hand, the weight applied to the prediction block derived from one having a dissimilar shape to the current block may be set to a small value.

[0476] Figure 34 is a diagram showing an example in which weights applied to prediction blocks are determined based on the shapes of candidate blocks.

[0477] Assume the current block is a non-square shape with a width greater than its height.

[0478] Based on the first merge candidate and the second merge candidate, a first prediction block and a second prediction block can be derived, and a combined prediction block can be generated based on a weighted sum operation of the first prediction block and the second prediction block. At this time, the weights applied to the first prediction block and the second prediction block can be determined based on the shapes of the first candidate block and the second candidate block.

[0479] For example, in the example illustrated in FIG. 34, the first candidate block is square, and the second candidate block is non-square, with a width greater than its height. Since the shape of the second candidate block is the same as that of the current block, the weight applied to the second prediction block can be set to have a larger value than the weight applied to the first prediction block. For example, a weight of 5 / 8 can be applied to the second prediction block, and a weight of 3 / 8 can be applied to the first prediction block. The following mathematical expression 14 shows an example of deriving a combined prediction block based on a weighted sum operation of the first prediction block and the second prediction block.

[0480]

[0481] P(x, y) represents a joint prediction block, P1(x, y) represents a first prediction block, and P2(x, y) represents a second prediction block.

[0482] As another example, based on the shape of the current block, the weights applied to the first prediction block and the second prediction block may be determined. For example, if the current block is a non-square shape in which the width is greater than the height, a greater weight may be applied to a prediction block generated based on a candidate block located above the current block among the first merge candidate and the second merge candidate. If both the first merge candidate and the second merge candidate are derived from candidate blocks located above, the weights applied to the first prediction block and the second prediction block may be set to be the same. On the other hand, if the current block is a non-square shape in which the height is greater than the width, a greater weight may be applied to a prediction block generated based on a candidate block located to the left of the current block among the first merge candidate and the second merge candidate. If both the first merge candidate and the second merge candidate are derived from candidate blocks located to the left, the weights applied to the first prediction block and the second prediction block may be set to be the same. If the current block is square, the weights applied to the first prediction block and the second prediction block can be set to be the same.

[0483] As another example, the weight applied to each prediction block can be determined based on the distance between the current block and the candidate block. Here, the distance can be derived based on the x-axis coordinate difference from the current block, the y-axis coordinate difference, or the minimum value therebetween. The weight applied to the prediction block derived from the merge candidate having a small distance from the current block can be set to have a larger value than the weight applied to the prediction block derived from the merge candidate having a large distance from the current block. For example, in the example illustrated in FIG. 31, the first merge candidate is derived from a neighboring block adjacent to the current block, and the second merge candidate is derived from a non-neighboring block that is not adjacent to the current block. In this case, since the x-axis distance between the first candidate block and the current block is smaller than the x-axis distance between the second candidate block and the current block, the weight applied to the first prediction block can be set to have a larger value than the weight applied to the second prediction block.

[0484] Alternatively, if both the first merge candidate and the second merge candidate are derived from non-neighboring blocks, a greater weight may be assigned to the prediction block derived from the non-neighboring block that is closer to the current block. For example, in the example illustrated in FIG. 32, since the y-axis distance between the first candidate block and the current block is smaller than the y-axis distance between the second candidate block and the current block, the weight applied to the first prediction block may be set to have a greater value than the weight applied to the second prediction block.

[0485] Integrated motion information can be derived based on the first and second merge candidates, and motion compensation prediction for the current block can also be performed based on the integrated motion information. For example, the motion vector of the current block can be derived based on an average operation or a weighted sum operation of the motion vector of the second merge candidate and the motion vector of the first merge candidate. In this case, the weight applied to the motion vector of the first merge candidate and the weight applied to the motion vector of the second merge candidate can be determined according to the embodiments described above.

[0486] If the first merge candidate is a non-affine merge candidate and the second affine merge candidate is an affine merge candidate, the motion vector of the current block can be derived by scaling the motion vector of the second merge candidate. Mathematical expression 15 illustrates an example of deriving the motion vector of the current block.

[0487]

[0488] In Equation 15, (mvX, mvY) represents the motion vector of the current block, (mv0x, mv0y) represents the motion vector of the first merge candidate, and (mv1x, mv1y) represents the motion vector of the second merge candidate. M represents a scaling parameter. M may be predefined in the encoder and decoder. Alternatively, the value of the scaling parameter M may be determined according to the size of the current block or candidate block. For example, when the width or height of the second candidate block is greater than 32, M may be set to 3, and otherwise, M may be set to 2.

[0489] Motion information of the first merge candidate or the second merge candidate can be stored as motion information of a block to which a combined prediction mode combining the merge mode and the merge mode is applied. The stored motion information can be used to derive motion information of a block to be encoded / decoded in the future.

[0490] Alternatively, the block may be divided into multiple sub-blocks, and either the motion information of the first merge candidate or the second candidate may be stored as the motion information of each sub-block. In this case, the motion information of some of the multiple sub-blocks may be set as the motion information of the first merge candidate, and the motion information of the remaining sub-blocks may be set as the motion information of the second merge candidate.

[0491] Alternatively, the integrated motion information derived based on the motion information of the first merge candidate and the second merge candidate can be stored as motion information of a block to which a combined prediction mode is applied, which is a combination of a merge mode and a merge mode.

[0492]

[0493] In a prediction mode that combines the merge mode and the motion vector prediction mode, a first prediction block can be generated using motion information derived from a merge candidate, and a second prediction block can be generated using a motion vector derived from a motion vector prediction candidate.

[0494] In motion vector prediction mode, a motion vector prediction candidate can be derived from a neighboring block adjacent to the current block or a collocated block within a collocated picture. Then, one of the multiple motion vector prediction candidates can be specified, and the specified motion vector prediction candidate can be set as the motion vector predictor of the current block. Then, the motion vector of the current block can be derived by adding the motion vector predictor of the current block and the motion vector differential.

[0495] In a prediction mode that combines merge mode and motion vector prediction mode, the merge candidate and the motion vector prediction candidate may be derived from the same candidate block. For example, if a merge candidate is specified by merge_idx, the motion vector of the candidate block used to derive the specified merge candidate may be set as the motion vector predictor. Alternatively, if a motion vector prediction candidate is specified by mvp_flag, a merge candidate derived from the candidate block used to derive the specified merge candidate may be selected.

[0496] Alternatively, the candidate block used to derive the merge candidate may be different from the candidate block used to derive the motion vector prediction candidate. For example, if a merge candidate derived from a candidate block located above the current block is selected, a motion vector prediction candidate derived from a candidate block located to the left of the current block may be set to be selected.

[0497] Alternatively, if the merge candidate selected by the index information and the motion vector prediction candidate selected by the index information are derived from the same candidate block, the motion vector prediction candidate may be replaced with a motion vector prediction candidate derived from a candidate block neighboring the candidate block, or the merge candidate may be replaced with a merge candidate derived from a candidate block neighboring the candidate block.

[0498] Figure 35 is a diagram showing an example of a merge candidate being replaced.

[0499] In the example illustrated in (a) of Fig. 35, the merge candidate and motion vector prediction candidate derived from the candidate block at position A2 are illustrated as being selected. As illustrated, when the merge candidate and the motion vector prediction candidate are derived from the same candidate block, the merge candidate or motion vector prediction candidate derived from a candidate block neighboring the candidate block may be used instead of the merge candidate or the motion vector prediction candidate. For example, as in the example illustrated in (b) of Fig. 35, the merge candidate at position A1 may be used instead of the merge candidate at position A2.

[0500] A first prediction block can be derived based on the merge candidate of the current block, and a second prediction block can be derived based on the motion vector prediction candidate. Thereafter, a combined prediction block can be derived through a weighted sum operation of the first and second prediction blocks. At this time, the weight applied to the second prediction block generated under motion vector prediction mode can be set to a value greater than the weight applied to the first prediction block generated under merge mode.

[0501]

[0502] A residual image can be derived by differentiating a predicted image from an original image. At this time, when the residual image is converted to the frequency domain, even if high-frequency components among the frequency components are removed, the subjective image quality of the image does not deteriorate significantly. Accordingly, if the values ​​of high-frequency components are converted to small values ​​or set to 0, there is an effect of increasing the compression efficiency without causing significant visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. The transformation can be performed using a transformation technique such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST).

[0503] DCT uses cosine transform to decompose (or transform) residual image into two-dimensional frequency components, and DST uses sine transform to decompose (or transform) residual image into two-dimensional frequency components. As a result of transforming residual image, frequency components can be expressed as basis images. For example, when performing DCT transform on a block of size NxN, N 2 The basic pattern components of the dog can be obtained. The size of each basic pattern component included in an NxN sized block can be obtained through transformation. Depending on the transformation technique used, the size of the basic pattern component can be called a DCT coefficient or a DST coefficient.

[0504] The DCT transform technique is primarily used to transform images with a high distribution of non-zero low-frequency components. The DST transform technique is primarily used for images with a high distribution of high-frequency components.

[0505] The residual image can also be transformed using a transform technique other than DCT or DST.

[0506] Hereinafter, the process of transforming a residual image into two-dimensional frequency components will be referred to as two-dimensional image transformation. In addition, the sizes of the basic pattern components obtained as a result of the transformation will be referred to as transformation coefficients. For example, the transformation coefficients may refer to DCT coefficients or DST coefficients. When both the first transformation and the second transformation described below are applied, the transformation coefficients may refer to the sizes of the basic pattern components generated as a result of the second transformation.

[0507] The transformation technique can be determined on a block-by-block basis. The transformation technique can be determined based on at least one of the prediction coding mode of the current block, the size of the current block, or the size of the current block. For example, if the current block is encoded in intra prediction mode and the size of the current block is smaller than NxN, the transformation can be performed using the DST transformation technique. On the other hand, if the above conditions are not met, the transformation can be performed using the DCT transformation technique.

[0508] 2D image transformation may not be performed on some blocks in the residual image. This process of not performing 2D image transformation is called transform skip. When transform skip is applied, quantization can be applied to residual values ​​that are not transformed.

[0509] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the transformation based on DCT or DST can be defined as the first transformation, and the transformation of the block to which the first transformation was applied can be defined as the second transformation.

[0510] The first transform may be performed using any one of a plurality of transform core candidates. For example, the first transform may be performed using any one of DCT2, DCT8, or DCT7.

[0511] Different transform cores may be used for the horizontal and vertical directions. Information indicating the combination of the horizontal transform core and the vertical transform core may be signaled via the bitstream.

[0512] The units in which the first and second transformations are performed may be different. For example, the first transformation may be performed on an 8x8 block, and the second transformation may be performed on a 4x4 sub-block of the transformed 8x8 block. In this case, the transformation coefficients of the remaining areas where the second transformation is not performed may be set to 0.

[0513] Alternatively, the first transformation can be performed on a 4x4 block, and the second transformation can be performed on an 8x8 sized area containing the transformed 4x4 block.

[0514] Information indicating whether a second transformation is performed can be signaled via the bitstream.

[0515] Alternatively, whether to perform the second transformation may be determined based on whether the horizontal transformation core and the vertical transformation core are the same. For example, the second transformation may be performed only when the horizontal transformation core and the vertical transformation core are the same. Alternatively, the second transformation may be performed only when the horizontal transformation core and the vertical transformation core are different.

[0516] Alternatively, the second transform may be allowed only when the horizontal transform and the vertical transform utilize a predefined transform core. For example, the second transform may be allowed when the DCT2 transform core is used for the horizontal transform and the vertical transform.

[0517] Alternatively, it may be determined whether to perform the second transform based on the number of non-zero transform coefficients of the current block. For example, if the non-zero transform coefficients of the current block are less than or equal to a threshold, the second transform may not be used, and if the non-zero transform coefficients of the current block are greater than the threshold, the second transform may be used. The second transform may also be used only if the current block is encoded using intra prediction.

[0518] Based on the shape of the current block, the size or shape of the sub-block on which the second transformation is to be performed can be determined.

[0519] Figures 36 and 37 are diagrams showing sub-blocks in which the second transformation is to be performed.

[0520] If the current block is square, after the first transformation is performed, a second transformation can be performed on a sub-block of size NxN at the upper left of the current block. For example, if the current block is a coding block of size 8x8, after performing the first transformation on the current block, a second transformation can be performed on a sub-block of size 4x4 at the upper left of the current block (see Fig. 36).

[0521] If the current block is a non-square shape with a width that is more than four times its height, then after the first transformation is performed, a second transformation can be performed on a sub-block of size (kN) x (4kN) at the upper left of the current block. For example, if the current block is a non-square shape with a size of 16x4, after performing the first transformation on the current block, a second transformation can be performed on a sub-block of size 2x8 at the upper left of the current block (see (a) of FIG. 37).

[0522] If the current block is a non-square shape with a height that is more than four times the width, then after the first transformation is performed, a second transformation can be performed on a sub-block of size (4kN)x(kN) at the upper left of the current block. For example, if the current block is a non-square shape with a size of 16x4, after performing the first transformation on the current block, the second transformation can be performed on a sub-block of size 2x8 at the upper left of the current block (see (b) of FIG. 37).

[0523] The decoder can perform the inverse transformation of the second transformation (second inverse transformation) and perform the inverse transformation of the first transformation (first inverse transformation) based on the result of the second inverse transformation. As a result of performing the second inverse transformation and the first inverse transformation, residual signals for the current block can be obtained.

[0524]

[0525] Information indicating the transformation type of the current block may be signaled via the bitstream. The information may be index information tu_mts_idx indicating one of combinations of a transformation type for the horizontal direction and a transformation type for the vertical direction.

[0526] Based on the transformation type candidates specified by the index information tu_mts_idx, a transformation core for the vertical direction and a transformation core for the horizontal direction can be determined. Tables 11 and 12 show combinations of transformation types according to tu_mts_idx.

[0527] tu_mts_idxtransform typehorizontalvertical0SKIPSKIP1DCT-IIDCT-II2DST-VIIDST-VII3DCT-VIIIDST-VII4DST-VIIDCT-VIII5DCT-VIIIDCT-VIII

[0528] tu_mts_idxtransform typehorizontalvertical0DCT-IIDCT-II1SKIPSKIP2DST-VIIDST-VII3DCT-VIIIDST-VII4DST-VIIDCT-VIII5DCT-VIIIDCT-VIII

[0529] The transform type can be determined as one of DCT2, DST7, DCT8, or transform skip. Alternatively, excluding transform skip, the transform type combination candidate can be constructed using only the transform cores.

[0530] When Table 11 is used, if tu_mts_idx is 0, transform skip can be applied to the horizontal and vertical directions. If tu_mts_idx is 1, DCT2 can be applied to the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 can be applied to the horizontal direction and DCT7 can be applied to the vertical direction.

[0531] When Table 12 is used, if tu_mts_idx is 0, DCT2 can be applied to the horizontal and vertical directions. If tu_mts_idx is 1, transform skip can be applied to the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 can be applied to the horizontal direction, and DCT7 can be applied to the vertical direction.

[0532] It is possible to determine whether to encode index information based on at least one of the size, shape, or number of non-zero coefficients of the current block. For example, if the number of non-zero coefficients is equal to or less than a threshold, the index information may not be signaled and a default transformation type may be applied to the current block. Here, the default transformation type may be DST7. Alternatively, the default mode may vary depending on the size, shape, or intra prediction mode of the current block.

[0533] The threshold value can be determined based on the size or shape of the current block. For example, if the size of the current block is less than or equal to 32x32, the threshold value can be set to 2, and if the size of the current block is greater than 32x32 (e.g., if the current block is a coding block of size 32x64 or 64x32), the threshold value can be set to 4.

[0534] Multiple lookup tables may be stored in the encoder / decoder. The multiple lookup tables may differ in at least one of the index values ​​assigned to the transformation type combination candidates, the types of the transformation type combination candidates, or the number of the transformation type combination candidates.

[0535] A lookup table for the current block can be selected based on at least one of the size, shape, prediction encoding mode, intra prediction mode, whether a secondary transform is applied to the current block, or whether a transform skip is applied to a neighboring block.

[0536] For example, if the size of the current block is 4x4 or less or if the current block is encoded with inter prediction, the lookup table of Table 11 can be used, and if the size of the current block is larger than 4x4 or if the current block is encoded with intra prediction, the lookup table of Table 12 can be used.

[0537] Alternatively, information pointing to one of multiple lookup tables can be signaled via the bitstream. The decoder can select a lookup table for the current block based on the information.

[0538] As another example, an index assigned to a candidate transform type combination may be adaptively determined based on at least one of the size of the current block, the shape, the prediction encoding mode, the intra prediction mode, whether a secondary transform is applied, or whether a transform skip is applied to a neighboring block. For example, when the size of the current block is 4x4, the index assigned to the transform skip may have a smaller value than the index assigned to the transform skip when the size of the current block is larger than 4x4. Specifically, when the size of the current block is 4x4, the index 0 may be assigned to the transform skip, and when the current block is larger than 4x4 and smaller than or equal to 16x16, an index greater than 0 (e.g., index 1) may be assigned to the transform skip. When the current block is larger than 16x16, the index of the transform skip may be assigned a maximum value (e.g., 5).

[0539] Alternatively, if the current block is encoded with inter prediction, an index of 0 can be assigned to the transform skip. If the current block is encoded with intra prediction, an index greater than 0 (e.g., index 1) can be assigned to the transform skip.

[0540] Alternatively, if the current block is a 4x4 sized block encoded with inter prediction, the transform skip may be assigned an index of 0. On the other hand, if the current block is not encoded with inter prediction or the current block is larger than 4x4, the transform skip may be assigned an index greater than 0 (e.g., index 1).

[0541] Transform type combination candidates different from the transformation type combination candidates listed in Tables 11 and 12 may be defined and used. For example, a transform type combination candidate may be used in which a transform skip is applied to one of the horizontal direction transformation or the vertical direction transformation, and a transform core such as DCT7, DCT8, or DST2 is applied to the other. In this case, whether to use a transform skip as a transform type candidate for the horizontal direction or the vertical direction may be determined based on at least one of the size (e.g., width and / or height), shape, prediction encoding mode, or intra prediction mode of the current block.

[0542] Alternatively, information indicating whether a specific transformation type candidate is available may be signaled via the bitstream. For example, a flag indicating whether transformation skip is available as a transformation type candidate for both horizontal and vertical directions may be signaled. Based on the flag, whether a specific transformation type combination candidate is included among multiple transformation type combination candidates may be determined.

[0543] Alternatively, whether a specific transform type candidate is applied to the current block can be signaled via the bitstream. For example, a flag cu_mts_flag indicating whether DCT2 is applied for the horizontal and vertical directions can be signaled. If the value of cu_mts_flag is 1, DCT2 can be set as the transform core for the vertical and horizontal directions. If the value of cu_mts_flag is 0, DCT8 or DST7 can be set as the transform core for the vertical and horizontal directions. Alternatively, if the value of cu_mts_flag is 0, information tu_mts_idx specifying one of multiple transform type combination candidates can be signaled.

[0544] If the current block is a non-square block with a width greater than its height or a non-square block with a height greater than its width, the encoding of cu_mts_flag may be omitted and the value of cu_mts_flag may be considered to be 0.

[0545] Depending on the size, shape, or intra prediction mode of the current block, the number of available transformation type combination candidates can be set differently. For example, if the current block is square, three or more transformation type combination candidates can be used, and if the current block is non-square, two transformation type combination candidates can be used. Alternatively, if the current block is square, only transformation type combination candidates having different transformation types for the horizontal direction and the vertical direction can be used.

[0546] If there are three or more transformation type combination candidates available to the current block, index information tu_mts_idx indicating one of the transformation type combination candidates can be signaled. On the other hand, if there are two transformation type combination candidates available to the current block, flag mts_flag indicating one of the transformation type combination candidates can be signaled. Table 13 below shows the encoding pattern of information for specifying transformation type combination candidates according to the form of the current block.

[0547] residual_coding( x0, y0, log2TbWidth, log2TbHeight, cIdx ) {Descriptor...if( cu_mts_flag[ && numSigCoeff > 2 ) | ( CuPredMode[ x0 ][ y0 ] = = MODE_INTER ) ) ) {if (cbHeight == cbWidth) {mts_idx[ x0 ][ y0 ]u(1)}}

[0548] Depending on the shape of the current block, the indices of the transformation type combination candidates can be rearranged (or reordered). For example, if the current block is square, the indices assigned to the transformation type combination candidates may be different from those assigned to the transformation type combination candidates if the current block is non-square. For example, if the current block is square, the transformation type combination may be selected based on Table 14 below, and if the current block is non-square, the transformation type combination may be selected based on Table 15 below.

[0549] mts_idxINTRAINTERmts_idxHorizontal conversion coreVertical conversion coreHorizontal conversion coreVertical conversion core0DST7DST7DCT8DCT81DCT8DST7DST7DCT82DST7DCT8DCT8DST73DCT8DCT8DST7DST7

[0550] mts_idxINTRAINTERmts_idxHorizontal conversion coreVertical conversion coreHorizontal conversion coreVertical conversion core0DCT8DST7DST7DCT81DST7DCT8DCT8DST72DST7DST7DST7DST73DCT8DCT8DST7DST7

[0551] The transformation type can be determined based on the number of horizontal non-zero coefficients or the number of vertical non-zero coefficients of the current block. Here, the number of horizontal non-zero coefficients represents the number of non-zero coefficients contained in 1xN (where N is the width of the current block), and the number of vertical non-zero coefficients represents the number of non-zero coefficients contained in Nx1 (where N is the height of the current block). When the maximum value of the horizontal non-zero coefficients is less than or equal to a threshold, the first transformation type can be applied to the horizontal direction, and when the maximum value of the horizontal non-zero coefficients is greater than the threshold, the second transformation type can be applied to the horizontal direction. When the maximum value of the vertical non-zero coefficients is less than or equal to a threshold, the first transformation type can be applied to the vertical direction, and when the maximum value of the vertical non-zero coefficients is greater than the threshold, the second transformation type can be applied to the vertical direction.

[0552] Figure 38 is a drawing for explaining an example in which the conversion type of the current block is determined.

[0553] For example, if the current block is encoded with intra prediction and the maximum value of the horizontal non-zero coefficient of the current block is 2 or less (see (a) of Fig. 38), the horizontal transformation type can be determined as DST7.

[0554] If the current block is encoded with intra prediction and the maximum value of the vertical non-zero coefficient of the current block is greater than 2 (see (b) of Fig. 38), DCT2 or DCT8 can be determined as the vertical direction transform type.

[0555]

[0556] Information indicating whether to explicitly determine the transform type of the current block based on information signaled from the bitstream may be signaled via the bitstream. For example, at the sequence level, information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for a block encoded with intra prediction and / or information sps_explicit_inter_mts_flag indicating whether explicit transform type determination is allowed for a block encoded with inter prediction may be signaled.

[0557] If explicit transformation type determination is allowed, the transformation type of the current block can be determined based on the index information tu_mts_idx signaled from the bitstream. On the other hand, if explicit transformation type determination is not allowed, the transformation type can be determined based on at least one of the size and shape of the current block, whether transformation in sub-block units is allowed, or the position of a sub-block containing a non-zero transformation coefficient. For example, the horizontal transformation type of the current block can be determined based on the width of the current block, and the vertical transformation type of the current block can be determined based on the height of the current block. For example, if the width of the current block is less than 4 or greater than 16, the horizontal transformation type can be determined as DCT2. Otherwise, the horizontal transformation type can be determined as DST7. If the height of the current block is less than 4 or greater than 16, the vertical transformation type can be determined as DCT2. Otherwise, the vertical transformation type can be determined as DST7. Here, to determine the transformation type in the horizontal direction and the transformation type in the vertical direction, the threshold value compared to the width and height can be determined based on at least one of the size, shape, or intra prediction mode of the current block.

[0558] Alternatively, if the current block is a square with the same height and width, the horizontal transformation type and the vertical transformation type can be set to the same value, while if the current block is a non-square with different heights and widths, the horizontal transformation type and the vertical transformation type can be set to different values. For example, if the width of the current block is greater than the height, the horizontal transformation type can be determined as DST7, and the vertical transformation type can be determined as DCT2. If the height of the current block is greater than the width, the vertical transformation type can be determined as DST7, and the horizontal transformation type can be determined as DCT2.

[0559] The number and / or types of transformation type candidates or the number and / or types of transformation type combination candidates may vary depending on whether explicit transformation type determination is allowed. For example, if explicit transformation type determination is allowed, DCT2, DST7, and DCT8 can be used as transformation type candidates. Accordingly, the horizontal transformation type and the vertical transformation type can be set to DCT2, DST8, or DCT8, respectively. If explicit transformation type determination is not allowed, only DCT2 and DST7 can be used as transformation type candidates. Accordingly, the horizontal transformation type and the vertical transformation type can be determined to DCT2 or DST7, respectively.

[0560] After the encoder performs transformation and quantization, the decoder can obtain a residual block through inverse quantization and inverse transformation. The decoder can then add the prediction block and residual block to obtain a reconstructed block for the current block.

[0561]

[0562] When a restoration block of the current block is obtained, information loss occurring during the quantization and encoding process can be reduced through in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the restoration block before the in-loop filter is applied is referred to as a first restoration block, and the restoration block after the in-loop filter is applied is referred to as a second restoration block.

[0563] A second restoration block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first restoration block. In this case, SAO or ALF can be applied after the deblocking filter is applied.

[0564] A deblocking filter is intended to mitigate image quality degradation (blocking artifacts) at block boundaries that occur when performing block-by-block quantization. To apply the deblocking filter, the blocking strength (BS) between the first reconstructed block and its neighboring reconstructed blocks can be determined.

[0565] Figure 39 is a flowchart showing the process of determining block strength.

[0566] In the example illustrated in Figure 39, P represents the first restoration block, and Q represents a neighboring restoration block. Here, the neighboring restoration block may be a neighbor to the left or above the current block.

[0567] In the example illustrated in Fig. 39, the block strength is determined by considering the predictive encoding mode of P and Q, whether non-zero transform coefficients are included, whether inter-prediction is performed using the same reference picture, or whether the difference value of motion vectors is greater than or equal to a threshold.

[0568] Based on the block strength, whether or not to apply a deblocking filter can be determined. For example, if the block strength is 0, filtering may not be performed.

[0569] SAO is intended to alleviate the ringing artifact that occurs when performing quantization in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. The offset determination method includes an edge offset (EO) or a band offset. EO refers to a method of determining the offset of the current sample according to the pattern of surrounding pixels. BO refers to a method of applying a common offset to a set of pixels with similar brightness values ​​within an area. Specifically, the pixel brightness can be divided into 32 equal sections, and pixels with similar brightness values ​​can be set as one set. For example, four adjacent bands among 32 bands can be set as one group, and the same offset value can be applied to samples belonging to the four bands.

[0570] ALF is a method of generating a second restored image by applying a filter of a predefined size / shape to a first restored image or a restored image to which a deblocking filter has been applied. The following mathematical expression 16 shows an example of applying ALF.

[0571]

[0572] Any one of the predefined filter candidates can be selected at the picture, coding tree unit, coding block, prediction block, or transform block level. Each filter candidate may differ in either size or shape.

[0573] Figure 40 shows predefined filter candidates.

[0574] As in the example illustrated in Figure 40, at least one of the diamond shapes of size 5x5, 7x7 or 9x9 can be selected.

[0575] For the chroma component, only a 5x5 sized diamond shape can be used.

[0576]

[0577] Applying the embodiments described above, focusing on the decoding or encoding process, to the encoding or decoding process is within the scope of the present invention. Changing the embodiments described above, in a given order, to a different order is also within the scope of the present invention.

[0578] Although the above-described embodiments have been described based on a series of steps or a flowchart, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each of the components (e.g., units, modules, etc.) constituting the block diagram in the above-described embodiments may be implemented as a hardware device or software, or a plurality of components may be combined to be implemented as a single hardware device or software. The above-described embodiments may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., singly or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands such as ROMs, RAMs, and flash memories. The above hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0579] The present invention can be applied to an electronic device that encodes / decodes an image.

Claims

1. A step for determining whether to apply the combined prediction mode to the current block; When a combined prediction mode is applied to the current block, a step of obtaining a first prediction block and a second prediction block for the current block, wherein the first prediction block is obtained based on motion information of a merge candidate of the current block, and the second prediction block is obtained based on an intra prediction mode of the current block; and An image decoding method, comprising a step of obtaining a third prediction block for the current block based on a weighted sum operation of the first prediction block and the second prediction block.

2. In paragraph 1, An image decoding method, characterized in that when the combined prediction mode is applied to the current block, application of triangular partitioning to the current block is not allowed.

3. In paragraph 1, A video decoding method, characterized in that the intra prediction mode of the current block is set to planar.

4. In paragraph 3, An image decoding method, characterized in that the second prediction block is obtained based on a reference sample line included in an adjacent reference sample line.

5. In paragraph 1, An image decoding method, characterized in that, in performing the weighted sum operation, the weights applied to the first prediction block and the second prediction block are determined based on the prediction encoding modes of neighboring blocks adjacent to the current block.

6. In paragraph 1, If at least one of the width or height of the current block is greater than the threshold, A video decoding method, characterized in that the above combined prediction mode is not applied to the current block.

7. In paragraph 1, A video decoding method, characterized in that the combined prediction mode is applicable to the current block when a flag indicating that the merge mode is applied to the current block is true.

8. A step for determining whether to apply the combined prediction mode to the current block; When a combined prediction mode is applied to the current block, a step of obtaining a first prediction block and a second prediction block for the current block, wherein the first prediction block is obtained based on motion information of a merge candidate of the current block, and the second prediction block is obtained based on an intra prediction mode of the current block; and A video encoding method, comprising a step of obtaining a third prediction block for the current block based on a weighted sum operation of the first prediction block and the second prediction block.

9. In paragraph 8, A video encoding method, characterized in that when the combined prediction mode is applied to the current block, application of triangular partitioning to the current block is not allowed.

10. In paragraph 8, A video encoding method, characterized in that the intra prediction mode of the current block is set to planar.

11. In paragraph 10, A video encoding method, characterized in that the second prediction block is obtained based on a reference sample line included in an adjacent reference sample line.

12. In paragraph 8, A video encoding method, characterized in that, in performing the above weighted sum operation, the weights applied to the first prediction block and the second prediction block are determined based on the prediction encoding modes of neighboring blocks adjacent to the current block.

13. In paragraph 8, If at least one of the width or height of the current block is greater than the threshold, A video encoding method, characterized in that the above combined prediction mode is not applied to the current block.

14. In paragraph 8, A video encoding method, characterized in that the combined prediction mode is applicable to the current block when a flag indicating that the merge mode is applied to the current block is true.

15. A prediction unit that determines whether to apply a combined prediction mode to the current block, and if the combined prediction mode is applied to the current block, obtains a first prediction block and a second prediction block for the current block, and obtains a third prediction block for the current block based on a weighted sum operation of the first prediction block and the second prediction block. An image decoding device, characterized in that the first prediction block is obtained based on motion information of a merge candidate of the current block, and the second prediction block is obtained based on an intra prediction mode of the current block.

Images