Video signal encoding / decoding method and device used for the method

Abstract

The video decoding method according to the present invention includes the following steps: determining whether to apply a combined prediction mode to a current block; obtaining a first prediction block and a second prediction block of the current block when the combined prediction mode is applied to the current block; and obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block.

Description

[0001] This application is a divisional application of an application filed on November 7, 2019, with application number 2019800706218 and invention name “Video signal encoding / decoding method and device for the method”. Technical Field

[0002] The present invention relates to a video signal encoding / decoding method and an apparatus used in the method. Background Art

[0003] As display panels continue to grow in size, there is a growing demand for video services with higher image quality. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, research is being actively conducted to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). The JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding) and was approved on January 25, 2013. Its compression performance is approximately twice that of H.264 / AVC. With the rapid development of high-definition video services, the performance of HEVC has gradually exposed its limitations. Summary of the Invention

[0004] Technical problems to be solved

[0005] An object of the present invention is to provide a combined prediction method for combining a plurality of prediction methods when encoding / decoding a video signal, and a device for executing the method.

[0006] An object of the present invention is to provide a method for dividing a coding block into a plurality of prediction units when encoding / decoding a video signal, and an apparatus for performing the method.

[0007] The technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and a person skilled in the art in the technical field to which the present invention belongs will clearly understand other technical problems not mentioned through the following description.

[0008] Technical Solution

[0009] The video signal decoding / encoding method according to the present invention includes the following steps: determining whether to apply a combined prediction mode to a current block; obtaining a first prediction block and a second prediction block of the current block when the combined prediction mode is applied to the current block; and obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block. In this case, the first prediction block can be obtained based on motion information of a merge candidate of the current block, and the second prediction block can be obtained based on an intra prediction mode of the current block.

[0010] In the video signal decoding / encoding method according to the present invention, when the combined prediction mode is applied to the current block, it may be configured to disable triangular partitioning for the current block.

[0011] In the video signal decoding / encoding method according to the present invention, the intra prediction mode of the current block may be set to a planar mode.

[0012] In the video signal decoding / encoding method according to the present invention, the second prediction block may be obtained based on a reference sample line included in adjacent reference sample lines.

[0013] In the video signal decoding / encoding method according to the present invention, when performing the weighted sum operation, weighted values applied to the first prediction block and the second prediction block may be determined based on prediction modes of neighboring blocks adjacent to the current block.

[0014] In the video signal decoding / encoding method according to the present invention, when at least one of a width or a height of the current block is greater than a threshold value, the combined prediction mode may not be applied to the current block.

[0015] In the video signal decoding / encoding method according to the present invention, when a flag indicating that the merge mode is applied to the current block is true, it may be set so that the combined prediction mode can be applied to the current block.

[0016] The features briefly summarized above are merely exemplary embodiments of the detailed description of the present invention to be described later and do not limit the scope of the present invention.

[0017] Effects of the Invention

[0018] According to the present invention, by providing a combined prediction method combining a plurality of prediction methods, inter-frame prediction efficiency can be improved.

[0019] According to the present invention, inter-frame prediction efficiency can be improved by providing a method of dividing a coding block into a plurality of prediction blocks and deriving motion information of each of the plurality of prediction blocks.

[0020] Effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those having ordinary skill in the technical field to which the present invention pertains through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0022] Figure 2 FIG. 4 is a block diagram of a video decoder according to an embodiment of the present invention.

[0023] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

[0024] Figure 4 is a diagram showing various types of partitioning of a coding block.

[0025] Figure 5 is a diagram showing an example of division of a coding tree unit.

[0026] Figure 6 is a diagram showing the shape of a basic data unit.

[0027] Figure 7 and Figure 8 is a diagram illustrating an example of dividing a coding block into a plurality of sub-blocks.

[0028] Figure 9 is a flowchart of an inter-frame prediction method according to an embodiment of the present invention.

[0029] Figure 10 is a flowchart of the process of deriving motion information of the current block in merge mode.

[0030] Figure 11 is a diagram illustrating an example of candidate blocks for deriving merge candidates.

[0031] Figure 12 is a diagram illustrating an example of dividing a coding block into a plurality of prediction units using diagonal lines.

[0032] Figure 13 is a diagram illustrating an example of dividing a coding block into two prediction units.

[0033] Figure 14 A diagram showing an example of dividing a coding block into a plurality of prediction blocks having different sizes.

[0034] Figure 15 is a diagram illustrating an example of applying diagonal partitioning to a transform unit.

[0035] Figure 16 is a diagram showing neighboring blocks used to derive triangle merging candidates.

[0036] Figure 17 is a diagram for describing an example of determining availability of neighboring blocks for each triangle prediction unit.

[0037] Figure 18 and Figure 19 is a diagram illustrating an example of deriving a prediction sample based on a weighted sum operation of a first prediction sample and a second prediction sample.

[0038] Figure 20 is a flowchart illustrating an intra-frame prediction method according to an embodiment of the present invention.

[0039] Figure 21 is a diagram showing intra prediction modes.

[0040] Figure 22 and Figure 23 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in a row.

[0041] Figure 24 is a diagram showing an angle formed between an angular intra prediction mode and a straight line parallel to the x-axis.

[0042] Figure 25 is a diagram illustrating an embodiment of obtaining prediction samples when the current block is non-square.

[0043] Figure 26 is a diagram showing the wide-angle intra prediction mode.

[0044] Figure 27 It is a diagram showing an embodiment to which PDPC is applied.

[0045] Figure 28 is a diagram showing an embodiment in which weighted values are applied.

[0046] Figure 29 is a diagram showing an example of assigning different weighting values to subblocks according to whether a merge candidate has bidirectional motion information.

[0047] Figure 30 is a diagram showing an example of assigning different weighting values to subblocks according to an intra prediction mode of a current block.

[0048] Figure 31 is a diagram illustrating an example of specifying a second merge candidate in consideration of the search order of candidate blocks.

[0049] Figure 32 is a diagram illustrating an example of selecting a first merge candidate and a second merge candidate from merge candidates derived from non-adjacent blocks.

[0050] Figure 33 is a diagram for describing an example of determining a second merge candidate.

[0051] Figure 34 is a diagram illustrating an example of determining a weight value applied to a prediction block based on the shape of a candidate block.

[0052] Figure 35 is a diagram illustrating an example of replacing a merge candidate.

[0053] Figure 36 and Figure 37 is a diagram showing sub-blocks on which secondary transform will be performed.

[0054] Figure 38 is a diagram for describing an example of determining a transform type of a current block.

[0055] Figure 39 is a flow chart illustrating a process for determining block strength.

[0056] Figure 40 is a diagram showing predefined filter candidates. DETAILED DESCRIPTION

[0057] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0058] Video encoding and decoding are performed on a block-by-block basis. For example, encoding / decoding processes such as transform, quantization, prediction, loop filtering, or reconstruction can be performed on a coding block, a transform block, or a prediction block.

[0059] Hereinafter, a block to be encoded / decoded is referred to as a “current block.” For example, depending on the current encoding / decoding process step, the current block may refer to a coding block, a transform block, or a prediction block.

[0060] In addition, the term "unit" used in this specification refers to a basic unit for performing a specific encoding / decoding process, and "block" can be understood to mean a sample array of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described later, encoding blocks and encoding units can be understood to have the same meaning.

[0061] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0062] Reference Figure 1 The video encoding apparatus 100 may include an image division unit 110 , prediction units 120 , 125 , a transformation unit 130 , a quantization unit 135 , a rearrangement unit 160 , an entropy encoding unit 165 , an inverse quantization unit 140 , an inverse transformation unit 145 , a filter unit 150 , and a memory 155 .

[0063] Figure 1The components shown are shown separately to illustrate distinct characteristic functions within the video encoding device and do not represent separate hardware or software components. Specifically, for ease of description, the components may be arranged such that at least two of the components are combined into a single component, or a single component is divided into multiple components, to perform the functions. Such integrated and separated components are also within the scope of the present invention, provided they do not depart from the essence of the present invention.

[0064] Furthermore, some structural elements are not essential structural elements for performing the essential functions of the present invention, but are optional structural elements used only to improve performance. The present invention can be implemented by including only components required to achieve the essence of the present invention, excluding structural elements used only to improve performance, and a structure including only essential structural elements excluding optional structural elements used only to improve performance also falls within the scope of the present invention.

[0065] The image division unit 110 may divide an input image into at least one processing unit. In this case, a processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image division unit 110 divides an image into a plurality of combinations of coding units, prediction units, and transform units, and may select a coding unit, prediction unit, and transform unit combination based on a predetermined criterion (e.g., a cost function) to encode the image.

[0066] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used. A video or the largest coding unit can be used as the root and the coding unit can be divided into other coding units with a number of child nodes equal to the number of divided coding units. Coding units that are no longer divided according to certain restrictions become leaf nodes. That is, when it is assumed that a coding unit can only be divided into squares, a coding unit can be divided into up to four other coding units.

[0067] Hereinafter, in the embodiments of the present invention, a coding unit may mean a unit that performs coding, and may also mean a unit that performs decoding.

[0068] The prediction units within a coding unit may be divided into at least one of squares or rectangles of the same size, or one prediction unit within a coding unit may be divided into a shape and / or size different from another prediction unit.

[0069] When a prediction unit on which intra prediction is performed based on a coding unit is not a minimum coding unit, intra prediction may be performed without being divided into a plurality of prediction units N×N.

[0070] The prediction units 120 and 125 may include an inter-frame prediction unit 120 for performing inter-frame prediction and an intra-frame prediction unit 125 for performing intra-frame prediction. Whether inter-frame prediction or intra-frame prediction is used for a prediction unit can be determined, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) can be determined based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode can be determined by the prediction unit, and the prediction can be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Furthermore, the prediction mode information, motion vector information, etc. used for the prediction can be encoded along with the residual value in the entropy coding unit 165 and transmitted to the decoder. When using a specific coding mode, the original block can also be directly encoded and transmitted to the decoder, without generating the prediction block through the prediction units 120 and 125.

[0071] The inter-frame prediction unit 120 can predict a prediction unit based on information about at least one image before or after the current image. In some cases, it can also predict a prediction unit based on information about a portion of an already encoded region within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0072] The reference image interpolation unit receives reference image information from the memory 155 and can generate pixel information for integer pixels or fractional pixels from the reference image. For luma pixels, to generate pixel information for fractional pixels in units of 1 / 4 pixels, an 8-tap DCT-based interpolation filter with different filter coefficients can be used. For chroma signals, to generate pixel information for fractional pixels in units of 1 / 8 pixels, a 4-tap DCT-based interpolation filter with different filter coefficients can be used.

[0073] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. The method for calculating the motion vector can use a full search-based Block Matching Algorithm (FBMA), a three-step search method (TSS), a new three-step search algorithm (NTS), and a variety of other methods. The motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. The current prediction unit can be predicted in the motion prediction unit by adopting different motion prediction methods. The motion prediction method can use a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, and a variety of other methods.

[0074] The intra-frame prediction unit 125 can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current image. If the neighboring block of the current prediction unit is an inter-frame predicted block and the reference pixel is an inter-frame predicted pixel, the reference pixel included in the inter-frame predicted block can be used as the reference pixel information for the surrounding intra-frame predicted block. In other words, if reference pixels are unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0075] In intra-frame prediction, the prediction mode can include an angular prediction mode that uses reference pixel information in the prediction direction and a non-angular mode that does not use directional information when performing prediction. The mode used to predict luma information and the mode used to predict chroma information can be different. To predict chroma information, the intra-frame prediction mode information used for luma prediction or the predicted luma signal information can be applied.

[0076] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixels to the left, the pixels to the upper left, and the pixels above the prediction unit. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning only for the minimum coding unit can be applied.

[0077] The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of the adaptive intra smoothing filter used for the reference pixels may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction modes of the prediction units existing in the periphery of the current prediction unit. In the case of predicting the prediction mode of the current prediction unit using the mode information predicted from the peripheral prediction units, if the intra prediction modes of the current prediction unit and the peripheral prediction units are the same, predetermined flag information may be used to transmit information indicating that the prediction modes of the current prediction unit and the peripheral prediction units are the same. If the prediction modes of the current prediction unit and the peripheral prediction units are different, the prediction mode information of the current block may be encoded by performing entropy coding.

[0078] Also, a residual block including residual information, which is a difference between a prediction unit predicted based on the prediction unit generated in the prediction units 120 and 125 and an original block of the prediction unit, may be generated. The generated residual block may be input to the transform unit 130.

[0079] In the transformation unit 130, a transformation method such as discrete cosine transform (DCT) or discrete sine transform (DST) can be used to transform the residual block, which includes residual information between the original block and the prediction unit generated by the prediction units 120 and 125. The DCT transform kernel includes at least one of DCT2 and DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST to the residual block can be determined based on the intra-prediction mode information of the prediction unit used to generate the residual block. Transformation of the residual block can also be skipped. A flag indicating whether to skip transforming the residual block can be encoded. Transform skipping can be allowed for residual blocks, luma components, or chroma components (under 4:4:4 format) whose size is below a threshold.

[0080] The quantization unit 135 may quantize the value transformed into the frequency domain in the transformation unit 130. The quantization coefficient may be changed according to the importance of the block or video. The value calculated in the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0081] The rearrangement part 160 may perform rearrangement of coefficient values on the quantized residual value.

[0082] The rearrangement unit 160 can convert the two-dimensional block-shaped coefficients into a one-dimensional vector form using a coefficient scanning method. For example, the rearrangement unit 160 can use a zigzag scanning method to scan the DC coefficients and even the coefficients in the high-frequency domain and convert them into a one-dimensional vector form. Depending on the size of the transform unit and the intra-frame prediction mode, instead of zigzag scanning, vertical scanning that scans the two-dimensional block-shaped coefficients along the column direction and horizontal scanning that scans the two-dimensional block-shaped coefficients along the row direction can also be used. In other words, the zigzag scanning, vertical scanning, and horizontal scanning method to be used can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0083] The entropy coding unit 165 may perform entropy coding based on the value calculated by the rearrangement unit 160. For example, the entropy coding may use various coding methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0084] The entropy coding unit 165 can encode various information such as residual value coefficient information and block type information, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc. of the coding units from the rearrangement unit 160 and the prediction units 120 and 125.

[0085] The coefficient values of the coding unit input from the rearrangement part 160 may be entropy-encoded at the entropy encoding part 165 .

[0086] The inverse quantization unit 140 and the inverse transformation unit 145 inversely quantize the multiple values quantized by the quantization unit 135 and inversely transform the values transformed by the transformation unit 130. The residual values generated by the inverse quantization unit 140 and the inverse transformation unit 145 can be combined with the prediction units predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0087] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0088] A deblocking filter removes block distortion in the reconstructed image caused by block boundaries. To determine whether to perform deblocking, the decision to apply the deblocking filter to the current block can be made based on the pixels in several columns or rows of the block. When a deblocking filter is applied to a block, a strong filter (strong filter) or a weak filter (weak filter) can be applied depending on the desired deblocking filter strength. Furthermore, when using a deblocking filter, when performing vertical and horizontal filtering, the horizontal and vertical filtering can be processed simultaneously.

[0089] The offset correction unit can correct the offset between the deblocked video and the original video on a pixel-by-pixel basis. To perform offset correction on a specific image, a method can be used, wherein the pixels included in the video are divided into a predetermined number of regions, the regions to be offset are determined, and the offset is applied to the corresponding regions, or the offset is applied while taking into account edge information of each pixel.

[0090] Adaptive loop filtering (ALF) can be performed based on the value of comparing the filtered reconstructed image with the original video. After dividing the pixels included in the video into predetermined groups, filtering can be performed differently for each group by determining a filter to be used for the corresponding group. Information related to whether to apply adaptive loop filtering and the luminance signal can be transmitted per coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied can be different for each block. In addition, it is also possible to apply the same type (fixed type) of adaptive loop filter regardless of the characteristics of the block to which it is applied.

[0091] The memory 155 may store the reconstructed block or image calculated by the filter unit 150 , and may provide the stored reconstructed block or image to the prediction units 120 and 125 when performing inter-frame prediction.

[0092] Figure 2 FIG. 4 is a block diagram of a video decoder according to an embodiment of the present invention.

[0093] Reference Figure 2 The video decoder 200 may include an entropy decoding unit 210 , a rearrangement unit 215 , an inverse quantization unit 220 , an inverse transformation unit 225 , a prediction unit 230 , a prediction unit 235 , a filter unit 240 , and a memory 245 .

[0094] When a video code stream is input from a video encoder, the input code stream can be decoded according to the reverse steps of the video encoder.

[0095] The entropy decoding unit 210 may perform entropy decoding in a manner opposite to the entropy encoding steps performed by the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied, corresponding to the method performed by the video encoder.

[0096] The entropy decoding unit 210 may decode information related to intra prediction and inter prediction performed by the encoder.

[0097] The rearrangement unit 215 can perform rearrangement based on the method used by the encoder to rearrange the code stream entropy-decoded by the entropy decoding unit 210. The rearrangement can be performed by reconstructing multiple coefficients represented by a one-dimensional vector form into two-dimensional block-shaped coefficients. The rearrangement unit 215 receives information related to coefficient scanning performed by the encoder and can perform rearrangement by performing a reverse scan based on the scanning order performed by the corresponding encoder.

[0098] The inverse quantization part 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient value of the rearranged block.

[0099] The inverse transform unit 225 can perform an inverse discrete cosine transform or an inverse discrete sine transform on the quantization result performed by the video encoder. The inverse discrete cosine transform and the inverse discrete sine transform are inverse transforms of the transform performed in the transform unit, that is, inverse transforms of the discrete cosine transform and the discrete sine transform. The DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform may not be performed in the inverse transform unit 225. The inverse transform may be performed based on the transmission unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, the transform method (e.g., DCT or DST) may be selectively performed based on multiple information such as the prediction method, the size of the current block, and the prediction direction.

[0100] The prediction units 230 and 235 can generate a prediction block based on information related to prediction block generation provided by the entropy decoding unit 210 and previously decoded block or image information provided by the memory 245 .

[0101] As described above, when intra prediction is performed in the same manner as in a video encoder, if the size of a prediction unit is the same as the size of a transform unit, intra prediction is performed on the prediction unit based on pixels to the left, pixels to the upper left, and pixels above the prediction unit. If the size of the prediction unit differs from the size of the transform unit when performing intra prediction, intra prediction is performed using reference pixels based on the transform unit. Furthermore, intra prediction using N×N partitioning only for the minimum coding unit can also be applied.

[0102] The prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit receives various information input from the entropy decoding unit 210, including prediction unit information, prediction mode information for the intra-frame prediction method, and motion prediction-related information for the inter-frame prediction method. It classifies the prediction unit based on the current coding unit and determines whether the prediction unit is performing inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 may use the information required for inter-frame prediction of the current prediction unit provided by the video encoder and perform inter-frame prediction on the current prediction unit based on information included in at least one of the previous or subsequent images of the current image to which the current prediction unit belongs. Alternatively, inter-frame prediction may also be performed based on information of a reconstructed portion of the current image to which the current prediction unit belongs.

[0103] In order to perform inter-frame prediction, it is possible to determine based on the coding unit which mode the motion prediction method of the prediction unit included in the corresponding coding unit is, among the skip mode, merge mode, advanced motion vector prediction mode (AMVP Mode), and intra-frame block copy mode.

[0104] The intra-frame prediction unit 235 can generate a prediction block based on pixel information within the current image. When the prediction unit is a prediction unit that has already performed intra-frame prediction, intra-frame prediction can be performed based on the intra-frame prediction mode information of the prediction unit provided by the video encoder. The intra-frame prediction unit 235 may include an adaptive intra-frame smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra-frame smoothing filter is a part that performs filtering on the reference pixels of the current block, and can determine whether to apply the filter based on the prediction mode of the current prediction unit. The prediction mode of the prediction unit provided by the video encoder and the adaptive intra-frame smoothing filter information can be used to perform adaptive intra-frame smoothing filtering on the reference pixels of the current block. If the prediction mode of the current block is a mode that does not perform adaptive intra-frame smoothing filtering, the adaptive intra-frame smoothing filter may not be applied.

[0105] For the reference pixel interpolation unit, if the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on the pixel values interpolated from the reference pixels, integer-valued or fractional-valued reference pixels can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. If the prediction mode of the current block is the DC mode, a DC filter can be used to generate a prediction block through filtering.

[0106] The reconstructed block or image may be provided to the filter part 240. The filter part 240 may include a deblocking filter, an offset correction part, and an ALF.

[0107] Information related to whether a deblocking filter is applied to a corresponding block or image and information related to whether strong filtering or weak filtering is applied when applying the deblocking filter can be received from the video encoder. Information related to the deblocking filter provided by the video encoder is received from the deblocking filter of the video decoder, and deblocking filtering can be performed on the corresponding block at the video decoder.

[0108] The offset correction unit can perform offset correction on the reconstructed video based on the type of offset correction used for the video during encoding, offset amount information, and the like.

[0109] ALF may be applied to a coding unit based on information related to whether ALF is applied, ALF coefficient information, etc. provided by an encoder. Such ALF information may be provided by being included in a specific parameter set.

[0110] The memory 245 stores the reconstructed image or block so that the image or block can be used as a reference image or reference block, and can provide the reconstructed image to the output section.

[0111] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

[0112] The largest coding block can be defined as a coding tree block. A picture can be divided into multiple coding tree units (CTUs). A coding tree unit is the largest coding unit and can also be called the largest coding unit (LCU). Figure 3 An example of dividing an image into a plurality of coding tree units is shown.

[0113] The size of the coding tree unit can be defined at the picture level or the sequence level. To this end, information indicating the size of the coding tree unit can be signaled through a picture parameter set or a sequence parameter set.

[0114] For example, the size of the coding tree unit of the entire image in the sequence may be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level may be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image may be set to 128×128, and the size of the coding tree unit in the second image may be set to 256×256.

[0115] The coding block can be generated by dividing the coding tree unit. The coding block represents the basic unit for performing the encoding / decoding process. For example, prediction or transformation can be performed according to different coding blocks, or the prediction mode can be determined according to different coding blocks. Among them, the prediction mode represents a method for generating a predicted image. For example, the prediction mode may include intra-frame prediction (IntraPrediction), inter-frame prediction (Inter Prediction), current picture referencing (CPR, or Intra Block Copy (IBC)) or combined prediction (Combined Prediction). For a coding block, at least one prediction mode of intra-frame prediction, inter-frame prediction, current image reference or combined prediction can be used to generate a prediction block related to the coding block.

[0116] Information indicating the prediction mode of the current block may be signaled via the codestream. For example, the information may be a 1-bit flag indicating whether the prediction mode is intra mode or inter mode. Current image reference or combined prediction may be used only when the prediction mode of the current block is determined to be inter mode.

[0117] The current image reference is used to set the current image as the reference image and obtain the prediction block for the current block from the encoded / decoded region within the current image. The current image refers to the image that includes the current block. Information indicating whether the current image reference is applied to the current block can be signaled via the codestream. For example, this information can be a 1-bit flag. When the flag is true, the prediction mode for the current block can be determined as current image reference; when the flag is false, the prediction mode for the current block can be determined as inter-frame prediction.

[0118] Alternatively, the prediction mode for the current block can be determined based on a reference image index. For example, when the reference image index points to the current image, the prediction mode for the current block can be determined as current image reference. When the reference image index points to another image other than the current image, the prediction mode for the current block can be determined as inter-frame prediction. In other words, current image reference is a prediction method that uses information from an already coded / decoded area within the current image, while inter-frame prediction is a prediction method that uses information from another already coded / decoded image.

[0119] Combined prediction refers to a coding mode that combines two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, when combined prediction is used, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or current image reference, and a second prediction block can be generated based on the other. If a first prediction block and a second prediction block are generated, a final prediction block can be generated by averaging or weighted summing the first and second prediction blocks. Information indicating whether combined prediction is used can be signaled via the codestream. This information can be a 1-bit flag.

[0120] Figure 4 is a diagram showing various types of partitioning of coding blocks.

[0121] The coding block may be divided into multiple coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. The divided coding block may be further divided into multiple coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0122] Quadtree partitioning refers to a partitioning technique that divides the current block into 4 blocks. As a result of quadtree partitioning, the current block can be divided into 4 square partitions (refer to Figure 4 'SPLIT_QT' in .

[0123] Binary tree partitioning is a technique for dividing the current block into two blocks. Vertical binary tree partitioning (i.e., using a vertical line that passes through the current block) is called vertical binary tree partitioning, while horizontal binary tree partitioning (i.e., using a horizontal line that passes through the current block) is called horizontal binary tree partitioning. Binary tree partitioning can be used to divide the current block into two non-square partitions. Figure 4 The "SPLIT_BT_VER" in the vertical direction represents the binary tree partition result, and Figure 4 The "SPLIT_BT_HOR" in the figure represents the result of horizontal binary tree partitioning.

[0124] Ternary tree partitioning is a technique for dividing the current block into three blocks. Vertical ternary tree partitioning (i.e., using two vertical lines that traverse the current block) is called vertical ternary tree partitioning, while horizontal ternary tree partitioning (i.e., using two horizontal lines that traverse the current block) is called horizontal ternary tree partitioning. After ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. Figure 4 The "SPLIT_TT_VER" in the vertical direction represents the ternary tree division result, and Figure 4 The "SPLIT_TT_HOR" in the figure indicates the horizontal ternary tree division result.

[0125] The number of times a coding tree unit is split can be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Thus, the maximum partitioning depth of a coding tree unit can be different for different sequences or pictures.

[0126] Alternatively, the maximum partition depth for each of the plurality of partitioning techniques may be determined separately. For example, the maximum partition depth allowed for quadtree partitioning may be different from the maximum partition depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0127] The encoder may signal information indicating at least one of a partition shape or a partition depth of the current block through a code stream, and the decoder may determine a partition shape and a partition depth of the coding tree unit based on the information parsed from the code stream.

[0128] Figure 5 is a diagram showing an example of division of a coding tree unit.

[0129] The process of partitioning a coding block using a partitioning technique such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning may be referred to as multi-tree partitioning.

[0130] The coding blocks generated by applying multi-tree partitioning to the coding block may be referred to as multiple downstream coding blocks. When the partition depth of the coding block is k, the partition depths of the multiple downstream coding blocks are set to k+1.

[0131] On the other hand, for a plurality of coding blocks split at a depth of k+1, the coding block split at a depth of k may be referred to as an upstream coding block.

[0132] The partition type of the current coding block may be determined based on at least one of a partition shape of an upstream coding block or a partition type of an adjacent coding block. The adjacent coding block is adjacent to the current coding block and may include at least one of an upper adjacent block, a left adjacent block, or a neighboring block adjacent to the upper left corner of the current coding block. The partition type may include at least one of whether to partition into a quadtree, whether to partition into a binary tree, a binary tree partition direction, whether to partition into a ternary tree, or a ternary tree partition direction.

[0133] In order to determine the division shape of the coding block, information indicating whether the coding block is divided can be sent by signaling in the code stream. The information is a 1-bit flag "split_cu_flag". When the flag is true, it indicates that the coding block is divided by the multi-tree division technique.

[0134] When "split_cu_flag" is true, information indicating whether the coding block is divided into quadtree blocks can be signaled through the codestream. The information is a 1-bit flag "split_qt_flag". When the flag is true, the coding block can be divided into 4 blocks.

[0135] For example, in Figure 5 The example shown illustrates that as the coding tree unit is quadtree-partitioned, four coding blocks with a partition depth of 1 are generated. Furthermore, quadtree partitioning is further applied to the first and fourth coding blocks of the four coding blocks generated as a result of the quadtree partitioning. Ultimately, four coding blocks with a partition depth of 2 are generated.

[0136] Also, a coding block having a split depth of 3 may be generated by applying quadtree partitioning again to the coding block having a split depth of 2.

[0137] When quadtree partitioning is not applied to a coding block, whether binary tree partitioning or ternary tree partitioning is performed on the coding block can be determined by considering at least one of the size of the coding block, whether the coding block is located at the image boundary, the maximum partition depth, or the partition shape of the adjacent blocks. When it is determined that binary tree partitioning or ternary tree partitioning is performed on the coding block, information indicating the partitioning direction can be signaled via the code stream. The information can be a 1-bit flag "mtt_split_cu_vertical_flag". Based on the flag, it can be determined whether the partitioning direction is vertical or horizontal. In addition, information indicating whether binary tree partitioning or ternary tree partitioning is applied to the coding block can be signaled via the code stream. The information can be a 1-bit flag "mtt_split_cu_binary_flag". Based on the flag, it can be determined whether binary tree partitioning or ternary tree partitioning is applied to the coding block.

[0138] For example, in Figure 5 The example shown illustrates the application of vertical binary tree partitioning to a coding block with a partition depth of 1, the application of vertical ternary tree partitioning to a left coding block in the coding block generated as a result of the partitioning, and the application of vertical binary tree partitioning to a right coding block.

[0139] When implementing a device for encoding or decoding video, there is a problem that it is difficult to process an area larger than a threshold due to hardware performance. For example, there may be a problem that when the hardware performance allows a maximum of 4096 samples to be processed simultaneously, a data unit of size 64×64 should be redundantly accessed and processed, and data in an area of more than 4096 samples cannot be processed simultaneously. As described above, the basic unit of data processing can be defined as a pipeline-based basic data unit (virtual processing data unit, VPDU, hereinafter referred to as a basic data unit).

[0140] The basic data units can be classified into square, non-square or non-rectangular types.

[0141] Figure 6 is a diagram showing the shape of a basic data unit.

[0142] A basic data unit may include samples equal to or less than the maximum number of samples that can be processed simultaneously. Figure 6 In the example shown in (a), a 64×64 square block can be set as the basic data unit. Alternatively, a non-square block can be set as the basic data unit. For example, as in Figure 6 In the examples shown in (b) or 6(c), a 32×128 block or a 64×32 block can be set as a basic data unit.

[0143] Although not shown, a basic data unit of a triangle, an L-shape, or a polygon may be defined.

[0144] Information for determining the basic data unit can be signaled via the code stream. The information can be used to determine at least one of a size or a shape of the basic data unit. Based on the information, it can be determined whether to allow non-square basic data units or to allow non-square basic data units.

[0145] Alternatively, at least one of the size or shape of the basic data unit may be predefined in the encoder and the decoder.

[0146] Whether the type of division of the coding block is allowed may be determined in consideration of the size of the basic data unit. For example, when the coding block generated as a result of dividing the coding block is larger than the basic data unit, such division may not be allowed. Alternatively, when the non-square coding block generated as a result of dividing the coding block is larger than the basic data unit, such division may not be allowed. For example, when the width or height of the coding block is greater than a threshold, or when the number of samples included in the coding block is greater than a threshold, binary tree or ternary tree division may not be allowed. Therefore, encoding of information related to binary tree or ternary tree division may be omitted.

[0147] Alternatively, it can be set so that coding blocks larger than the basic data unit must be divided. Alternatively, it can be set so that binary tree division or ternary tree division must be performed on coding blocks larger than the basic data unit. Therefore, for coding blocks larger than the basic data unit, although the flag split_flag indicating whether the coding block is divided is not encoded, the value of this flag can be derived as 1.

[0148] As another example, a coding block larger than a basic data unit can be divided into multiple sub-blocks. Here, the sub-block can be set as a prediction unit as a basic unit for prediction, or as a transform unit as a basic unit for transformation and / or quantization. In this case, dividing the coding block into multiple prediction units can be defined as VPDU prediction unit division, and dividing the coding block into multiple transform units can be defined as VPDU transform unit division.

[0149] At least one of VPDU prediction unit partitioning and VPDU transformation unit partitioning may be applied to the coding block. The partition type of the coding block according to applying VPDU prediction unit partitioning may be set to be the same as the partition type of the coding block according to applying VPDU transformation unit partitioning.

[0150] When only VPDU prediction unit partitioning is applied to a coding block, prediction is performed on each subblock, but transformation and / or quantization may be performed on the coding block. In this case, a prediction mode such as prediction mode, intra prediction mode, or inter prediction mode may be determined for the coding block.

[0151] When only VPDU transform unit partitioning is applied to a coding block, prediction is performed on subblocks, but transform and / or quantization may be performed on each subblock.

[0152] Figure 7 and Figure 8 is a diagram illustrating an example of dividing a coding block into a plurality of sub-blocks.

[0153] Figure 7 is a diagram showing a division pattern when only square basic data units are allowed, Figure 8 is a diagram showing a division pattern when a square basic data unit and a non-square basic data unit are allowed.

[0154] Assuming that only square basic data units are allowed, Figure 7 (a) and Figure 7 In (b), CU0 and CU2 are defined as two different VPDUs, and CU1 is defined as four different VPDUs. Therefore, CU0 and CU2 can be divided into two sub-blocks, and CU1 can be divided into four sub-blocks.

[0155] Assuming that square basic data units and non-square basic data units are allowed, Figure 8 (a) and Figure 8 In (b), CU0 and CU2 can be defined as one VPDU, and CU1 can be defined as using two different VPDUs. Therefore, CU0 and CU2 are not divided into sub-blocks, while CU1 can be divided into two sub-blocks.

[0156] In this case, CU1 can be divided into square sub-blocks or non-square sub-blocks. For example, based on the horizontal line that divides CU1 up and down, CU1 can be divided into two square sub-blocks. Alternatively, CU1 can be divided into two non-square sub-blocks based on the vertical line that divides CU1 left and right.

[0157] When there are multiple partition type candidates applicable to a coding block, information indicating any one of the multiple partition type candidates can be signaled through the codestream. For example, the information can indicate whether the coding block is divided into square sub-blocks or whether the coding block is divided into non-square sub-blocks.

[0158] Alternatively, dividing the coding block into square sub-blocks may be set to have a higher priority than dividing the coding block into non-square sub-blocks. For example, when the coding block cannot be divided into square sub-blocks, dividing the coding block into non-square sub-blocks may be allowed.

[0159] Alternatively, the partition type of the coding block can be determined based on the partition type of the parent coding block. For example, when the parent coding block is partitioned based on a ternary tree, the coding block can be divided into square sub-blocks. On the other hand, when the parent coding block is partitioned based on a binary tree or ternary tree, the coding block can be divided into non-square sub-blocks.

[0160] Inter-frame prediction refers to a prediction mode that uses information from a previous image to predict the current block. For example, a block in the previous image that is co-located with the current block (hereinafter referred to as a collocated block) can be set as the prediction block for the current block. Hereinafter, a prediction block generated based on a block co-located with the current block is referred to as a collocated prediction block.

[0161] On the other hand, if an object that existed in the previous image has moved to a different location in the current image, the object's motion can be used to effectively predict the current block. For example, if the direction and size of the object's movement can be known by comparing the previous image with the current image, the object's motion information can be used to generate a prediction block (or predicted image) for the current block. Hereinafter, a prediction block generated using motion information may be referred to as a motion prediction block.

[0162] A residual block can be generated by subtracting a prediction block from a current block. In this case, when there is motion of an object, the energy of the residual block can be reduced by using a motion prediction block instead of a co-located prediction block, thereby improving the compression performance of the residual block.

[0163] As described above, the process of generating a prediction block using motion information may be referred to as motion compensated prediction. In most inter-frame predictions, a prediction block may be generated based on motion compensated prediction.

[0164] Motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weighted value index. A motion vector indicates the direction and size of movement of an object. A reference image index specifies a reference image for the current block among multiple reference images included in a reference image list. The prediction direction refers to either unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of the L0 motion information or the L1 motion information may be used depending on the prediction direction of the current block. A bidirectional weighted value index specifies a weighting value for the L0 prediction block and a weighting value applied to the L1 prediction block.

[0165] Figure 9 is a flowchart of an inter-frame prediction method according to an embodiment of the present invention.

[0166] Reference Figure 9 , the inter-frame prediction method includes: determining an inter-frame prediction mode of a current block (S901); obtaining motion information of the current block according to the determined inter-frame prediction mode (S902); and performing motion compensation prediction of the current block based on the obtained motion information (S903).

[0167] Among them, the inter-frame prediction mode represents a variety of technologies used to determine the motion information of the current block, which may include an inter-frame prediction mode using translation motion information and an inter-frame prediction mode using affine motion information. For example, the inter-frame prediction mode using translation motion information may include a merge mode and an advanced motion vector prediction mode, and the inter-frame prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. Depending on the inter-frame prediction mode, the motion information of the current block may be determined based on neighboring blocks adjacent to the current block or information parsed from the code stream.

[0168] The motion information of the current block can be derived from the motion information of other blocks related to the current block. The other blocks may be blocks that are encoded / decoded using inter-frame prediction with higher priority than the current block. A merge mode may be defined in which the motion information of the current block is set to be the same as the motion information of the other blocks. Furthermore, a motion vector prediction mode may be defined in which the motion vectors of the other blocks are set to be the predicted value of the motion vector of the current block.

[0169] Figure 10 is a flowchart of the process of deriving motion information of the current block in merge mode.

[0170] A merging candidate for a current block may be derived ( S1001 ). The merging candidate for the current block may be derived from a block that was encoded / decoded using inter-prediction before the current block.

[0171] Figure 11 is a diagram illustrating an example of candidate blocks for deriving merge candidates.

[0172] The candidate blocks may include at least one of a neighboring block containing samples adjacent to the current block or a non-neighboring block containing samples not adjacent to the current block. Hereinafter, samples used to determine candidate blocks are referred to as reference samples. Furthermore, reference samples adjacent to the current block are referred to as neighboring reference samples, and reference samples not adjacent to the current block are referred to as non-neighboring reference samples.

[0173] The adjacent reference samples may be included in the adjacent columns of the leftmost column of the current block, or in the adjacent rows of the top row of the current block. For example, if the coordinates of the top left sample of the current block are (0, 0), at least one of a block including a reference sample at position (-1, H-1), a block including a reference sample at position (W-1, -1), a block including a reference sample at position (W, -1), a block including a reference sample at position (-1, H), or a block including a reference sample at position (-1, -1) may be used as a candidate block. Referring to the accompanying drawings, adjacent blocks indexed 0 to 4 may be used as candidate blocks.

[0174] Non-adjacent reference samples represent samples for which at least one of the x-axis distance and the y-axis distance from adjacent reference samples of the current block has a predefined value. For example, at least one of a block containing reference samples whose x-axis distance from a left reference sample is a predefined value, a block containing non-adjacent samples whose y-axis distance from an upper reference sample is a predefined value, or a block containing non-adjacent samples whose x-axis and y-axis distances from an upper-left reference sample are predefined values can be used as candidate blocks. The predefined value can be an integer such as 4, 8, 12, or 16. Referring to the accompanying drawings, at least one of the blocks indexed 5 to 26 can be used as a candidate block.

[0175] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block may be set as unavailable as merge candidates. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, the candidate block including the reference sample may be set as unavailable as merge candidates.

[0176] Merge candidates may also be derived from temporally adjacent blocks included in a different image than the current block. For example, merge candidates may be derived from co-located blocks included in a co-located image. Any one of the multiple reference images included in the reference image list may be set as a co-located image. Index information identifying the co-located image among the multiple reference images may be signaled via the codestream. Alternatively, a reference image with a predefined index among the multiple reference images may be determined as the co-located image.

[0177] The motion information of the merge candidate may be set to be the same as the motion information of the candidate block. For example, at least one of the motion vector, reference image index, prediction direction, or bidirectional weighted value index of the candidate block may be set as the motion information of the merge candidate.

[0178] A merge candidate list including merge candidates may be generated ( S1002 ).

[0179] The indexes of the plurality of merge candidates in the merge candidate list may be assigned in a predetermined order. For example, the indexes may be assigned in the order of the merge candidate derived from the left neighboring block, the merge candidate derived from the above neighboring block, the merge candidate derived from the above-right neighboring block, the merge candidate derived from the below-left neighboring block, the merge candidate derived from the above-left neighboring block, and the merge candidate derived from the temporal neighboring block.

[0180] When the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates may be selected (S1003). Specifically, information for specifying any one of the multiple merge candidates may be signaled via the codestream. For example, information merge_idx indicating an index of any one of the multiple merge candidates included in the merge candidate list may be signaled via the codestream.

[0181] When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter-frame motion information list may be added to the merge candidate list. The threshold may be the maximum number of merge candidates that can be included in the merge candidate list or the value of subtracting an offset from the maximum number of merge candidates. The offset may be an integer such as 1 or 2. The inter-frame motion information list may include merge candidates derived based on blocks encoded / decoded before the current block.

[0182] The inter-frame motion information list includes merge candidates derived from blocks encoded / decoded using inter-frame prediction within the current image. For example, the motion information of the merge candidates included in the inter-frame motion information list may be set to be the same as the motion information of the blocks encoded / decoded based on inter-frame prediction. The motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weighted value index.

[0183] The coding block may be divided into a plurality of prediction units, and prediction may be performed on each of the divided prediction units, where a prediction unit represents a basic unit for prediction.

[0184] The coding block may be divided using at least one of a vertical line, a horizontal line, a slant line, or a diagonal line. Information for determining at least one of the number, angle, or position of lines that divide the coding block may be signaled via the codestream. For example, information representing any one of the division type candidates for the coding block may be signaled via the codestream, or information specifying any one of a plurality of line candidates for dividing the coding block may be signaled via the codestream. Alternatively, information for determining the number or type of line candidates that divide the coding block may be signaled via the codestream. For example, using a 1-bit flag, it may be determined whether a slant line having an angle greater than a diagonal line and / or a slant line having an angle less than a diagonal line can be used as a line candidate.

[0185] Alternatively, at least one of the number, angle or position of lines dividing the coding block may be adaptively determined based on at least one of the intra prediction mode, inter prediction mode, position of available merge candidates or partition type of neighboring blocks of the coding block.

[0186] If a coding block is divided into multiple prediction units, intra prediction or inter prediction may be performed on each of the divided prediction units.

[0187] Figure 12 is a diagram illustrating an example of dividing a coding block into a plurality of prediction units using diagonal lines.

[0188] As in Figure 12 (a) and Figure 12 In the example shown in (b), the coding block can be divided into two triangular prediction units using diagonal lines.

[0189] exist Figure 12 (a) and Figure 12 In (b), the coding block is divided into two prediction units using a diagonal line connecting two vertices of the coding block. However, the coding block can be divided into two prediction units using an oblique line in which at least one end of the line does not pass through a vertex of the coding block.

[0190] Figure 13 is a diagram illustrating an example of dividing a coding block into two prediction units.

[0191] As in Figure 13 (a) and Figure 13 In the example shown in (b), the coding block may be divided into two prediction units using a slanted line whose ends touch the upper boundary and the lower boundary of the coding block, respectively.

[0192] Alternatively, as in Figure 13 (c) and Figure 13 In the example shown in (d), the coding block may be divided into two prediction units using an oblique line whose ends touch the left and right boundaries of the coding block, respectively.

[0193] Alternatively, the coding block can be divided into two prediction blocks of different sizes. For example, the oblique line dividing the coding block is set to contact two boundary surfaces forming a vertex, thereby dividing the coding block into two prediction units of different sizes.

[0194] Figure 14 A diagram showing an example of dividing a coding block into a plurality of prediction blocks having different sizes.

[0195] As in Figure 14 (a) and Figure 14 In the example shown in (b), the coding block can be divided into two prediction units with different sizes by setting the diagonal line connected at the upper left corner or lower right corner of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block instead of passing through the upper left corner or lower right corner of the coding block.

[0196] Alternatively, as in Figure 14 (c) and Figure 14 In the example shown in (d), the coding block can be divided into two prediction units with different sizes by setting the diagonal line connected at the upper right corner or lower left corner of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block instead of passing through the upper left corner or lower right corner of the coding block.

[0197] Each prediction unit generated by dividing the coding block is called an "Nth prediction unit". Figures 12 to 14 In the example shown, PU1 can be defined as the first prediction unit, and PU2 can be defined as the second prediction unit. The first prediction unit may refer to a prediction unit including samples located at the lower left or the upper left within the coding block, and the second prediction unit may refer to a prediction unit including samples located at the upper right or the lower right within the coding block.

[0198] In contrast to the above, a prediction unit including a sample located at the upper right or a sample located at the lower right within the coding block may be defined as a first prediction unit, and a prediction unit including a sample located at the lower left or a sample located at the upper left within the coding block may be defined as a second prediction unit.

[0199] The embodiments described later will primarily illustrate examples of partitioning using diagonal lines. In particular, the process of partitioning a coding block into two prediction units using diagonal lines is called diagonal partitioning or triangular partitioning, and the prediction units generated based on diagonal partitioning are called triangular prediction units. However, of course, partitioning using diagonal lines at angles other than vertical, horizontal, or diagonal lines may also be used in the embodiments described later.

[0200] Whether to apply diagonal splitting to the coding block can be determined based on at least one of a slice type, a maximum number of merge candidates that a merge candidate list can include, a size of the coding block, a shape of the coding block, a prediction mode of the coding block, or a split type of a parent node.

[0201] For example, whether to apply diagonal partitioning to the coding block may be determined based on whether the current slice is a B-slice. Diagonal partitioning is allowed only when the current slice is a B-slice.

[0202] Alternatively, whether to apply diagonal splitting to the coding block may be determined based on whether the maximum number of merge candidates included in the merge candidate list is greater than 2. Diagonal splitting is allowed only when the maximum number of merge candidates included in the merge candidate list is greater than 2.

[0203] Alternatively, when at least one of the width or height of the hardware is greater than 64, the disadvantage of redundantly accessing the 64×64 size data processing unit occurs. Thus, when at least one of the width or height of the coding block is greater than a threshold, it is possible that the coding block is not allowed to be divided into multiple prediction blocks. For example, when at least one of the height and width of the coding block is greater than 64 (for example, when at least one of the width and height is 128), diagonal division may not be used.

[0204] Alternatively, considering the maximum number of samples that can be processed simultaneously in hardware implementations, diagonal splits may not be allowed for coding blocks with a sample number greater than a threshold. For example, diagonal splits may not be allowed for coding tree blocks with a sample number greater than 4096.

[0205] Alternatively, for a coding block whose number of samples is less than a threshold, diagonal splitting may not be allowed. For example, when the number of samples is less than 64, diagonal splitting may not be applied to the coding block.

[0206] Alternatively, whether to apply diagonal splitting to the coding block can be determined based on whether the width-to-height ratio of the coding block is less than a first threshold or whether the width-to-height ratio of the coding block is greater than a second threshold. Here, the width-to-height ratio whRatio of the coding block can be determined as the ratio of the width CbW to the height CbH of the coding block, as shown in the following equation 1.

[0207] Equation 1

[0208] whRatio=CbW / CbH

[0209] The second threshold may be the reciprocal of the first threshold. For example, when the first threshold is k, the second threshold may be 1 / k.

[0210] Diagonal partitioning may be applied to the coding block only when the ratio of the width to the height of the coding block is between the first threshold and the second threshold.

[0211] Alternatively, triangular partitioning can be used only when the width-to-height ratio of the coding block is less than a first threshold or greater than a second threshold. For example, when the first threshold is 16, diagonal partitioning is not allowed for coding blocks of size 64×4 or 4×64.

[0212] Alternatively, whether diagonal splitting is allowed can be determined based on the partitioning type of the parent node. For example, when the coding block as the parent node is partitioned based on a quadtree, diagonal splitting can be applied to the coding block as a leaf node. On the other hand, when the coding block as the parent node is partitioned based on a binary tree or ternary tree partitioning, the coding block as a leaf node is set to not allow diagonal splitting.

[0213] Alternatively, whether diagonal splitting is allowed can be determined based on the prediction mode of the coding block. For example, when the coding block is encoded using intra prediction, diagonal splitting is only allowed when the coding block is encoded using inter prediction or when the coding block is encoded using a predefined inter prediction mode. The predefined inter prediction mode can be at least one of merge mode, advanced motion vector prediction mode, affine merge mode, or affine motion vector prediction mode.

[0214] Alternatively, whether diagonal partitioning is allowed may be determined based on the size of the parallel processing region. For example, when the size of the coding block is larger than the size of the parallel processing region, diagonal partitioning may not be used.

[0215] It is also possible to consider two or more of the listed conditions to determine whether to apply diagonal partitioning to the coding block.

[0216] As another example, information indicating whether diagonal partitioning is applied to a coding block can be signaled via the codestream. The information can be signaled at the sequence level, picture level, slice level, or block level. For example, a flag triangle_partition_flag indicating whether triangle partitioning is applied to the coding block can be signaled at the coding block level.

[0217] When it is determined that diagonal partitioning is to be applied to the coding block, information indicating the number of lines partitioning the coding block or the positions of the lines may be signaled through a code stream.

[0218] For example, when the coding block is divided by a diagonal line, information indicating the direction of the diagonal line dividing the coding block can be sent by a code stream signal. For example, a flag triangle_partition_type_flag indicating the direction of the diagonal line can be sent by a code stream signal. The flag indicates whether the coding block is divided by a diagonal line connecting the upper left and lower right or whether it is divided by a diagonal line connecting the upper right and lower left. When the coding block is divided by a diagonal line connecting the upper left and lower right, it can be called a left triangle partition type, and when the coding block is divided by a diagonal line connecting the upper right and lower left, it can be called a right triangle partition type. For example, a value of the flag is 0, indicating that the division type of the coding block is a left triangle division type, and a value of the flag is 1, indicating that the division type of the coding block is a right triangle division type.

[0219] In addition, information indicating whether the prediction units have the same size or information indicating the position of the diagonal line used to divide the coding block may be signaled via the code stream. For example, if the information indicating the size of the prediction unit indicates that the sizes of the prediction units are the same, encoding of the information indicating the position of the diagonal line is omitted, and the coding block may be divided into two prediction units using a diagonal line passing through two vertices of the coding block. On the other hand, when the information indicating the size of the prediction unit indicates that the sizes of the prediction units are not the same, the position of the diagonal line used to divide the coding block may be determined based on the information indicating the position of the diagonal line. For example, when a left triangle partitioning type is applied to a coding block, the position information may indicate whether the diagonal line touches the left and lower boundaries or the upper and right boundaries of the coding block. Alternatively, when a right triangle partitioning type is applied to a coding block, the position information may indicate whether the diagonal line touches the right and lower boundaries or the upper and left boundaries of the coding block.

[0220] The information indicating the partition type of the coding block can be signaled at the coding block level. Thus, the partition type can be determined for different coding blocks to which diagonal partitioning is applied.

[0221] As another example, information indicating the partition type may be signaled for a sequence, picture, slice, tile, or coding tree unit. In this case, the partition type of the coding block to which diagonal partitioning is applied may be set to be the same within the sequence, picture, slice, tile, or coding tree unit.

[0222] Alternatively, for the first coding unit to which diagonal partitioning is applied within the coding tree unit, information for determining the partitioning type is encoded and signaled, and the second and subsequent coding units to which diagonal partitioning is applied are set to use the same partitioning type as the first coding unit.

[0223] As another example, the partition type of the coding block can be determined based on the partition type of the adjacent blocks. The adjacent blocks may include at least one of the adjacent blocks adjacent to the upper left corner of the coding block, the adjacent blocks adjacent to the upper right corner, the adjacent blocks adjacent to the lower left corner, the adjacent blocks located above, or the adjacent blocks located on the left. For example, the partition type of the current block may be set the same as the partition type of the adjacent blocks. Alternatively, the partition type of the current block may be determined based on whether the upper left adjacent block applies the left triangle partition type, and whether the upper right adjacent block or the lower left adjacent block applies the right triangle partition type.

[0224] The size of the prediction unit can be set to be no larger than the size of the transform unit. In addition, it can be set to include one prediction unit in one transform unit. Therefore, when VPDU transform unit splitting is applied to the coding block, instead of applying diagonal splitting to the coding block, diagonal splitting can be applied to the transform unit.

[0225] Figure 15 is a diagram illustrating an example of applying diagonal partitioning to a transform unit.

[0226] As in Figure 15 In the example shown, when a coding block is divided into a plurality of transform units, it may be determined whether diagonal splitting is applied to each transform unit. Alternatively, it may be determined whether diagonal splitting is applied in units of blocks smaller than the size of the transform unit.

[0227] Alternatively, when VPDU transform unit partitioning is applied to a coding block, it is determined whether diagonal partitioning is applied to the coding block, and all transform units included in the coding block may be set to follow the determination. For example, when it is determined that diagonal partitioning is applied to the coding block, all transform units included in the coding block may be divided into two prediction units.

[0228] In order to perform motion prediction compensation on the first triangle prediction unit and the second triangle prediction unit, the motion information of each of the first triangle prediction unit and the second triangle prediction unit can be derived. In this case, the motion information of the first triangle prediction unit and the second triangle prediction unit can be derived from the merge candidates included in the merge candidate list. In order to distinguish between a general merge candidate list and a merge candidate list used when deriving the motion information of a triangle prediction unit, the merge candidate list used to derive the motion information of the triangle prediction unit is referred to as a triangle merge candidate list, and the merge candidates included in the triangle merge candidate list are referred to as triangle merge candidates. However, applying the aforementioned merge candidate derivation method and merge candidate list construction method to triangle merge candidates and the triangle merge candidate list construction method is also included in the spirit of the present invention.

[0229] Information for determining the maximum number of triangle merging candidates that can be included in the triangle merging candidate list can be signaled through the code stream. The information can represent the difference between the maximum number of merging candidates that can be included in the merging candidate list and the maximum number of triangle merging candidates that can be included in the triangle merging candidate list.

[0230] Triangle merging candidates may be derived from spatial and temporal neighboring blocks of a coding block.

[0231] Figure 16 is a diagram showing neighboring blocks used to derive triangle merging candidates.

[0232] Triangle merging candidates may be derived using at least one of a neighboring block located above the coding block, a neighboring block located to the left of the coding block, or a co-located block included in a different image from the coding block. The upper neighboring block may include at least one of a block including a sample (xCb+CbW-1, yCb-1) located above the coding block, a block including a sample (xCb+CbW, yCb-1) located above the coding block, or a block including a sample (xCb-1, yCb-1) located above the coding block. The left neighboring block may include at least one of a block including a sample (xCb-1, yCb+CbH-1) located to the left of the coding block or a block including a sample (xCb-1, yCb+CbH) located to the left of the coding block. The co-located block may be determined as either a block including samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coding block within the co-located image or a block including samples (xCb / 2, yCb / 2) located at the center of the coding block.

[0233] Neighboring blocks may be searched in a predefined order, and triangle merging candidates may be constructed into a triangle merging candidate list in a predefined order. For example, triangle merging candidates may be searched in the order of B1, A1, B0, A0, C0, B2, and C1 to construct a triangle merging candidate list.

[0234] The motion information of the triangle prediction units may be derived based on the triangle merging candidate list. That is, the triangle prediction units may share one triangle merging candidate list.

[0235] To derive motion information of a triangle merging unit, information specifying at least one of the triangle merging candidates included in the triangle merging candidate list may be signaled via a bitstream. For example, index information merge_triangle_idx specifying at least one of the triangle merging candidates may be signaled via a bitstream.

[0236] The index information may specify a combination of a merge candidate of the first triangle prediction unit and a merge candidate of the second triangle prediction unit. For example, Table 1 below shows an example of a combination of merge candidates according to the index information merge_triangle_idx.

[0237]

Table 1

[0238]

[0239]

[0240] The value of the index information merge_triangle_idx being 1 indicates that the motion information of the first triangle prediction unit is derived from the merge candidate of index 1, and the motion information of the second triangle prediction unit is derived from the merge candidate of index 0. The triangle merge candidate used to derive the motion information of the first triangle prediction unit and the triangle merge candidate used to derive the motion information of the second triangle prediction unit can be determined by the index information merge_triangle_idx.

[0241] The partition type of the coding block to which diagonal partitioning is applied can also be determined based on index information. That is, the index information can specify a combination of a merge candidate for the first triangle prediction unit, a merge candidate for the second triangle prediction unit, and the partition direction of the coding block. When the partition type of the coding block is determined based on the index information, the information triangle_partition_type_flag indicating the direction of the diagonal line that divides the coding block may not be encoded. Table 2 shows the partition type of the coding block with respect to the index information merge_triangle_idx.

[0242]

Table 2

[0243] merge_triangle_idx 0 1 2 3 4 5 6 7 8 TriangleDir 0 1 1 0 0 1 1 1 0 merge_triangle_idx 9 10 11 12 13 14 15 16 17 TriangleDir 0 0 0 1 0 0 0 0 1 merge_triangle_idx 18 19 20 21 22 23 24 25 26 TriangleDir 1 1 1 0 0 1 1 1 1 merge_triangle_idx 27 28 29 30 31 32 33 34 35 TriangleDir 1 1 1 0 0 1 0 1 0 merge_triangle_idx 36 37 38 39 TriangleDir 0 1 0 0

[0244] When the variable TriangleDir is 0, it indicates that the coding block applies the left triangle partition type. When the variable TriangleDir is 1, it indicates that the coding block applies the right triangle partition type. By combining Table 1 and Table 2, it can be set to specify the combination of the merge candidate of the first triangle prediction unit, the merge candidate of the second triangle prediction unit, and the partition direction of the coding block according to the index information merge_triangle_idx.

[0245] As another example, index information for only one of the first and second triangle prediction units may be signaled, and the index of a triangle merge candidate for the other of the first and second triangle prediction units may be determined based on the index information. For example, the triangle merge candidate for the first triangle prediction unit may be determined based on index information merge_triangle_idx indicating the index of any of the triangle merge candidates. Furthermore, the triangle merge candidate for the second triangle prediction unit may be specified based on merge_triangle_idx. For example, the triangle merge candidate for the second triangle prediction unit may be derived by adding or subtracting an offset from the index information merge_triangle_idx. The offset may be an integer such as 1 or 2. For example, the triangle merge candidate for the second triangle prediction unit may be determined as the triangle merge candidate indexed by 1 plus merge_triangle_idx. When merge_triangle_idx indicates the triangle merge candidate with the largest index value among the triangle merge candidates, motion information for the second triangle prediction unit may be derived from the triangle merge candidate indexed by 0 or the triangle merge candidate indexed by 1 minus merge_triangle_idx.

[0246] Alternatively, the motion information of the second triangle prediction unit may be derived from a triangle merging candidate having the same reference image as the triangle merging candidate of the first triangle prediction unit specified according to the index information. The triangle merging candidate having the same reference image as the triangle merging candidate of the first triangle prediction unit may represent at least one of an L0 reference image or an L1 reference image having the same reference image as the triangle merging candidate of the first triangle prediction unit. When there are multiple triangle merging candidates having the same reference image as the triangle merging candidate of the first triangle prediction unit, any one of them may be selected based on at least one of whether the merging candidate includes bidirectional motion information or a difference between the index of the merging candidate and the index information.

[0247] As another example, index information may be signaled for each of the first and second triangle prediction units. For example, first index information 1st_merge_idx for determining a triangle merge candidate for the first triangle prediction unit and second index information 2nd_merge_idx for determining a triangle merge candidate for the second triangle prediction unit may be signaled via a bitstream. Motion information for the first triangle prediction unit may be derived from the triangle merge candidate determined based on the first index information 1st_merge_idx, and motion information for the second triangle prediction unit may be derived from the triangle merge candidate determined based on the second index information 2nd_merge_idx.

[0248] The first index information 1st_merge_idx may indicate any index of the triangle merging candidates included in the triangle merging candidate list. The triangle merging candidate of the first triangle prediction unit may be determined as the triangle merging candidate indicated by the first index information 1st_merge_idx.

[0249] The triangle merge candidate indicated by the first index information 1st_merge_idx is set to be unavailable as a triangle merge candidate for the second triangle prediction unit. Thus, the second index information 2nd_merge_idx of the second triangle prediction unit may indicate the index of any one of the remaining triangle merge candidates other than the triangle merge candidate indicated by the first index information. When the value of the second index information 2nd_merge_idx is less than the value of the first index information 1st_merge_idx, the triangle merge candidate for the second triangle prediction unit may be determined to be a triangle merge candidate having the index information indicated by the second index information 2nd_merge_idx. On the other hand, when the value of the second index information 2nd_merge_idx is the same as or greater than the value of the first index information 1st_merge_idx, the triangle merge candidate for the second triangle prediction unit may be determined to be a triangle merge candidate having a value equal to or greater than the value of the first index information 1st_merge_idx, the triangle merge candidate for the second triangle prediction unit may be determined to be a triangle merge candidate having an index equal to the value of the second index information 2nd_merge_idx plus 1.

[0250] Alternatively, whether to signal the second index information may be determined based on the number of triangle merge candidates included in the triangle merge candidate list. For example, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is no more than 2, signaling the second index information may be omitted. When signaling the second index information is omitted, the first index information may be added to or subtracted from an offset to derive the second triangle merge candidate. For example, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information is index 0, the second triangle merge candidate may be derived by adding 1 to the first index information. Alternatively, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information is 1, the second triangle merge candidate may be derived by subtracting 1 from the first index information.

[0251] Alternatively, when the second index information is omitted from signaling, the second index information may be set to a default value. The default value may be 0. By comparing the first index information and the second index information, a second triangle merge candidate may be derived. For example, when the second index information is less than the first index information, the merge candidate with index 0 is set as the second triangle merge candidate. When the second index information is the same as or greater than the first index information, the merge candidate with index 1 is set as the second triangle merge candidate.

[0252] When a triangle merging candidate has unidirectional motion information, the unidirectional motion information of the triangle merging candidate is set as the motion information of the triangle prediction unit. On the other hand, when a triangle merging candidate has bidirectional motion information, only either the L0 motion information or the L1 motion information is set as the motion information of the triangle prediction unit. Which of the L0 motion information or the L1 motion information to obtain can be determined based on the index of the triangle merging candidate or the motion information of another triangle prediction unit.

[0253] For example, when the index of the triangle merging candidate is even, the L0 motion information of the triangle prediction unit is set to 0, and the L1 motion information of the triangle merging candidate is set to the L1 motion information of the triangle prediction unit. On the other hand, when the index of the triangle merging candidate is odd, the L1 motion information of the triangle prediction unit is set to 0, and the L0 motion information of the triangle merging candidate is set to 0. Conversely, when the index of the triangle merging candidate is even, the L0 motion information of the triangle merging candidate is set to the L0 motion information of the triangle prediction unit, and when the index of the triangle merging candidate is odd, the L1 motion information of the triangle merging candidate may also be set to the L1 motion information of the triangle prediction unit. Alternatively, for the first triangle prediction unit, when the triangle merging candidate is even, the L0 motion information of the triangle merging candidate is set to the L0 motion information of the first triangle prediction unit, and on the other hand, for the second triangle prediction unit, when the triangle merging candidate is odd, the L1 motion information of the triangle merging candidate is set to the L1 motion information of the second triangle prediction unit.

[0254] Alternatively, when the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit is set to 0, and the L1 motion information of the triangle merging candidate is set to the L1 information of the second triangle prediction unit. On the other hand, when the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit is set to 0, and the L0 motion information of the triangle merging candidate is set to the L0 motion signal of the second triangle prediction unit.

[0255] It is also possible to set the triangle merging candidate list used for deriving the motion information of the first triangle prediction unit and the triangle merging candidate list used for deriving the motion information of the second triangle prediction unit to be different.

[0256] For example, when a triangle merging candidate for deriving motion information of a first triangle prediction unit within a triangle merging candidate list is specified based on index information related to the first triangle prediction unit, the motion information of the second triangle prediction unit can be derived using the triangle merging list including the remaining triangle merging candidates other than the triangle merging candidate indicated by the index information. Specifically, the motion information of the second triangle prediction unit can be derived from any of the remaining triangle merging candidates.

[0257] Thus, the maximum number of triangle merging candidates included in the triangle merging candidate list of the first triangle prediction unit and the maximum number of triangle merging candidates included in the triangle merging candidate list of the second triangle prediction unit may be different. For example, when the triangle merging candidate list of the first triangle prediction unit includes M merging candidates, the triangle merging candidate list of the second triangle prediction unit may include M-1 merging candidates in addition to the triangle merging candidate indicated by the index information of the first triangle prediction unit.

[0258] As another example, merging candidates for each triangle prediction unit may be derived based on neighboring blocks adjacent to the coding block, and availability of the neighboring blocks may be determined by considering the shape or position of the triangle prediction unit.

[0259] Figure 17 is a diagram for describing an example of determining availability of neighboring blocks for each triangle prediction unit.

[0260] Neighboring blocks not adjacent to the first triangle prediction unit may be set as neighboring blocks unavailable to the first triangle prediction unit, and neighboring blocks not adjacent to the second triangle prediction unit may be set as neighboring blocks unavailable to the second triangle prediction unit.

[0261] For example, as in Figure 17 In the example shown in (a), when the left triangle partition type is applied to the coding block, it can be determined that blocks A1, A0, and A2 adjacent to the first triangle prediction unit among the neighboring blocks adjacent to the coding block are available for the first triangle prediction unit, while blocks B0 and B1 are not available for the first triangle prediction unit. Therefore, the triangle merging candidate list associated with the first triangle prediction unit includes triangle merging candidates derived from blocks A1, A0, and A2, but does not include triangle merging candidates derived from blocks B0 and B1.

[0262] As in Figure 17 In the example shown in (b), when the left triangle partition type is applied to the coding block, it can be determined that blocks B0 and B1 adjacent to the second triangle prediction unit are available for the second triangle prediction unit, while blocks A1, A0, and A2 are not available for the second triangle prediction unit. Therefore, the triangle merging candidate list associated with the second triangle prediction unit includes triangle merging candidates derived from blocks B0 and B1, but does not include triangle merging candidates derived from blocks A1, A0, and A2.

[0263] Thus, the number of triangle merging candidates or the range of triangle merging candidates that can be used by the triangle prediction unit may be determined based on at least one of the position of the triangle prediction unit or the partition type of the coding block.

[0264] As another example, the merge mode may be applied to only one of the first triangle prediction unit and the second triangle prediction unit. Furthermore, the motion information of the other of the first triangle prediction unit and the second triangle prediction unit may be set to be the same as the motion information of the triangle prediction unit to which the merge mode is applied, or the motion information of the triangle prediction unit to which the merge mode is applied may be refined to be derived.

[0265] For example, a motion vector and a reference image index of a first triangle prediction unit may be derived based on the triangle merging candidate. The motion vector of the first triangle prediction unit may be refined to derive a motion vector of a second triangle prediction unit. For example, the motion vector {mvD1LXx, mvD1LXy} of the first triangle prediction unit may be added to or subtracted from the refined motion vector {Rx, Ry} to derive the motion vector of the second triangle prediction unit. The reference image index of the second triangle prediction unit may be set to be the same as the reference image index of the first triangle prediction unit.

[0266] Information for determining a fine motion vector representing a difference between a motion vector of a first triangle prediction unit and a motion vector of a second triangle prediction unit may be signaled via a code stream, and the information may include at least one of information indicating a magnitude of the fine motion vector or information indicating a sign of the fine motion vector.

[0267] Alternatively, the sign of the fine motion vector may be derived based on at least one of the position, index, or partition type applied to the coding block of the triangle prediction unit.

[0268] As another example, a motion vector and a reference image index of any one of the first triangle prediction unit and the second triangle prediction unit may be signaled, and the other motion vector of the first triangle prediction unit and the second triangle prediction unit may be derived by refining the signaled motion vector.

[0269] For example, based on information signaled from the bitstream, the motion vector and reference image index of the first triangle prediction unit can be determined. Furthermore, the motion vector of the second triangle prediction unit can be derived by refining the motion vector of the first triangle prediction unit. For example, the motion vector of the second triangle prediction unit can be derived by adding or subtracting the refined motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangle prediction unit. The reference image index of the second triangle prediction unit can be set to be the same as the reference image index of the first triangle prediction unit.

[0270] Motion prediction compensation prediction can be performed on the coding block based on the motion information of the first triangle prediction unit and the motion information of the second triangle prediction unit. In this case, the boundary between the first triangle prediction unit and the second triangle prediction unit may experience image quality degradation. For example, the edge existing on the boundary between the first triangle prediction unit and the second triangle prediction unit may cause the continuity of the image quality to deteriorate. In order to reduce the image quality degradation at the boundary, the prediction samples can be derived through smoothing filtering or weighted prediction.

[0271] Prediction samples for applying diagonal partitioning within a coding block can be derived based on a weighted sum of a first prediction sample obtained based on motion information of the first triangular prediction unit and a second prediction sample obtained based on motion information of the second triangular prediction unit. Alternatively, the prediction samples of the first triangular prediction unit can be derived from a first prediction block determined based on motion information of the first triangular prediction unit, and the prediction samples of the second triangular prediction unit can be derived from a second prediction block determined based on motion information of the second triangular prediction unit. Furthermore, prediction samples located in a boundary region between the first triangular prediction unit and the second triangular prediction unit can be derived based on a weighted sum of the first prediction sample included in the first prediction block and the second prediction sample included in the second prediction block. For example, the following Equation 2 shows an example of deriving prediction samples for the first and second triangular prediction units.

[0272] Equation 2

[0273] P(x, y)=w1*P1(x, y)+(1-w1)*P2(x, y)

[0274] In Equation 2, P1 represents the first prediction sample, and P2 represents the second prediction sample. w1 represents the weighting value applied to the first prediction sample, and (1-w1) represents the weighting value applied to the second prediction sample. As shown in Equation 2, the weighting value applied to the second prediction sample can be derived by subtracting the weighting value applied to the first prediction sample from a constant.

[0275] When the left triangle partition type is applied to the coding block, the boundary area may include prediction samples having the same x-axis coordinate and y-axis coordinate. On the other hand, when the right triangle partition type is applied to the coding block, the boundary area may include prediction samples whose sum of the x-axis coordinate and the y-axis coordinate is greater than a first threshold and less than a second threshold.

[0276] The size of the boundary area may be determined based on at least one of the size of the coding block, the shape of the coding block, motion information of the triangle prediction unit, a motion vector difference value of the triangle prediction unit, an output order of the reference image, or a difference value between a first prediction sample and a second prediction sample in a diagonal boundary.

[0277] Figure 18 and Figure 19 is a diagram illustrating an example of deriving a prediction sample based on a weighted sum operation of a first prediction sample and a second prediction sample. Figure 18 An example of applying a left triangle partitioning type to a coding block is shown, and Figure 19 An example of applying the right triangle partitioning type to the coding block is shown. Figure 18 (a) and Figure 19 (a) is a diagram showing a prediction pattern of a luminance component, and Figure 18 (b) and Figure 19 (b) is a diagram showing a prediction pattern of a chrominance component.

[0278] In the illustrated diagram, the number written in the prediction sample near the boundary between the first prediction unit and the second prediction unit represents the weighting value applied to the first prediction sample. For example, when the number written in the prediction sample is N, a weighting value of N / 8 is applied to the first prediction sample, and a weighting value of (1-(N / 8)) is applied to the second prediction sample, thereby deriving the prediction sample.

[0279] In the non-boundary area, the first prediction sample or the second prediction sample can be determined as the prediction sample. Figure 18 In an example, in an area belonging to the first triangle prediction unit where the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, a first prediction sample derived based on the motion information of the first triangle prediction unit may be determined as a prediction sample. On the other hand, in an area belonging to the second triangle prediction unit where the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is greater than the threshold, a second prediction sample derived based on the motion information of the second triangle prediction unit may be determined as a prediction sample.

[0280] Reference Figure 19 In an example, in an area where the sum of the x-axis coordinate and the y-axis coordinate is less than a first threshold, a first prediction sample derived based on the motion information of the first triangle prediction unit may be determined as the prediction sample. On the other hand, in an area where the sum of the x-axis coordinate and the y-axis coordinate is greater than a second threshold, a second prediction sample derived based on the motion information of the second triangle prediction unit may be determined as the prediction sample.

[0281] The threshold for determining the non-boundary area may be determined based on at least one of the size of the coding block, the shape of the coding block, or the color component. For example, when the threshold associated with the luma component is set to N, the threshold associated with the chroma component may be set to N / 2.

[0282] The prediction samples included in the boundary region may be derived based on a weighted sum operation of the first prediction sample and the second prediction sample. In this case, the weighted values applied to the first prediction sample and the second prediction sample may be determined based on at least one of the position of the prediction sample, the size of the coding block, the shape of the coding block, or the color component.

[0283] For example, as in Figure 18 In the example shown in (a), the same weighting value can be applied to the first prediction sample and the second prediction sample to derive a prediction sample at a position having the same x-axis coordinate and y-axis coordinate. The weighting value ratio applied to the first prediction sample and the second prediction sample can be set to (3:1) or (1:3) to derive a prediction sample whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1. Furthermore, the weighting value ratio applied to the first prediction sample and the second prediction sample can be set to (7:1) or (1:7) to derive a prediction sample whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 2.

[0284] Alternatively, as Figure 18 In the example shown in (b), a predicted sample at a position having the same x-axis coordinate and y-axis coordinate can be derived by applying the same weighting value to the first predicted sample and the second predicted sample, and a predicted sample having an absolute value of the difference between the x-axis coordinate and the y-axis coordinate of 1 can be derived by setting the weighting value ratio applied to the first predicted sample and the second predicted sample to (7:1) or (1:7).

[0285] For example, as in Figure 19 In the example shown in (a), a prediction sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived by applying the same weighting value to the first prediction sample and the second prediction sample. The weighting value ratio applied to the first prediction sample and the second prediction sample can be set to (3:1) or (1:3) to derive a prediction sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as or 2 less than the width or height of the coding block. The weighting value ratio applied to the first prediction sample and the second prediction sample can be set to (7:1) or (1:7) to derive a prediction sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 greater than or 3 less than the width or height of the coding block.

[0286] Alternatively, as in Figure 19 In the example shown in (b), a prediction sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived by applying the same weighting value to the first prediction sample and the second prediction sample. The ratio of the weighting values applied to the first prediction sample and the second prediction sample can be set to (7:1) or (1:7) to derive a prediction sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as or 2 less than the width or height of the coding block.

[0287] As another example, the weighting value may be determined taking into account the position of the prediction sample or the shape of the coding block. Equations 3 to 5 show examples of deriving weighting values when the left triangle partition type is applied to the coding block. Equation 3 shows an example of deriving a weighting value applied to the first prediction sample when the coding block is a square.

[0288] Equation 3

[0289] w1=(x-y+4) / 8

[0290] In Equation 3, x and y represent the position of the prediction sample. When the coding block is non-square, the weighted value applied to the first prediction sample can be derived as shown in Equation 4 or 5. Equation 4 shows the case where the width of the coding block is greater than the height, and Equation 5 shows the case where the width of the coding block is less than the height.

[0291] Equation 4

[0292] w1=((x / whRatio)-y+4) / 8

[0293] Equation 5

[0294] w1=(x-(y*whRatio)+4) / 8

[0295] When the right triangle partition type is applied to the coding block, the weighted value applied to the first prediction sample may be determined as shown in Equations 6 to 8. Equation 6 shows an example of deriving the weighted value applied to the first prediction sample when the coding block is a square.

[0296] Equation 6

[0297] w1=(CbW-1-xy)+4) / 8

[0298] In Equation 6, CbW represents the width of the coding block. When the coding block is non-square, the weighting value applied to the first prediction sample can be derived as shown in Equation 7 or 8. Equation 7 shows the case where the width of the coding block is greater than the height, and Equation 8 shows the case where the width of the coding block is less than the height.

[0299] Equation 7

[0300] w1=(CbH-1-(x / whRatio)-y)+4) / 8

[0301] Equation 8

[0302] w1=(CbW-1-x-(y*whRatio)+4) / 8

[0303] In Equation 7, CbH represents the height of the coding block.

[0304] As shown in the example, for the prediction samples within the boundary area, the samples included in the first triangular prediction unit can be derived by assigning a larger weight value to the first prediction sample than to the second prediction sample, and the samples included in the second triangular prediction unit can be derived by assigning a larger weight value to the second prediction sample than to the first prediction sample.

[0305] When diagonal partitioning is applied to a coding block, the coding block may be set to a combined prediction mode that does not apply a combination of intra prediction mode and merge mode.

[0306] Intra prediction uses reconstructed samples from the surrounding coded / decoded blocks to predict the current block. In this case, intra prediction of the current block can use reconstructed samples before applying the loop filter.

[0307] Intra-frame prediction techniques include matrix-based intra-frame prediction and general intra-frame prediction that takes into account directionality with surrounding reconstructed samples. Information indicating the intra-frame prediction technique for the current block can be signaled via the codestream. The information can be a 1-bit flag. Alternatively, the intra-frame prediction technique for the current block can be determined based on at least one of the position, size, shape, or intra-frame prediction techniques of neighboring blocks. For example, when the current block exists across a picture boundary, the current block is set to not apply matrix-based intra-frame prediction.

[0308] Matrix-based intra prediction is a method that obtains a prediction block for the current block by performing matrix multiplication between matrices stored in the encoder and decoder and reconstructed samples around the current block. Information specifying any one of multiple stored matrices can be signaled via the bitstream. The decoder can determine the matrix to use for intra prediction of the current block based on this information and the size of the current block.

[0309] General intra prediction is a method of obtaining a prediction block related to a current block based on a non-angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in more detail with reference to the accompanying drawings.

[0310] Figure 20 is a flowchart illustrating an intra-frame prediction method according to an embodiment of the present invention.

[0311] A reference sample line for the current block may be determined (S2001). A reference sample line refers to a set of reference samples included in a K-th row offset from above and / or to the left of the current block. Reference samples may be derived from reconstructed samples that have been encoded / decoded around the current block.

[0312] Index information identifying a reference sample line of a current block among a plurality of reference sample lines may be signaled via a code stream. For example, index information intra_luma_ref_idx for specifying a reference sample line of the current block may be signaled via a code stream. The index information may be signaled per coding block.

[0313] The multiple reference sample lines may include at least one of a first line, a second line, a third line, or a fourth line above and / or to the left of the current block. Among the multiple reference sample lines, reference sample lines consisting of rows adjacent to the top of the current block and columns adjacent to the left of the current block may be referred to as adjacent reference sample lines, and reference sample lines other than these may be referred to as non-adjacent reference sample lines.

[0314] Only some of the multiple reference sample lines can be selected as reference sample lines for the current block. For example, the remaining reference sample lines in the multiple reference sample lines except the third non-adjacent reference sample line can be set as candidate reference sample lines. Table 3 shows the index assigned to each candidate reference sample line.

[0315]

Table 3

[0316] Index (intra_luma_ref_idx) Reference sample line 0 Adjacent reference sample lines 1 First non-adjacent reference sample line 2 Second non-adjacent reference sample line

[0317] More candidate reference sample lines than the sample lines described above may be set, or fewer candidate reference sample lines may be set. Furthermore, the number or positions of non-adjacent reference sample lines set as candidate reference sample lines are not limited to the examples described. For example, the first non-adjacent reference sample line and the third non-adjacent reference sample line may be set as candidate reference sample lines, or the second non-adjacent reference sample line and the third non-adjacent reference sample line may be set as candidate reference sample lines. Alternatively, the first non-adjacent reference sample line, the second non-adjacent reference sample line, and the third non-adjacent reference sample line may all be set as candidate reference sample lines.

[0318] The number or type of candidate reference sample lines may be determined based on at least one of the size, shape, position, sub-block partitioning, or intra prediction mode of the current block.

[0319] The reference sample line of the current block may also be determined based on at least one of the position, size, shape, or prediction mode of the current block or a neighboring block. For example, when the current block touches a boundary of an image, tile, slice, or coding tree unit, the first reference sample line may be determined as the reference sample line of the current block.

[0320] Alternatively, when the current block is non-square, the adjacent reference sample line may be set as the reference sample line of the current block. Alternatively, when the width-to-height ratio of the current block is above or below a threshold, the adjacent reference sample line may be determined as the reference sample line of the current block.

[0321] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper reference sample and the left reference sample may be derived from reconstructed samples surrounding the current block. The reconstructed samples may be in a state before a loop filter is applied.

[0322] Next, an intra prediction mode for the current block may be determined (S2002). For the intra prediction mode of the current block, at least one of a non-angular intra prediction mode or an angular intra prediction mode may be determined as the intra prediction mode for the current block. The non-angular intra prediction modes include planar and DC, and the angular intra prediction modes include 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

[0323] Figure 21 is a diagram showing intra prediction modes.

[0324] Figure 21 (a) shows 35 intra prediction modes, and Figure 21 (b) shows 67 intra prediction modes.

[0325] Can be defined with Figure 21 A greater or lesser number of intra prediction modes than shown.

[0326] The most probable mode (MPM) may be set based on the intra prediction modes of neighboring blocks adjacent to the current block, where the neighboring blocks may include a left neighboring block adjacent to the left side of the current block and an upper neighboring block adjacent to the top of the current block.

[0327] The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs can be signaled via the codestream. Alternatively, the number of MPMs can be determined based on at least one of the prediction mode of the adjacent block, the size of the current block, or the shape of the current block. For example, N MPMs are used when an adjacent reference sample line is determined as the reference sample line of the current block, and M MPMs can be used when a non-adjacent reference sample line is determined as the reference sample line of the current block. M is an integer less than N, for example, N can be 6, and M can be 5, 4, or 3. Thus, when the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra-frame prediction mode of the current block can be determined as any one of the 6 candidate intra-frame prediction modes, and when the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra-frame prediction mode of the current block can be determined as any one of the 5 candidate intra-frame prediction modes.

[0328] Alternatively, a fixed number (eg, 6 or 5) of MPM candidates may also be used regardless of the index of the reference sample line of the current block.

[0329] An MPM list including multiple MPMs may be generated, and information indicating whether the MPM identical to the intra-frame prediction mode of the current block is included in the MPM list may be signaled via the codestream. The information is a 1-bit flag, which may be referred to as an MPM flag. When the MPM flag indicates that the MPM identical to the current block is included in the MPM list, index information identifying one of the MPMs may be signaled via the codestream. For example, index information mpm_idx specifying any one of the multiple MPMs may be signaled via the codestream. The MPM specified by the index information may be set as the intra-frame prediction mode of the current block. When the MPM flag indicates that the MPM identical to the current block is not included in the MPM list, remaining mode information indicating any one of the remaining intra-frame prediction modes other than the MPM may be signaled via the codestream. The remaining mode information represents an index value corresponding to the intra-frame prediction mode of the current block when reassigning indices to the remaining intra-frame prediction modes other than the MPM. The decoder may arrange the MPMs in ascending order and determine the intra-frame prediction mode of the current block by comparing the remaining mode information with the MPM. For example, when the remaining mode information is the same as or smaller than the MPM, the intra prediction mode of the current block may be derived by adding 1 to the remaining mode information.

[0330] When deriving the intra prediction mode of the current block, some of the comparisons with the remaining mode information in the MPM can be omitted. For example, the MPM of the non-angle intra prediction mode can be excluded from the comparison object. When the non-angle intra prediction mode is set to MPM, since the remaining mode information clearly indicates the angle intra prediction mode, the intra prediction mode of the current block can be derived by comparing the remaining MPMs other than the non-angle intra prediction mode with the remaining mode information. Instead of excluding the non-angle intra prediction mode from the comparison object, the result value after adding the number of non-angle intra prediction modes to the remaining mode information can be compared with the MPM.

[0331] Instead of setting the default mode to MPM, information indicating whether the intra-frame prediction mode of the current block is the default mode may be signaled through the code stream. The information is a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be signaled only when the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode may include at least one of plane, DC, vertical direction mode, or horizontal direction mode. For example, when plane is set as the default mode, the default mode flag may indicate whether the intra-frame prediction mode of the current block is plane. When the default mode flag indicates that the intra-frame prediction mode of the current block is not the default mode, one of the MPMs indicated by means of the index information may be set as the intra-frame prediction mode of the current block.

[0332] When the default mode flag is used, the intra prediction mode that is the same as the default mode may not be set as the MPM. For example, when the default mode flag indicates whether the intra prediction mode of the current block is planar, the intra prediction mode of the current block may be derived by using 5 MPMs other than the MPM equivalent to planar.

[0333] When multiple intra prediction modes are set as default modes, index information indicating any one of the default modes may be further signaled. The intra prediction mode of the current block may be set to the default mode indicated by the index information.

[0334] When the index of the reference sample line of the current block is not 0, the default mode is not used. For example, when a non-adjacent reference sample line is determined as the reference sample line of the current block, a non-angular intra prediction mode such as DC mode or planar mode may not be used. Thus, when the index of the reference sample line is not 0, the default mode flag is not signaled, and the value of the default mode flag may be set to a predefined value (i.e., false).

[0335] When the intra prediction mode of the current block is determined, a prediction sample of the current block may be obtained based on the determined intra prediction mode ( S2003 ).

[0336] When DC mode is selected, prediction samples associated with the current block are generated based on the average of the reference samples. Specifically, the values of all samples within the prediction block are generated based on the average of the reference samples. The average value can be derived using at least one of an upper reference sample located above the current block and a left reference sample located to the left of the current block.

[0337] Depending on the shape of the current block, the number or range of reference samples used when deriving the average value may vary. For example, when the current block is non-square with a width greater than its height, only the upper reference samples may be used to calculate the average value. On the other hand, when the current block is non-square with a width less than its height, only the left reference samples may be used to calculate the average value. In other words, when the width and height of the current block are different, only the reference samples adjacent to the longer side may be used to calculate the average value. Alternatively, whether to use only the upper reference samples or only the left reference samples to calculate the average value may be determined based on the ratio of the current block's width to its height.

[0338] When planar mode is selected, prediction samples are obtained using horizontal and vertical prediction samples. The horizontal prediction sample is obtained based on left and right reference samples located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on upper and lower reference samples located on the same vertical line as the prediction sample. The right reference sample can be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample can be generated by copying the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample can be obtained based on a weighted sum of the left and right reference samples, and the vertical prediction sample can be obtained based on a weighted sum of the upper and lower reference samples. In this case, the weight assigned to each reference sample can be determined based on the position of the prediction sample. The prediction sample can be obtained based on an average or weighted sum of the horizontal and vertical prediction samples. When performing the weighted sum, the weight assigned to the horizontal and vertical prediction samples can be determined based on the position of the prediction sample.

[0339] When the angular prediction mode is selected, a parameter representing the prediction direction (or prediction angle) of the selected angular prediction mode may be determined. Table 4 below shows an intra-prediction parameter intraPredAng for each intra-prediction mode.

[0340]

Table 4

[0341]

[0342]

[0343] Table 4 shows the intra direction parameters of each intra prediction mode with an index of any one of 2 to 34 when 35 intra prediction modes are defined. When more than 33 angular intra prediction modes are defined, Table 4 is further subdivided to set the intra direction parameters of each angular intra prediction mode.

[0344] After arranging the upper reference samples and the left reference samples of the current block in a column, the prediction sample can be obtained based on the value of the intra-frame direction parameter. In this case, when the value of the intra-frame direction parameter is negative, the left reference samples and the upper reference samples can be arranged in a column.

[0345] Figure 22 and Figure 23 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in a row.

[0346] Figure 22 An example of a vertical one-dimensional array in which reference samples are arranged in the vertical direction is shown, and Figure 23 An example of a horizontal one-dimensional array in which reference samples are arranged in the horizontal direction is shown. The following description will be made under the assumption that 35 intra prediction modes are defined. Figure 22 and 23 Example of .

[0347] When the intra prediction mode index is any one of 11 to 18, a horizontal one-dimensional arrangement of the upper reference sample rotated counterclockwise can be applied. When the intra prediction mode index is any one of 19 to 25, a vertical one-dimensional arrangement of the left reference sample rotated clockwise can be applied. When arranging the reference samples in a column, the intra prediction mode angle can be taken into account.

[0348] The reference sample determination parameters may be determined based on the intra direction parameters. The reference sample determination parameters may include a reference sample index for specifying a reference sample and a weighting value parameter for determining a weighting value applied to the reference sample.

[0349] The reference sample index iIdx and the weight value parameter ifact can be obtained by the following equations 9 and 10, respectively.

[0350] Equation 9

[0351] iIdx=(y+1)*P ang / 32

[0352] Equation 10

[0353] i fact =[(y+1)*P ang ]&31

[0354] In equations 9 and 10, P ang Indicates the intra-frame direction parameter. The reference sample specified by the reference sample index iIdx is equivalent to an integer pixel (Integer pel).

[0355] To derive prediction samples, you can specify one or more reference samples. Specifically, considering the slope of the prediction mode, you can specify the position of the reference samples used when deriving the prediction samples. For example, using the reference sample index iIdx, you can specify the reference samples used when deriving the prediction samples.

[0356] In this case, when the slope of the intra-frame prediction mode is not represented by a single reference sample, the prediction sample can be generated by interpolating multiple reference samples. For example, when the slope of the intra-frame prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample can be obtained by interpolating the first reference sample and the second reference sample. In other words, when the angular line (AngularLine) following the intra-frame prediction angle does not pass through the reference sample located at an integer pixel, the prediction sample can be obtained by interpolating the reference samples adjacent to the left, right, or above and below the position where the angular line passes.

[0357] The following Equation 11 shows an example of obtaining a predicted sample based on a reference sample.

[0358] Equation 11

[0359] P(x, y) = ((32-i fact ) / 32)*Ref_1D(x+iIdx+1)+(i fact / 32)*Ref_1D(x+iIdx+2)

[0360] In Equation 11, P represents a prediction sample, and Ref_1D represents any one of the reference samples arranged in one dimension. In this case, the position of the reference sample can be determined according to the position (x, y) of the prediction sample and the reference sample index iIdx.

[0361] When the slope of the intra prediction mode can be represented as a reference sample, the weighting parameter i fact is set to 0. Therefore, Equation 11 can be simplified as shown in the following Equation 12.

[0362] Equation 12

[0363] P(x, y) = Ref_1D(x+iIdx+1)

[0364] Intra-frame prediction may also be performed on the current block based on multiple intra-frame prediction modes. For example, intra-frame prediction modes may be derived for different prediction samples, and prediction samples may be derived based on the intra-frame prediction modes assigned to the respective prediction samples.

[0365] Alternatively, intra-frame prediction modes may be derived for different regions, and intra-frame prediction may be performed on each region based on the intra-frame prediction mode assigned to each region. The region may include at least one sample. At least one of the size or shape of the region may be adaptively determined based on at least one of the size or shape of the current block, or the intra-frame prediction mode. Alternatively, at least one of the size or shape of the region may be predefined in the encoder and decoder, regardless of the size or shape of the current block.

[0366] Alternatively, intra-frame predictions may be performed based on multiple intra-frame predictions, and a final prediction sample may be derived based on an average operation or a weighted sum operation of the multiple prediction samples obtained through the multiple intra-frame predictions. For example, intra-frame predictions may be performed based on a first intra-frame prediction mode to obtain a first prediction sample, and intra-frame predictions may be performed based on a second intra-frame prediction mode to obtain a second prediction sample. Subsequently, a final prediction sample may be obtained based on an average operation or a weighted sum operation between the first prediction sample and the second prediction sample. In this case, the weight values assigned to the first prediction sample and the second prediction sample may be determined based on at least one of whether the first intra-frame prediction mode is a non-angle / angle prediction mode, whether the second intra-frame prediction mode is a non-angle / angle prediction mode, or the intra-frame prediction mode of a neighboring block.

[0367] The multiple intra prediction modes may be a combination of a non-angular intra prediction mode and an angular prediction mode, a combination of angular prediction modes, or a combination of non-angular prediction modes.

[0368] Figure 24 is a diagram showing an angle formed between an angular intra prediction mode and a straight line parallel to the x-axis.

[0369] like Figure 24 In the example shown, the angular prediction mode can exist between the lower left diagonal direction and the upper right diagonal direction. When described as the angle formed by the x-axis and the angular prediction mode, the angular prediction mode can exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0370] In the case where the current block is non-square in shape, the following occurs: according to the intra prediction mode of the current block, a reference sample farther away from the prediction sample is used rather than a reference sample closer to the prediction sample among reference samples located on a corner line following the intra prediction angle to derive the prediction sample.

[0371] Figure 25 is a diagram illustrating an embodiment of obtaining prediction samples when the current block is non-square.

[0372] For example, as in Figure 25In the example shown in (a), it is assumed that the current block is a non-square block with a width greater than a height, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between 0 and 45 degrees. In this case, when deriving a prediction sample A near the right column of the current block, a left reference sample L farther from the prediction sample among the reference samples in the angular mode is used instead of an upper reference sample T closer to the prediction sample.

[0373] As another example, in Figure 25 In the example shown in (b), it is assumed that the current block is a non-square block with a height greater than its width, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between -90 degrees and -135 degrees. In this case, when deriving a prediction sample A near the lower row of the current block, an upper reference sample T farther from the prediction sample among the reference samples in the angular mode is used instead of a left reference sample L closer to the prediction sample.

[0374] In order to solve the above problem, when the current block is non-square, the intra prediction mode of the current block can be replaced with the intra prediction mode in the opposite direction. Figure 21 The angle prediction mode shown is an angle prediction mode with a larger or smaller angle. This angular intra prediction mode can be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode refers to an angular intra prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0375] Figure 26 is a diagram showing the wide-angle intra prediction mode.

[0376] exist Figure 26 In the illustrated example, the intra prediction modes with indexes of -1 to -14 and the intra prediction modes with indexes of 67 to 80 represent the wide-angle intra prediction mode.

[0377] Despite Figure 26 , 14 wide-angle intra prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) with angles less than -135 degrees are shown, but a greater or lesser number of wide-angle intra prediction modes may be defined.

[0378] When the wide-angle intra prediction mode is used, the length of the upper reference sample is set to 2W+1, and the length of the left reference sample is set to 2H+1.

[0379] When using wide-angle intra prediction mode, the reference sample T can be used to predict Figure 25 (a) shows sample A, and can be predicted using reference sample L Figure 25(b) Sample A shown.

[0380] By adding the existing intra prediction mode and N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 5 shows the intra direction parameters of the intra prediction mode when 20 wide-angle intra prediction modes are defined.

[0381]

Table 5

[0382] PredModeIntra -10 -9 -8 -7 -6 -5 -4 -3 -2 intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -1 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 11 12 13 14 15 16 17 18 intraPredAngle 13 11 9 7 5 3 2 1 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -1 -2 -3 -5 -7 -9 -11 -13 -15 PredModeIntra 28 29 30 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -11 -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -1 0 1 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 11 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

[0383] When the current block is non-square and the intra prediction mode of the current block obtained in step S2002 falls within the transformation range, the intra prediction mode of the current block may be transformed into the wide-angle intra prediction mode. The transformation range may be determined based on at least one of the size, shape, or ratio of the current block. The ratio may represent the ratio between the width and height of the current block.

[0384] When the current block is non-square and has a width greater than a height, the transform range may be set from the intra prediction mode index in the upper right diagonal direction (e.g., 66) to (the index of the intra prediction mode in the upper right diagonal direction - N). N may be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transform range, the intra prediction mode may be transformed into the wide-angle intra prediction mode. The transform may be performed by subtracting a predefined value from the intra prediction mode, which may be the total number of intra prediction modes excluding the wide-angle intra prediction mode (e.g., 67).

[0385] According to the embodiment, the intra prediction modes between the 66th and 53rd may be transformed into wide-angle intra prediction modes between the -1st and -14th, respectively.

[0386] When the current block is a non-square block with a height greater than a width, the transformation range may be set from the intra prediction mode index in the lower left diagonal direction (e.g., 2) to (the index of the intra prediction mode in the lower left diagonal direction + M). M may be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transformation range, the intra prediction mode may be transformed into the wide-angle intra prediction mode. The transformation may be performed by adding a predefined value to the intra prediction mode, which may be the total number of angular intra prediction modes excluding the wide-angle intra prediction mode (e.g., 65).

[0387] According to the embodiment, the intra prediction modes between the 2nd to the 15th are transformed into the wide-angle intra prediction modes between the 67th to the 80th, respectively.

[0388] Hereinafter, the intra prediction mode falling within the transformation range is referred to as a wide-angle intra replacement prediction mode.

[0389] The transform range may be determined based on the ratio of the current block. For example, Table 6 and Table 7 each show a transform range when 35 intra prediction modes and 67 intra prediction modes other than the wide-angle intra prediction mode are defined.

[0390]

Table 6

[0391] condition Replace intra prediction mode W / H=2 Mode 2, 3, 4 W / H>2 Mode 2, 3, 4, 5, 6 W / H=1 none H / W=1 / 2 Mode 32, 33, 34 H / W<1 / 2 Mode 30, 31, 32, 33, 34

[0392]

Table 7

[0393] condition Replace intra prediction mode W / H=2 Mode 2, 3, 4, 5, 6, 7 W / H>2 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H=1 none H / W=1 / 2 Modes 61, 62, 63, 64, 65, 66 H / W<1 / 2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0394] As shown in the examples of Table 6 and Table 7, the number of wide-angle intra replacement prediction modes falling within the transform range may differ according to the ratio of the current block.

[0395] By subdividing the ratio of the current block, the transform range can be set as shown in Table 8.

[0396]

Table 8

[0397] condition Replace intra prediction mode W / H=16 Mode 12, 13, 14, 15 W / H=8 Mode 12, 13 W / H=4 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 H / W=2 Mode 2, 3, 4, 5, 6, 7 H / W=1 none W / H=1 / 2 Modes 61, 62, 63, 64, 65, 66 W / H=1 / 4 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66 W / H=1 / 8 Mode 55,56 H / W=1 / 16 Modes 53, 54, 55, 56

[0398] When a non-adjacent reference sample line is determined as the reference sample line of the current block, or when a multi-line intra-frame prediction encoding method for selecting any one of multiple reference sample lines is used, it can be set not to use the wide-angle intra-frame prediction mode. In other words, even if the current block is non-square and the intra-frame prediction mode of the current block falls within the conversion range, the intra-frame prediction mode of the current block may not be converted to the wide-angle intra-frame prediction mode.

[0399] Alternatively, when the intra prediction mode of the current block is determined to be the wide-angle intra prediction mode, the non-adjacent reference sample line may be set to be unusable as the reference sample line of the current block, or may be set to not use the multi-line intra prediction encoding method for selecting any one of the multiple reference sample lines. When the multi-line intra prediction encoding method is not used, the adjacent reference sample line may be set as the reference sample line of the current block.

[0400] When the wide-angle intra prediction mode is not used, refW and refH are set to the sum of nTbW and nTbH. Thus, in addition to the upper left reference sample, non-adjacent reference samples at a distance i from the current block may include (nTbW + nTbH + offsetX[i]) upper reference samples and (nTbW + nTbH + offsetY[i]) left reference samples. That is, non-adjacent reference samples at a distance i from the current block may include (2nTbW + 2nTbH + offsetX[i] + offsetY[i] + 1) reference samples. For example, when the value of whRatio is greater than 1, the value of offsetX is set to a value greater than offsetY. For example, the value of offsetX is set to 1 and the value of offsetY is set to 0. On the other hand, when the value of whRatio is less than 1, the value of offsetY is set to a value greater than offsetX. For example, the value of offsetX is set to 0 and the value of offsetY is set to 1.

[0401] When the Wide intra-frame prediction mode is used in addition to the existing intra-frame prediction mode, the resources required to encode the Wide intra-frame prediction mode increase, which may reduce encoding efficiency. Therefore, instead of directly encoding the Wide intra-frame prediction mode, an alternative intra-frame prediction mode related to the Wide intra-frame prediction mode is encoded, thereby improving encoding efficiency.

[0402] For example, when the current block is encoded using the 67th Wide intra prediction mode, the number 2, which is the 67th Wide alternative intra prediction mode, may be encoded as the intra prediction mode of the current block. Furthermore, when the current block is encoded using the -1th Wide intra prediction mode, the number 66, which is the -1th Wide alternative intra prediction mode, may be encoded as the intra prediction mode of the current block.

[0403] The decoder may decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode is included in the transformation range. When the decoded intra prediction mode is the wide-angle replacement intra prediction mode, the intra prediction mode may be transformed into the wide-angle intra prediction mode.

[0404] Alternatively, when the current block is encoded in the wide-angle intra prediction mode, the wide-angle intra prediction mode may also be directly encoded.

[0405] The encoding of the intra prediction mode can be implemented based on the MPM list. Specifically, when encoding adjacent blocks in the wide-angle intra prediction mode, the MPM can be set based on the wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode.

[0406] If a prediction block is generated using intra prediction, the prediction samples can be updated based on the position of each prediction sample included in the prediction block. This updating method is referred to as a position-based intra weighted prediction method (or position-dependent prediction combination (PDPC)).

[0407] Whether to use PDPC may be determined by considering the intra-frame prediction mode of the current block, the reference sample line of the current block, the size or color component of the current block. For example, PDPC may be used when the intra-frame prediction mode of the current block is at least one of a planar mode, a DC mode, a vertical direction mode, a horizontal direction mode, a mode with an index value smaller than that in the vertical direction, or a mode with an index value larger than that in the horizontal direction. Alternatively, PDPC may be used only when at least one of the width or height of the current block is greater than 4. Alternatively, PDPC may be used only when the index of the reference image line of the current block is 0. Alternatively, PDPC may be used only when the index of the reference image line of the current block is greater than or equal to a predefined value. Alternatively, PDPC may be used only for the luminance component. Alternatively, whether to use PDPC may be determined based on whether two or more of the enumerated conditions are met.

[0408] As another example, information indicating whether PDPC is applied may be signaled via the code stream.

[0409] If a prediction sample is obtained through intra-frame prediction, a reference sample for correcting the prediction sample can be determined based on the position of the obtained prediction sample. For ease of explanation, in the subsequent embodiments, the reference sample used to correct the prediction sample is referred to as a PDPC reference sample. Furthermore, the prediction sample obtained through intra-frame prediction is referred to as a first prediction sample, and the prediction sample obtained by correcting the first prediction sample is referred to as a second prediction sample.

[0410] Figure 27 It is a diagram showing an embodiment to which PDPC is applied.

[0411] The first prediction sample may be corrected using at least one PDPC reference sample, wherein the PDPC reference sample may include at least one of a reference sample adjacent to the upper left corner of the current block, an upper reference sample located above the current block, or a left reference sample located to the left of the current block.

[0412] At least one of the reference samples of the reference sample line of the current block may be set as a PDPC reference sample. Alternatively, regardless of the reference sample line of the current block, at least one of the reference samples of the reference sample line with index 0 may be set as a PDPC reference sample. For example, even if the first prediction sample is obtained using the reference samples included in the reference sample line with index 1 or index 2, the second prediction sample may be obtained using the reference samples included in the reference sample line with index 0.

[0413] The number or position of PDPC reference samples used to correct the first prediction sample may be determined considering at least one of the intra prediction mode of the current block, the size of the current block, the shape of the current block, or the position of the first prediction sample.

[0414] For example, when the intra prediction mode of the current block is planar mode or DC mode, the second prediction sample can be obtained using the upper reference sample and the left reference sample. In this case, the upper reference sample can be a reference sample perpendicular to the first prediction sample (for example, a reference sample with the same x coordinate), and the left reference sample can be a reference sample horizontal to the first prediction sample (for example, a reference sample with the same y coordinate).

[0415] When the intra prediction mode of the current block is a horizontal intra prediction mode, the second prediction sample may be obtained using an upper reference sample, where the upper reference sample may be a reference sample perpendicular to the first prediction sample.

[0416] When the intra prediction mode of the current block is a vertical intra prediction mode, the second prediction sample may be obtained using the left reference sample. In this case, the left reference sample may be a reference sample horizontal to the first prediction sample.

[0417] When the intra prediction mode of the current block is the lower-left diagonal intra prediction mode or the upper-right diagonal intra prediction mode, the second prediction sample may be obtained based on the upper-left reference sample, the upper reference sample, and the left reference sample. The upper-left reference sample may be a reference sample adjacent to the upper-left corner of the current block (e.g., a reference sample at the (-1, -1) position). The upper reference sample may be a reference sample located in the upper-right diagonal direction of the first prediction sample, and the left reference sample may be a reference sample located in the lower-left diagonal direction of the first prediction sample.

[0418] In summary, when the position of the first prediction sample is (x, y), R(-1, -1) can be set as the upper left reference sample, R(x+y+1, -1) or R(x, -1) can be set as the upper reference sample. In addition, R(-1, x+y+1) or R(-1, y) can be set as the left reference sample.

[0419] A prediction mode may be applied to the current block multiple times, or multiple prediction modes may be applied to the current block repeatedly. In this way, the prediction method using the same or different prediction modes may be referred to as a combined prediction mode (or multi-hypothesis prediction mode).

[0420] The combined prediction mode may include at least one of a mode combining merge mode and merge mode, a mode combining inter prediction and intra prediction, a mode combining merge mode and advanced motion vector prediction mode, and a mode combining merge mode and intra prediction.

[0421] In the combined prediction mode, a first prediction block may be generated based on the first prediction mode, and a second prediction block may be generated based on the second prediction mode. A third prediction block may then be generated based on a weighted sum operation of the first and second prediction blocks. The third prediction block may be set as the final prediction block for the current block.

[0422] Whether to use the combined prediction mode may be determined based on the size or shape of the current block. For example, whether to use the combined prediction mode may be determined based on at least one of the size of the coding block, the number of samples included in the coding block, the width of the coding block, and the height of the coding block. For example, when at least one of the width or height of the coding block is greater than or equal to 128, or when the number of samples included in the coding block is less than or equal to 64, the combined prediction mode may not be applied to the coding block.

[0423] In addition, information indicating whether a combined prediction mode is applied to the current block may be signaled via the codestream. For example, the information may be a 1-bit flag. For example, a flag mh_intra_flag indicating whether a combined prediction mode combining merge mode and intra prediction is used may be signaled via the codestream. A value of 1 for mh_intra_flag indicates that a combined prediction mode combining merge mode and intra prediction is used, and a value of 0 for mh_intra_flag indicates that a combined prediction mode combining merge mode and intra prediction is not used.

[0424] Combined prediction mode, which combines merge mode and intra prediction, can be applied to the current block only when the prediction mode of the current block is inter prediction mode and merge mode is applied to the current block. That is, when the value of the merge_flag flag indicating whether merge mode is applied to the current block is 1, mh_intra_flag can be signaled.

[0425] In the combined prediction mode that combines the merge mode and the intra prediction, the intra prediction mode of the current block can be set to a predefined intra prediction mode. For example, when the combined prediction mode is used, the intra prediction mode of the current block can be set to the planar mode.

[0426] As another example, when using a combined prediction mode that combines merge mode and intra-frame prediction, one of the candidate intra-frame prediction modes can be determined as the intra-frame prediction mode for the current block. Here, the candidate intra-frame prediction mode may include at least one of a non-angular intra-frame prediction mode and an angular intra-frame prediction mode in a specific direction. Here, the non-angular intra-frame prediction mode includes at least one of a DC mode and a planar mode, and the angular intra-frame prediction mode includes at least one of a horizontal intra-frame prediction mode, a vertical intra-frame prediction mode, and a diagonal intra-frame prediction mode. For example, when using the combined prediction mode, only the DC mode, the planar mode, the horizontal intra-frame prediction mode, or the vertical intra-frame prediction mode can be set as the intra-frame prediction mode for the current block. Alternatively, when using the combined prediction mode, only the planar mode, the horizontal intra-frame prediction mode, or the vertical intra-frame prediction mode can be set as the intra-frame prediction mode for the current block. Alternatively, when using the combined prediction mode, only the DC mode, the planar mode, or the vertical intra-frame prediction mode can be set as the intra-frame prediction mode for the current block. Alternatively, when the combined prediction mode is used, only the DC mode, the planar mode, or the intra prediction mode in the horizontal direction may be set as the intra prediction mode of the current block.

[0427] Index information for specifying any one of the candidate intra-frame prediction modes may be signaled via the codestream. For example, the index mh_intra_idx specifying any one of the candidate intra-frame prediction modes may be signaled via the codestream. Tables 9 and 10 show the intra-frame prediction modes according to the value of mh_intra_idx. The intra-frame prediction mode indicated by mh_intra_idx may be determined as the intra-frame prediction mode for the current block.

[0428]

Table 9

[0429] mh_intra_idx 0 1 2 3 Intra-frame mode Plane Mode DC mode Vertical Mode Horizontal Mode

[0430]

Table 10

[0431] mh_intra_idx 0 1 2 Intra-frame mode Plane Mode Vertical Mode Horizontal Mode

[0432] In the encoder and decoder, the number of candidate intra prediction modes may have a fixed value. Alternatively, the number or type of candidate intra prediction modes may differ according to at least one of the size of the current block, the shape of the current block, or whether a neighboring block is encoded by intra prediction.

[0433] As another example, when a combined prediction mode that combines merge mode and intra prediction is applied, the intra prediction mode of the current block can be determined based on the MPM. In this case, when general intra prediction is applied, the number of MPMs included in the MPM list may be different from the number of MPMs included in the MPM list when the combined prediction mode is applied. For example, when the combined prediction mode is not applied, the MPM list may include 6 or 5 MPMs, while when the combined prediction mode is applied, the MPM list may include 4 or 3 MPMs. That is, when general intra prediction is performed, N MPMs are used, while when the combined prediction mode is applied, fewer than N MPMs may be used.

[0434] For example, assuming that when the general intra prediction mode is applied, the MPMs derived based on the intra prediction modes of neighboring blocks adjacent to the current block are plane mode, DC mode, INTRA_MODE32, INTRA_MODE31, INTRA_MODE33, and INTRA_MODE30. When the combined prediction mode is applied to the current block, only some of the six MPMs may be used to determine the intra prediction mode of the current block. For example, an MPM list including the three MPMs with the smallest indexes among the MPMs (i.e., plane mode, DC mode, and INTRA_MODE32) may be used to determine the intra prediction mode of the current block. Alternatively, any one of the MPMs with a predetermined index value may be determined as the intra prediction mode of the current block.

[0435] As another example, the number of MPMs used to determine the intra prediction mode of the current block when general intra prediction is applied may be set to be the same as the number of MPMs used to determine the intra prediction mode of the current block when combined prediction mode is applied.

[0436] When the combined prediction mode is applied, the encoding of the MPM flag can be omitted and the MPM flag can be regarded as true. That is, when the combined prediction mode is applied, any one of the MPMs can be determined as the intra prediction mode of the current block.

[0437] When the combined prediction mode combining the merge mode and the intra prediction is applied to the current block, the multi-line intra prediction encoding method can be set not to be used. When the multi-line intra prediction encoding method is not used, the adjacent reference sample lines can be used for the intra prediction of the current block.

[0438] Alternatively, when a combined prediction mode combining merge mode and intra prediction is applied to the current block, a multi-line intra prediction encoding method may be used. Information for determining a reference sample line for the current block from among a plurality of reference sample lines may be signaled via a codestream. Alternatively, the reference sample line for the current block may be determined by considering at least one of the size of the current block, the shape of the current block, the intra prediction mode of the current block, or reference sample lines of neighboring blocks.

[0439] The number of available candidate intra-frame prediction modes can be set differently according to the reference sample line of the current block. For example, when using adjacent reference sample lines, the candidate intra-frame prediction modes may include at least one of a planar mode, a DC mode, a horizontal intra-frame prediction mode, a vertical intra-frame prediction mode, a lower left diagonal intra-frame prediction mode, and an upper right diagonal intra-frame prediction mode. On the other hand, when using non-adjacent reference sample lines, the candidate intra-frame prediction modes may include at least one of a horizontal intra-frame prediction mode, a vertical intra-frame prediction mode, a lower left diagonal intra-frame prediction mode, and an upper right diagonal intra-frame prediction mode.

[0440] The first prediction block can be generated using the merge candidate selected in merge mode, and the second prediction block can be generated using intra prediction mode. In this case, when generating the first prediction block, it can be set not to apply triangular partitioning to the current block. That is, in the combined prediction mode that combines merge mode and intra prediction, triangular partitioning may not be allowed. In addition, when generating the second prediction block, it can be set not to apply PDPC to the current block. That is, in the combined prediction mode that combines merge mode and intra prediction, PDPC may not be allowed.

[0441] The third prediction block may be generated based on a weighted sum operation of the first prediction block and the second prediction block. For example, Equation 13 shows a diagram of an example of generating the third prediction block based on a weighted sum operation of the first prediction block and the second prediction block.

[0442] Equation 13

[0443] P comb =(w*P merge +(Nw)*P intra +4)>>log2N

[0444] In Equation 13, P merge represents the first prediction block obtained based on the merge mode, and P intra P represents the second prediction block obtained based on intra-frame prediction. combrepresents a third prediction block obtained by combining the first prediction block and the second prediction block. w represents a first weighting value applied to the first prediction block. The second weighting value applied to the second prediction block can be derived by subtracting the first weighting value w from a constant N. Here, N can have a value predefined by the encoder and decoder. For example, N can be 4 or 8. Alternatively, the constant N can be derived based on at least one of the size, shape, and intra-frame prediction mode of the current block.

[0445] Unlike the example shown in Equation 13, w may be set as the second weighting value, and a value obtained by subtracting the second weighting value from a predetermined constant N may be set as the first weighting value.

[0446] The weighting values applied to the first prediction block and the second prediction block may be determined based on at least one of the intra prediction mode of the current block and whether the merge mode has bidirectional motion information. For example, the first weighting value w when the intra prediction mode of the current block is an angular prediction mode may be greater than the first weighting value w when the intra prediction mode of the current block is a non-angular prediction mode. Alternatively, the first weighting value w when bidirectional prediction is performed using a merge candidate may be greater than the first weighting value w when unidirectional prediction is performed using the merge candidate.

[0447] As another example, the weighting value can be determined based on the prediction mode of the neighboring blocks adjacent to the current block. Here, the neighboring blocks may include at least one of an upper neighboring block adjacent to the top of the current block, a left neighboring block adjacent to the left of the current block, and an upper left neighboring block adjacent to the upper left corner of the current block. The weighting value can be determined based on the number of neighboring blocks adjacent to the current block that are encoded using intra-frame prediction. For example, as the number of neighboring blocks adjacent to the current block that are encoded using intra-frame prediction increases, the first weighting value may have a smaller value, and the second weighting value may have a larger value. On the other hand, as the number of neighboring blocks adjacent to the current block that are encoded using non-intra-frame prediction decreases, the first weighting value may have a larger value, and the second weighting value may have a smaller value. For example, when both the left neighboring block and the upper neighboring block are encoded using intra-frame prediction, the second weighting value may be set to n. When only one of the left neighboring block and the upper neighboring block is encoded using intra-frame prediction, the second weighting value may be set to n / 2. When neither the left neighboring block nor the upper neighboring block is encoded by intra prediction, the second weighting value may be set to n / 4. Here, n represents an integer equal to or smaller than N.

[0448] The weighting value can be determined per sample or subblock in the current block. For example, the current block can be divided into multiple subblocks, and the weighting value applied to each subblock can be set to a different value. In this case, the number of subblocks can be fixed in the encoder and decoder. Alternatively, the number of subblocks can be adaptively determined based on at least one of the size of the current block, the shape of the current block, the intra-frame prediction mode of the current block, and the prediction mode of the neighboring block.

[0449] Figure 28 is a diagram showing an embodiment in which weighted values are applied.

[0450] The current block may be divided into a plurality of sub-blocks, and the weighting value applied to each sub-block may be set to a different value. Figure 28 (a) and Figure 28 In the example shown in (b), for the sub-block including the sample located at the upper left corner of the current block, the second weighting value is set to be greater than the first weighting value. On the other hand, for the sub-block including the sample located at the lower right corner of the current block, the first weighting value can be set to be greater than the second weighting value.

[0451] The third predicted sub-block of the sub-block can be obtained by weighted prediction of the first predicted sub-block and the second predicted sub-block corresponding to the sub-block. Figure 28 In the example shown, the weight value used for weighted prediction may be set differently for each sub-block.

[0452] Depending on whether the merge candidate has bidirectional motion information, the weight value assigned to the subblock may be determined differently.

[0453] Figure 29 is a diagram showing an example of assigning different weighting values to subblocks according to whether a merge candidate has bidirectional motion information.

[0454] When the merging candidate has bidirectional motion information, the first weight value w assigned to at least one subblock may be set to a larger value than when the merging candidate has unidirectional motion information. Figure 29 (a) and Figure 29 In (b), it is shown that, for all subblocks, the weighting value w applied when bidirectional prediction is applied has a value greater than the weighting value w applied when unidirectional prediction is applied.

[0455] Alternatively, the weighting value applied to the subblock may be determined based on the intra prediction mode of the current block.

[0456] Figure 30 is a diagram showing an example of assigning different weighting values to subblocks according to an intra prediction mode of a current block.

[0457] When the intra prediction mode of the current block is the vertical direction, the second weighted value of the subblock located above the current block may be set to be greater than the second weighted value of the subblock located below the current block.

[0458] For example, for a sub-block located above the current block, the second weighted value may be set to be greater than the first weighted value, while for a sub-block located below the current block, the first weighted value and the second weighted value may be set to be equal, or the first weighted value may be set to be greater than the second weighted value.

[0459] When the intra prediction mode of the current block is horizontal, the second weighted value of the subblock located on the left side of the current block may be set to be greater than the second weighted value of the subblock located on the right side of the current block.

[0460] For example, for the sub-block located on the left side of the current block, the second weighted value can be set to be greater than the first weighted value, and for the sub-block located on the right side of the current block, the first weighted value and the second weighted value can be set to be equal, or the first weighted value can be set to be greater than the second weighted value.

[0461] As another example, when the intra-frame prediction mode of the current block is vertical, the first weighted value of the sub-block located above the current block is set to be greater than the first weighted value of the sub-block located below the current block, or when the intra-frame prediction mode of the block is horizontal, the first weighted value of the sub-block located on the left side of the current block can be set to be greater than the first weighted value of the sub-block located on the right side of the current block.

[0462] As another example, the weighting value assigned to each subblock can be determined based on the prediction mode of the neighboring block adjacent to the current block. For example, when the upper neighboring block of the current block is encoded by intra-frame prediction and the left neighboring block of the current block is not encoded by intra-frame prediction, the second weighting value of the subblock located above the current block can be set to be greater than the second weighting value of the subblock located below the current block. On the other hand, when the upper neighboring block of the current block is not encoded by intra-frame prediction and the left neighboring block of the current block is encoded by intra-frame prediction, the second weighting value of the subblock located to the left of the current block can be set to be greater than the second weighting value of the subblock located to the right of the current block.

[0463] For at least one of the plurality of subblocks, the first weighting value or the second weighting value may be set to 0. That is, for at least one of the plurality of subblocks, the first predicted subblock or the second predicted subblock may be set to the third predicted subblock. Therefore, combined prediction may not be performed on at least one of the plurality of subblocks. For example, for at least one of the plurality of subblocks, the first weighting value w may be set to 0, or the first weighting value w may be set to a constant N.

[0464] A block to which a combined prediction mode combining merge mode and intra prediction is applied can be considered to be encoded using inter prediction. Therefore, the intra prediction mode of a block encoded in the combined prediction mode can be set to be unavailable for deriving the intra prediction mode of a block to be encoded / decoded later. When deriving the MPM, the intra prediction mode of the unavailable block can be considered to be planar mode.

[0465] On the other hand, the motion information of the block encoded in the combined prediction mode may be set as motion information that may be used to derive a block to be encoded / decoded later.

[0466] In a mode that combines merge mode and merge mode, multiple merge candidates can be used to perform motion compensated prediction. Specifically, a first prediction block can be generated using a first merge candidate, and a second prediction block can be generated using a second merge candidate. A third prediction block can be generated based on a weighted sum operation of the first prediction block and the second prediction block.

[0467] Information for specifying the first merge candidate and the second merge candidate may be signaled separately through the codestream. For example, index information merge_idx for specifying the first merge candidate and index information merge_2nd_idx for specifying the second merge candidate may be signaled through the codestream. The second merge candidate may be determined based on the index information merge_2nd_idx and the index information merge_idx.

[0468] The index information merge_idx specifies any one of the merge candidates included in the merge candidate list.

[0469] The index information merge_2nd_idx may specify any one of the remaining merge candidates other than the merge candidate specified by merge_idx. Therefore, when the value of merge_2nd_idx is less than the value of merge_idx, the merge candidate indexed by the value of merge_2nd_idx may be set as the second merge candidate. When the value of merge_2nd_idx is equal to or greater than the value of merge_idx, the merge candidate indexed by 1 plus the value of merge_2nd_idx may be set as the second merge candidate.

[0470] Alternatively, the second merging candidate may be specified in consideration of the search order of the candidate blocks.

[0471] Figure 31 is a diagram illustrating an example of specifying a second merge candidate in consideration of the search order of candidate blocks.

[0472] exist Figure 31In the example shown, the indices marked on the adjacent samples and non-adjacent samples indicate the order in which the candidate blocks are searched. For example, the candidate blocks may be searched sequentially from position A0 to position A14.

[0473] When block A4 is selected as the first merge candidate, a merge candidate derived from a candidate block after A4 in the search order may be designated as the second merge candidate. For example, a merge candidate derived from A5 may be selected as the second merge candidate. When the candidate block at position A5 cannot be used as a merge candidate, a merge candidate derived from the next candidate block may be selected as the second merge candidate.

[0474] The first merge candidate and the second merge candidate may also be selected from merge candidates derived from non-adjacent blocks.

[0475] Figure 32 is a diagram illustrating an example of selecting a first merge candidate and a second merge candidate from merge candidates derived from non-adjacent blocks.

[0476] like Figure 32 In the example shown, the first and second merge candidates derived from the first and second candidate blocks that are not adjacent to the current block can be selected as the first and second merge candidates, respectively. In this case, the block rows to which the first and second candidate blocks belong can be different. For example, the first merge candidate can be derived from any candidate block from A5 to A10, and the second merge candidate can be derived from any candidate block from A11 to A15.

[0477] Alternatively, it may be arranged such that the first candidate block and the second candidate block are not included in the same line (eg, row or column).

[0478] As another example, the second merge candidate can be specified based on the first merge candidate. In this case, the first merge candidate can be specified by the index information merge_idx sent by the code stream signal. For example, a merge candidate adjacent to the first merge candidate can be designated as the second merge candidate. Here, the merge candidate adjacent to the first merge candidate may mean a merge candidate whose index difference with the first merge candidate is 1. For example, the merge candidate with an index value of merge_idx+1 can be set as the second merge candidate. In this case, when the value of merge_idx+1 is greater than the maximum index value (or when the index value of the first merge candidate is the maximum index), the merge candidate with an index value of merge_idx-1 or a merge candidate with an index value of a predefined value (for example, 0) can be set as the second merge candidate.

[0479] Alternatively, a merge candidate adjacent to the first merge candidate may refer to a merge candidate derived from a candidate block that is spatially adjacent to the candidate block used to derive the first merge candidate. Here, a neighboring candidate block of a candidate block may refer to a block adjacent to the left, right, above, below, or diagonally of the candidate block.

[0480] As another example, the second merge candidate may be specified based on the motion information of the first merge candidate. For example, a merge candidate having the same reference image as the first merge candidate may be selected as the second merge candidate. When there are multiple merge candidates that share the same reference image as the first merge candidate, the merge candidate with the smallest index, the merge candidate with the smallest index difference from the first merge candidate, or the merge candidate with the smallest motion vector difference from the first merge candidate among the multiple merge candidates may be selected as the second merge candidate. Alternatively, the second merge candidate may be selected based on index information specifying any one of the multiple merge candidates.

[0481] Alternatively, when the first merge candidate is a unidirectional prediction in the first direction, a merge candidate including motion information in the second direction may be set as the second merge candidate. For example, when the first merge candidate has motion information in the direction L0, a merge candidate having motion information in the direction L1 may be set as the second merge candidate. When there are multiple merge candidates having motion information in the direction L1, the merge candidate with the smallest index among the multiple merge candidates or the merge candidate with the smallest index difference from the first merge candidate is set as the second merge candidate. Alternatively, the second merge candidate may be selected based on index information specifying any one of the multiple merge candidates.

[0482] As another example, one of the merge candidates derived from neighboring blocks adjacent to the current block may be set as the first merge candidate, and one of the merge candidates derived from non-neighboring blocks not adjacent to the current block may be set as the second merge candidate.

[0483] As another example, one of the merge candidates derived from the candidate block located above the current block may be set as the first merge candidate, and one of the merge candidates derived from the candidate block located on the left may be set as the second merge candidate.

[0484] In the above-mentioned combined prediction mode combining the merge mode and the merge mode, the merge mode may refer to a merge mode based on a translational motion model (hereinafter referred to as the translational merge mode) or a merge mode based on an affine motion model (hereinafter referred to as the affine merge mode). That is, motion compensation prediction can be performed by combining the translational merge mode and the translational merge mode or by combining the affine merge mode and the affine merge mode.

[0485] As another example, when the neighboring blocks used to derive the first merge candidate are encoded based on affine motion information, the merge candidate derived from the neighboring blocks encoded based on the affine motion information can be set as the second merge candidate. For example, when the first merge candidate is an affine merge candidate, or when the first merge candidate is derived based on the motion information of a sub-block in a coding block encoded based on affine motion information, the affine merge candidate or the merge candidate derived based on the motion information of the sub-block in the coding block encoded based on the affine motion information can be selected as the second merge candidate. Here, the second merge candidate can include at least one of the merge candidate that is closest to the first merge candidate in the search order among the merge candidates that meet the above conditions, the merge candidate with the smallest index difference from the first merge candidate, the merge candidate with the smallest index, and the merge candidate with the smallest motion vector difference from the first merge candidate.

[0486] In contrast to the above example, when the neighboring blocks used to derive the first merge candidate are encoded based on translational motion information (i.e., non-affine motion information), the merge candidate derived from the neighboring blocks encoded based on the translational motion information can be set as the second merge candidate. For example, when the first merge candidate is a non-affine merge candidate, the non-affine merge candidate can be selected as the second merge candidate. Here, the second merge candidate can include at least one of the following: a merge candidate that is closest to the first merge candidate in search order among the non-affine merge candidates, a merge candidate with the smallest index difference from the first merge candidate, a merge candidate with the smallest index, and a merge candidate with the smallest motion vector difference from the first merge candidate.

[0487] Alternatively, the second merge candidate is derived by adding or subtracting an offset from the index of the first merge candidate, and when the encoding methods of the adjacent blocks used to derive the first merge candidate and the second merge candidate are different, the second merge candidate may be reset to another merge candidate.

[0488] Figure 33 is a diagram for describing an example of determining a second merge candidate.

[0489] When the first merge candidate is selected, a merge candidate having an index derived by adding or subtracting an offset from the index of the first merge candidate may be selected as the second merge candidate. For example, a merge candidate having a value indicated by merge_idx signaled from the codestream as an index may be selected as the first merge candidate, and a merge candidate having merge_idx+1 as an index may be selected as the second merge candidate.

[0490] In this case, when the first merge candidate is an affine merge candidate or is derived from a sub-block belonging to a coding block encoded using an affine motion model, the affine merge candidate or the merge candidate derived from the sub-block belonging to the coding block encoded using an affine motion model should be set as the second merge candidate. When the merge candidate with merge_idx+1 as an index is not an affine merge candidate or is not derived from a sub-block belonging to a coding block encoded using an affine motion model, another merge candidate may be set as the second merge candidate. Here, the other merge candidate may be the merge candidate having the smallest difference from merge_idx+1 among the first merge candidate or the affine merge candidate or the merge candidate derived from the sub-block belonging to the coding block encoded using an affine motion model.

[0491] On the other hand, when the first merge candidate is a non-affine merge candidate, the non-affine merge candidate should be set as the second merge candidate. When the merge candidate with merge_idx+1 as the index is an affine merge candidate or is derived from a sub-block belonging to a coding block encoded by an affine motion model, other merge candidates can be set as the second merge candidate. Here, the other merge candidate can be the merge candidate with the minimum difference from merge_idx+1 among the first merge candidate or the non-affine merge candidate. For example, in Figure 33 In the example shown, it is shown that the merge candidate A3 with the index merge_idx+2 is set as the second merge candidate.

[0492] As another example, motion compensated prediction can be performed by combining the translation merge mode with the affine merge mode. That is, either the first merge candidate or the second merge candidate can be an affine merge candidate, while the other can be a non-affine merge candidate.

[0493] The combined prediction block can be obtained by performing a weighted sum operation on the first prediction block derived from the first merge candidate and the second prediction block derived based on the second merge candidate. In this case, the weighted value applied to the first prediction block can be set to be greater than the weighted value applied to the second prediction block.

[0494] Alternatively, the weighting value may be determined based on the motion information of the first merging candidate and the motion information of the second merging candidate. For example, the weighting value applied to the first prediction block and the second prediction block may be determined based on the difference in the output order of the reference image and the current image. Specifically, the greater the difference in the output order of the reference image and the current image, the smaller the weighting value applied to the prediction block may be set.

[0495] Alternatively, the weighting values applied to the first prediction block and the second prediction block may be determined taking into account the size or shape of the candidate block used to derive the first merge candidate (hereinafter referred to as the first candidate block) and the candidate block used to derive the second merge candidate (hereinafter referred to as the second candidate block). For example, the weighting value applied to the prediction block derived from a candidate block of the first candidate block or the second candidate block that has a shape similar to the current block may be set to a larger value. On the other hand, the weighting value applied to the prediction block derived from a candidate block of the first candidate block or the second candidate block that does not have a shape similar to the current block may be set to a smaller value.

[0496] Figure 34 is a diagram illustrating an example of determining a weight value applied to a prediction block based on the shape of a candidate block.

[0497] Assume that the current block is non-square with a width greater than a height.

[0498] The first prediction block and the second prediction block may be derived based on the first merge candidate and the second merge candidate, and a combined prediction block may be generated based on a weighted sum operation of the first prediction block and the second prediction block. In this case, the weight values applied to the first prediction block and the second prediction block may be determined based on the shapes of the first candidate block and the second candidate block.

[0499] For example, in Figure 34 In the example shown, the first candidate block is a square, and the second candidate block is a non-square block with a width greater than a height. Because the second candidate block has the same shape as the current block, the weighting value applied to the second prediction block can be set to be greater than the weighting value applied to the first prediction block. For example, a weighting value of 5 / 8 can be applied to the second prediction block, and a weighting value of 3 / 8 can be applied to the first prediction block. Equation 14 shows an example of deriving a combined prediction block based on a weighted sum operation of the first and second prediction blocks.

[0500] Equation 14

[0501] P(x, y)=(3*P1(x, y)+5*P2(x, y))>>3

[0502] P(x, y) represents a combined prediction block, P1(x, y) represents a first prediction block, and P2(x, y) represents a second prediction block.

[0503] As another example, the weighting values applied to the first prediction block and the second prediction block can be determined based on the shape of the current block. For example, when the current block is non-square with a width greater than its height, a larger weighting value can be applied to the prediction block generated based on the merge candidate derived from the candidate block located above the current block from among the first and second merge candidates. When both the first and second merge candidates are derived from candidate blocks located above the current block, the weighting values applied to the first and second prediction blocks can be set identically. On the other hand, when the current block is non-square with a height greater than its width, a larger weighting value can be applied to the prediction block generated based on the merge candidate derived from the candidate block located to the left of the current block from among the first and second merge candidates. When both the first and second merge candidates are derived from candidate blocks located to the left of the current block, the weighting values applied to the first and second prediction blocks can be set identically. When the current block is square, the weighting values applied to the first and second prediction blocks can be set identically.

[0504] As another example, the weighting value applied to each prediction block may be determined based on the distance between the current block and the candidate block. Here, the distance may be derived based on the x-axis coordinate difference, the y-axis coordinate difference, or the minimum of the two with respect to the current block. The weighting value applied to the prediction block derived from the merge candidate with a smaller distance from the current block may be set to be greater than the weighting value applied to the prediction block derived from the merge candidate with a larger distance from the current block. For example, in Figure 31 In the example shown, the first merge candidate is derived from a neighboring block adjacent to the current block, and the second merge candidate is derived from a non-neighboring block not adjacent to the current block. In this case, since the x-axis distance between the first candidate block and the current block is smaller than the x-axis distance between the second candidate block and the current block, the weighting value applied to the first prediction block can be set to be greater than the weighting value applied to the second prediction block.

[0505] Alternatively, when both the first merge candidate and the second merge candidate are derived from non-adjacent blocks, a larger weight value may be assigned to the prediction block derived from a non-adjacent block that is closer to the current block among the non-adjacent blocks. Figure 32 In the example shown, since the y-axis distance between the first candidate block and the current block is smaller than the y-axis distance between the second candidate block and the current block, the weighting value applied to the first prediction block can be set to be greater than the weighting value applied to the second prediction block.

[0506] The combined motion information may be derived based on the first merge candidate and the second merge candidate, and motion compensated prediction may be performed on the current block based on the combined motion information. For example, the motion vector of the current block may be derived based on an average operation or a weighted sum operation of the motion vector of the first merge candidate and the motion vector of the second merge candidate. In this case, the weighted value applied to the motion vector of the first merge candidate and the weighted value applied to the motion vector of the second merge candidate may be determined according to the above-described embodiment.

[0507] In the case where the first merging candidate is a non-affine merging candidate and the second affine merging candidate is an affine merging candidate, the motion vector of the current block may be derived by scaling the motion vector of the second merging candidate. Equation 15 shows an example of deriving the motion vector of the current block.

[0508] Equation 15

[0509] (mvX, mvY)=(mvOx, mvOy)+((mv1x, mv1y)>>M)

[0510] In Equation 15, (mvX, mvY) represents the motion vector of the current block, (mv0x, mv0y) represents the motion vector of the first merge candidate, and (mv1x, mv1y) represents the motion vector of the second merge candidate. M represents a scaling parameter. M can be predefined in the encoder and decoder. Alternatively, the value of the scaling parameter M can be determined based on the size of the current block or candidate block. For example, when the width or height of the second candidate block is greater than 32, M can be set to 3, otherwise M can be set to 2.

[0511] The motion information of the first or second merging candidate may be stored as the motion information of the block to which the merge prediction mode combining the merge mode and the merge mode is applied. The stored motion information may be used to derive motion information of the block to be encoded / decoded later.

[0512] Alternatively, the block may be divided into a plurality of sub-blocks, and one of the motion information of the first merge candidate and the motion information of the second merge candidate may be stored as the motion information of each sub-block. In this case, the motion information of some of the plurality of sub-blocks may be set as the motion information of the first merge candidate, and the motion information of the other sub-blocks may be set as the motion information of the second merge candidate.

[0513] Alternatively, the integrated motion information derived based on the motion information of the first merge candidate and the second merge candidate may be stored as the motion information of the block to which the merge prediction mode combining the merge mode and the merge mode is applied.

[0514] In a prediction mode in which the merge mode and the advanced motion vector prediction mode are combined, a first prediction block may be generated using motion information derived from a merge candidate, and a second prediction block may be generated using a motion vector derived from a motion vector prediction candidate.

[0515] In advanced motion vector prediction mode, motion vector prediction candidates can be derived from neighboring blocks or co-located blocks within a co-located image. You can then specify any one of the multiple motion vector prediction candidates and set the specified motion vector prediction candidate as the motion vector prediction result for the current block. The motion vector for the current block can then be derived by adding the motion vector prediction result for the current block to the motion vector difference.

[0516] In a prediction mode that combines merge mode and advanced motion vector prediction mode, a merge candidate and a motion vector prediction candidate can be derived from the same candidate block. For example, if a merge candidate is specified by merge_idx, the motion vector of the candidate block used to derive the specified merge candidate can be set as the motion vector prediction result. Alternatively, if a motion vector prediction candidate is specified by mvp_flag, a merge candidate derived from the candidate block used to derive the specified merge candidate can be selected.

[0517] Alternatively, the candidate blocks used to derive the merge candidate and the candidate blocks used to derive the motion vector prediction candidate can be different. For example, if a merge candidate derived from a candidate block located above the current block is selected, the motion vector prediction candidate derived from a candidate block located to the left of the current block can be selected.

[0518] Alternatively, when the merge candidate selected by index information and the motion vector prediction candidate selected by index information are derived from the same candidate block, the motion vector prediction candidate can be replaced by a motion vector prediction candidate derived from an adjacent candidate block adjacent to the candidate block, or the merge candidate can be replaced by a merge candidate derived from a candidate block adjacent to the candidate block.

[0519] Figure 35 is a diagram illustrating an example of replacing a merge candidate.

[0520] exist Figure 35 In the example shown, the merging candidate and the motion vector prediction candidate derived from the candidate block at position A2 are selected. As shown in the figure, when the merging candidate and the motion vector prediction candidate are derived from the same candidate block, the merging candidate or the motion vector prediction candidate derived from the candidate block adjacent to the candidate block can be used instead of the merging candidate or the motion vector prediction candidate. For example, Figure 35As shown, the merge candidate at position A1 may be used instead of the merge candidate at position A2.

[0521] A first prediction block may be derived based on a merge candidate for the current block, and a second prediction block may be derived based on the motion vector prediction candidate. A combined prediction block may then be derived through a weighted sum operation of the first prediction block and the second prediction block. In this case, the weighted value applied to the second prediction block generated using the advanced motion vector prediction mode may be set to be greater than the weighted value applied to the first prediction block generated using the merge mode.

[0522] The derived residual image can be derived by subtracting the original image from the predicted image. In this case, when the residual image is changed to the frequency domain, even if the high-frequency components in the frequency components are removed, the subjective image quality of the video will not be significantly reduced. Therefore, if the value of the high-frequency component is reduced or the value of the high-frequency component is set to 0, the compression efficiency is improved without causing obvious visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into 2D frequency components. The transformation can be performed using transformation techniques such as discrete cosine transform (DCT) or discrete sine transform (DST).

[0523] DCT uses cosine transform to decompose (or transform) the residual image into 2-dimensional frequency components, while DST uses sine transform to decompose (or transform) the residual image into 2-dimensional frequency components. As a result of the transformation of the residual image, the frequency components can be represented as the basic image. For example, when DCT transform is performed on a block of size N×N, N 2 The size of each basic pattern component included in the block of size N×N can be obtained by transformation. Depending on the transformation technology used, the size of the basic pattern component can be called a DCT coefficient or a DST coefficient.

[0524] The DCT transform technique is mainly used to transform images with a large number of non-zero low-frequency components. The DST transform technique is mainly used to transform images with a large number of high-frequency components.

[0525] The residual image may also be transformed using transformation techniques other than DCT or DST.

[0526] Hereinafter, the process of transforming the residual image into two-dimensional frequency components is referred to as two-dimensional image transformation. Furthermore, the magnitudes of the basic pattern components obtained from the transformation results are referred to as transform coefficients. For example, transform coefficients may refer to DCT coefficients or DST coefficients. When both the primary transform and the secondary transform (described later) are applied simultaneously, the transform coefficients may represent the magnitudes of the basic pattern components generated from the secondary transform results.

[0527] The transform technique can be determined on a block-by-block basis. The transform technique can be determined based on at least one of the prediction mode of the current block, the size of the current block, or the shape of the current block. For example, when the current block is encoded in intra-frame prediction mode and the size of the current block is less than N×N, the transform technique DST can be used to perform the transform. On the other hand, when the above conditions cannot be met, the transform technique DCT can be used to perform the transform.

[0528] In the residual image, some blocks may not be subjected to a 2D image transform. Not performing a 2D image transform is called a transform skip. When transform skipping is applied, quantization may be applied to the residual values that were not transformed.

[0529] After the current block is transformed using DCT or DST, the transformed current block can be transformed again. In this case, the DCT or DST-based transform can be defined as a primary transform, and the process of transforming the block to which the primary transform is applied again can be called a secondary transform.

[0530] The main transform may be performed using any one of a plurality of transform core candidates. For example, the main transform may be performed using any one of DCT2, DCT8, or DCT7.

[0531] Different transform kernels may be used for the horizontal and vertical directions. Information indicating the combination of the transform kernel for the horizontal direction and the transform kernel for the vertical direction may also be signaled in the bitstream.

[0532] The primary and secondary transforms may be performed on different units. For example, the primary transform may be performed on an 8×8 block, and the secondary transform may be performed on 4×4 sub-blocks within the transformed 8×8 block. In this case, the transform coefficients of the remaining area where the secondary transform is not performed may be set to 0.

[0533] Alternatively, the primary transform may be performed on the 4x4 block, and the secondary transform may be performed on an 8x8 sized region including the transformed 4x4 block.

[0534] Information indicating whether to perform secondary transformation may be signaled through the code stream.

[0535] Alternatively, whether to perform a secondary transform may be determined based on whether the horizontal transform kernel and the vertical transform kernel are the same. For example, a secondary transform may be performed only when the horizontal transform kernel and the vertical transform kernel are the same. Alternatively, a secondary transform may be performed only when the horizontal transform kernel and the vertical transform kernel are different.

[0536] Alternatively, the secondary transform may be allowed only when the horizontal transform and the vertical transform utilize a predefined transform kernel. For example, the secondary transform may be allowed when the horizontal transform and the vertical transform use a DCT2 transform kernel.

[0537] Alternatively, whether to perform a secondary transform may be determined based on the number of non-zero transform coefficients in the current block. For example, when the non-zero transform coefficients of the current block are less than or equal to a threshold, the secondary transform may be disabled, and when the non-zero transform coefficients of the current block are greater than the threshold, the secondary transform may be enabled. Alternatively, the secondary transform may be enabled only when the current block is encoded using intra-frame prediction.

[0538] Based on the shape of the current block, the size or shape of the sub-block on which the secondary transform is to be performed may be determined.

[0539] Figure 36 and Figure 37 is a diagram showing sub-blocks on which secondary transform will be performed.

[0540] When the current block is square, after the primary transform is performed, a secondary transform may be performed on the N×N sub-block at the upper left corner of the current block. For example, when the current block is an 8×8 coding block, a secondary transform may be performed on the 4×4 sub-block at the upper left corner of the current block after the primary transform is performed on the current block (see Figure 36 ).

[0541] When the current block is a non-square block with a width greater than or equal to 4 times its height, after performing the primary transform, a secondary transform may be performed on the sub-block of size (kN)×(4kN) at the upper left corner of the current block. For example, when the current block is a non-square block of size 16×4, a secondary transform may be performed on the sub-block of size 2×8 at the upper left corner of the current block after performing the primary transform on the current block (see Figure 37 (a)).

[0542] When the height of the current block is more than 4 times the width of the non-square block, after the main transform is performed, the secondary transform can be performed on the sub-block of size (4kN)×(kN) in the upper left corner of the current block. For example, when the current block is a non-square block of size 16×4, the secondary transform can be performed on the sub-block of size 2×8 in the upper left corner of the current block after the main transform is performed on the current block (see Figure 37 (b)).

[0543] The decoder may perform an inverse transform of the secondary transform (second inverse transform), and perform an inverse transform of the primary transform (first inverse transform) on the result thereof. As a result of performing the second inverse transform and the first inverse transform, a residual signal of the current block may be obtained.

[0544] Information indicating the transform type of the current block may be signaled through a code stream, and may be index information tu_mts_idx indicating one of a combination of a horizontal transform type and a vertical transform type.

[0545] A transform kernel in a vertical direction and a transform kernel in a horizontal direction may be determined based on the transform type candidate specified according to the index information tu_mts_idx. Tables 11 and 12 show transform type combinations according to tu_mts_idx.

[0546]

Table 11

[0547]

[0548]

Table 12

[0549]

[0550] The transform type may be determined to be any one of DCT2, DST7, DCT8, or transform skip. Alternatively, in addition to transform skipping, transform type combination candidates may be constructed using only transform kernels.

[0551] When using Table 11, when tu_mts_idx is 0, transform skipping can be applied in the horizontal and vertical directions. If tu_mts_idx is 1, DCT2 can be applied in the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 can be applied in the horizontal direction and DCT7 can be applied in the vertical direction.

[0552] In the case of using Table 12, when tu_mts_idx is 0, DCT2 can be applied in the horizontal and vertical directions. If tu_mts_idx is 1, transform skip can be applied in the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 can be applied in the horizontal direction and DCT7 can be applied in the vertical direction.

[0553] Whether to encode index information can be determined based on at least one of the size, shape, or number of non-zero coefficients of the current block. For example, when the number of non-zero coefficients is equal to or less than a threshold, index information is not signaled, and a default transform type can be applied to the current block. The default transform type can be DST7. Alternatively, the default mode may differ depending on the size, shape, or intra-prediction mode of the current block.

[0554] The threshold value may be determined based on the size or shape of the current block. For example, when the size of the current block is less than or equal to 32×32, the threshold value may be set to 2, and when the current block is larger than 32×32 (for example, when the current block is a coding block of size 32×64 or 64×32), the threshold value may be set to 4.

[0555] A plurality of lookup tables may be pre-stored in the encoder / decoder. In the plurality of lookup tables, at least one of index values assigned to transform type combination candidates, types of transform type combination candidates, or the number of transform type combination candidates may be different.

[0556] The lookup table of the current block may be selected based on at least one of the size, shape, prediction mode, intra prediction mode, whether a second transform is applied, or whether a transform is skipped to be applied to a neighboring block of the current block.

[0557] For example, when the size of the current block is 4×4 or smaller or the current block is encoded by inter-frame prediction, the lookup table in Table 11 can be used, and when the size of the current block is greater than 4×4 or when the current block is encoded by intra-block prediction, the lookup table in Table 12 can be used.

[0558] Alternatively, information indicating any one of a plurality of lookup tables may be signaled through a code stream, and a decoder may select a lookup table for a current block based on the information.

[0559] As another example, the index assigned to the transform type combination candidate may be adaptively determined based on the size, shape, prediction mode, intra-prediction mode, whether the current block is transformed for the second time, or whether transform skipping is applied to at least one of the adjacent blocks. For example, when the size of the current block is 4×4, the index assigned to transform skipping may have a smaller value than the index assigned to transform skipping when the size of the current block is greater than 4×4. Specifically, when the size of the current block is 4×4, index 0 may be assigned to transform skipping, and when the current block is greater than 4×4 and less than 16×16, an index greater than 0 (e.g., index 1) may be assigned to transform skipping. When the current block is greater than 16×16, a maximum value (e.g., 5) may be assigned as the index of transform skipping.

[0560] Alternatively, when the current block is encoded with inter prediction, the transform skip may be assigned an index of 0. When the current block is encoded with intra prediction, an index greater than 0 (eg, index 1) may be assigned to the transform skip.

[0561] Alternatively, when the current block is a 4×4 block coded with inter-frame prediction, the transform skip may be assigned an index of 0. On the other hand, when the current block is not coded with inter-frame prediction or the current block is larger than 4×4, an index greater than 0 (e.g., index 1) may be assigned to the transform skip.

[0562] Transform type combination candidates different from those listed in Tables 11 and 12 may be defined and used. For example, a transform type combination candidate to which a transform kernel such as DCT7, DCT8, or DST2 is applied may be used to apply transform skipping to horizontal or vertical transforms and other transforms. In this case, whether to use transform skipping as a transform type candidate for the horizontal or vertical direction may be determined based on at least one of the size (e.g., width and / or height), shape, prediction mode, or intra prediction mode of the current block.

[0563] Alternatively, information indicating whether a specific transform type candidate is available can be signaled via the codestream. For example, a flag indicating whether transform skipping can be used as a transform type candidate for both the horizontal and vertical directions can be signaled. Based on this flag, it can be determined whether the specific transform type combination candidate is included in multiple transform type combination candidates.

[0564] Alternatively, information indicating whether a transform type candidate is applied to the current block may be signaled via the codestream. For example, a flag cu_mts_flag indicating whether DCT2 is applied to both the horizontal and vertical directions may be signaled. When the value of cu_mts_flag is 1, DCT2 may be used as the transform kernel for both the vertical and horizontal directions. When the value of cu_mts_flag is 0, DCT8 or DST7 may be used as the transform kernel for both the vertical and horizontal directions. Alternatively, when the value of cu_mts_flag is 0, information tu_mts_idx specifying any one of multiple transform type combination candidates may be signaled.

[0565] When the current block is a non-square block with a width greater than its height or a non-square block with a height greater than its width, the encoding of cu_mts_flag may be omitted and the value of cu_mts_flag may be regarded as 0.

[0566] The number of available transform type combination candidates can be set differently depending on the size, shape, or intra prediction mode of the current block. For example, when the current block is square, more than three transform type combination candidates can be used, and when the current block is non-square, two transform type combination candidates can be used. Alternatively, when the current block is square, only transform type combination candidates that differ in horizontal and vertical transform types can be used.

[0567] When there are three or more transform type combination candidates available for the current block, index information tu_mts_idx indicating one of the transform type combination candidates may be signaled. On the other hand, when there are two transform type combination candidates available for the current block, a flag mts_flag indicating any one of the transform type combination candidates may be signaled. Table 13 below shows a process for encoding information specifying transform type combination candidates according to the shape of the current block.

[0568]

Table 13

[0569]

[0570]

[0571] The indices of the transform type combination candidates may be rearranged (or reordered) based on the shape of the current block. For example, the indices assigned to the transform type combination candidates when the current block is square may be different from the indices assigned to the transform type combination candidates when the current block is non-square. For example, when the current block is square, a transform type combination may be selected based on the following Table 14, and when the current block is non-square, a transform type combination may be selected based on the following Table 15.

[0572]

Table 14

[0573]

[0574]

Table 15

[0575]

[0576] The transform type can be determined based on the number of horizontal non-zero coefficients or the number of vertical non-zero coefficients of the current block. The number of horizontal non-zero coefficients represents the number of non-zero coefficients included in 1×N (where N is the width of the current block), and the number of vertical non-zero coefficients represents the number of non-zero coefficients included in N×1 (where N is the height of the current block). When the maximum value of the horizontal non-zero coefficients is less than or equal to a threshold, the primary transform type can be applied in the horizontal direction. When the maximum value of the horizontal non-zero coefficients is greater than the threshold, the secondary transform type can be applied in the horizontal direction. When the maximum value of the vertical non-zero coefficients is less than or equal to the threshold, the primary transform type can be applied in the vertical direction. When the maximum value of the vertical non-zero coefficients is greater than the threshold, the secondary transform type can be applied in the vertical direction.

[0577] Figure 38 is a diagram for describing an example of determining a transform type of a current block.

[0578] For example, when the current block is encoded by intra prediction and the maximum value of the horizontal non-zero coefficients of the current block is 2 or less (see Figure 38 (a)), the transform type in the horizontal direction can be determined as DST7.

[0579] When the current block is encoded by intra prediction and the maximum value of the vertical non-zero coefficients of the current block is greater than 2 (see Figure 38 (b)), the transform type in the vertical direction may be determined to be DCT2 or DCT8.

[0580] Information indicating whether the transform type of the current block is explicitly determined based on information signaled from the codestream may be signaled via the codestream. For example, information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for blocks encoded using intra-frame prediction and / or information sps_explicit_inter_mts_flag indicating whether explicit transform type determination is allowed for blocks encoded using inter-frame prediction may be signaled at the sequence level.

[0581] When explicit transform type determination is allowed, the transform type of the current block can be determined based on index information tu_mts_idx signaled from the codestream. On the other hand, when explicit transform type determination is not allowed, the transform type can be determined based on at least one of the current block's size, its shape, whether sub-block transforms are allowed, and the position of sub-blocks containing non-zero transform coefficients. For example, the horizontal transform type of the current block can be determined based on its width, and the vertical transform type of the current block can be determined based on its height. For example, when the width of the current block is less than 4 or greater than 16, the horizontal transform type can be determined as DCT2. Otherwise, the horizontal transform type can be determined as DST7. When the height of the current block is less than 4 or greater than 16, the vertical transform type can be determined as DCT2. Otherwise, the vertical transform type can be determined as DST7. Here, to determine the horizontal and vertical transform types, a threshold value compared to the width and height can be determined based on at least one of the current block's size, shape, and intra-prediction mode.

[0582] Alternatively, when the current block is a square with the same height and width, the horizontal transform type and the vertical transform type are set to be the same, while when the current block is a non-square with different heights and widths, the horizontal transform type and the vertical transform type can be set to be different from each other. For example, when the width of the current block is greater than the height, the horizontal transform type can be determined as DST7, and the vertical transform type can be determined as DCT2. When the height of the current block is greater than the width, the vertical transform type can be determined as DST7, and the horizontal transform type can be determined as DCT2.

[0583] Depending on whether explicit transform type determination is permitted, the number and / or types of transform type candidates, or the number and / or types of transform type combination candidates, may differ. For example, when explicit transform type determination is permitted, DCT2, DST7, and DCT8 may be used as transform type candidates. Therefore, each of the horizontal transform type and the vertical transform type may be set to DCT2, DST8, or DCT8. If explicit transform type determination is not permitted, only DCT2 and DST7 may be used as transform type candidates. Therefore, each of the horizontal transform type and the vertical transform type may be determined to be DCT2 or DST7.

[0584] If the encoder performs transformation and quantization, the decoder can obtain a residual block by inverse quantization and inverse transformation. The decoder adds the predicted block and the residual block to obtain a reconstructed block of the current block.

[0585] If a reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss during quantization and encoding. In-loop filtering can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before the in-loop filter is applied is referred to as the first reconstructed block, and the reconstructed block after the in-loop filter is applied is referred to as the second reconstructed block.

[0586] At least one of a deblocking filter, SAO, or ALF may be applied to the first reconstructed block to obtain the second reconstructed block. In this case, SAO or ALF may be applied after the deblocking filter is applied.

[0587] The deblocking filter is used to alleviate image quality degradation (blocking artifact) generated at block boundaries when quantization is performed in block units. To apply the deblocking filter, a block strength (BS) between a first reconstructed block and adjacent reconstructed blocks may be determined.

[0588] Figure 39 is a flow chart illustrating a process for determining block strength.

[0589] exist Figure 39 In the example shown, P represents the first reconstructed block, and Q represents the adjacent reconstructed block, wherein the adjacent reconstructed block may be adjacent to the left side or the top side of the current block.

[0590] exist Figure 39 In the illustrated example, block strength is determined considering prediction modes of P and Q, whether non-zero transform coefficients are included, whether inter-frame prediction is performed using the same reference image, and whether the difference in motion vectors is greater than or equal to a threshold.

[0591] Based on the block strength, it can be determined whether a deblocking filter is applied. For example, when the block strength is 0, no filtering may be performed.

[0592] SAO is used to alleviate the ringing artifact generated when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. The offset determination method includes edge offset (EO) or band offset (Band Offset). EO represents a method of determining the offset of the current sample based on the pattern of the surrounding pixels. BO represents a method of applying a common offset to a set of pixels with similar brightness values in an area. Specifically, the pixel brightness is divided into 32 equal intervals, and pixels with similar brightness values are set as a set. For example, four adjacent bands among the 32 bands are set as a group, and the same offset can be applied to the samples belonging to the four bands.

[0593] ALF is a method of applying a filter of a predefined size / shape to a first reconstructed image or a reconstructed image to which a deblocking filter is applied to generate a second reconstructed image. The following Equation 16 shows an example of applying ALF.

[0594] Equation 16

[0595]

[0596] Any of the predefined filter candidates may be selected in units of a picture, a coding tree unit, a coding block, a prediction block, or a transform block. The filter candidates may have different sizes or shapes.

[0597] Figure 40 is a diagram showing predefined filter candidates.

[0598] As in Figure 40 In the example shown, at least one of 5×5, 7×7, and 9×9 diamond shapes may be selected.

[0599] Only diamonds of size 5×5 can be used for chroma components.

[0600] The invention also includes the following aspects: applying the embodiments described with the decoding process or encoding process as the center to the encoding process or decoding process; and changing the embodiments described in the predetermined order to an order different from the order described.

[0601] The embodiments have been described based on a series of steps or flow charts, but this does not limit the chronological order of the invention, and can be executed simultaneously or in another order as needed. In addition, in the above-mentioned embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagram can also be implemented as hardware devices or software respectively, and multiple structural elements can be combined and implemented as a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc. alone or in combination. Examples of computer-readable recording media can include magnetic media such as hard disks, floppy disks and tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and ROMs, RAMs, flash memories, etc., which are hardware devices specially configured to store program instructions and execute the instructions. The hardware device can be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0602] Industrial Applicability

[0603] The present invention can be applied to electronic devices that encode / decode videos.

Claims

1. A video decoding method comprising the following steps: determining whether to apply the combined prediction mode to the current block; When the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block, wherein the first prediction block is obtained based on motion information of a merge candidate of the current block and the second prediction block is obtained based on an intra prediction mode of the current block; and Obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, wherein, when the combined prediction mode is applied to the current block, triangulation is disabled for the current block, wherein the intra prediction mode of the current block is a planar mode, Wherein, when performing the weighted sum operation, the weighted values applied to the first prediction block and the second prediction block are determined based on the prediction modes of the neighboring blocks adjacent to the current block.

2. A video encoding method comprising the following steps: determining whether to apply the combined prediction mode to the current block; When the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block, wherein the first prediction block is obtained based on motion information of a merge candidate of the current block and the second prediction block is obtained based on an intra prediction mode of the current block; and Obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, wherein, when the combined prediction mode is applied to the current block, triangulation is disabled for the current block, The intra prediction mode of the current block is planar mode. Wherein, when performing the weighted sum operation, the weighted values applied to the first prediction block and the second prediction block are determined based on the prediction modes of the neighboring blocks adjacent to the current block.

3. A video decoding device, comprising a prediction unit, the prediction unit configured to: determine whether to apply a combined prediction mode to a current block; obtain a first prediction block and a second prediction block of the current block when the combined prediction mode is applied to the current block; and obtain a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, in, obtaining the first prediction block based on motion information of a merge candidate of the current block and obtaining the second prediction block based on an intra prediction mode of the current block, disabling triangulation for the current block when the combined prediction mode is applied to the current block, The intra prediction mode of the current block is planar mode. Wherein, when performing the weighted sum operation, the weighted values applied to the first prediction block and the second prediction block are determined based on the prediction modes of the neighboring blocks adjacent to the current block.

4. A video encoding device, comprising a prediction unit, the prediction unit configured to: determine whether to apply a combined prediction mode to a current block; obtain a first prediction block and a second prediction block of the current block when the combined prediction mode is applied to the current block; and obtain a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, in, obtaining the first prediction block based on motion information of a merge candidate of the current block and obtaining the second prediction block based on an intra prediction mode of the current block, disabling triangulation for the current block when the combined prediction mode is applied to the current block, The intra prediction mode of the current block is planar mode. Wherein, when performing the weighted sum operation, the weighted values applied to the first prediction block and the second prediction block are determined based on the prediction modes of the neighboring blocks adjacent to the current block.

5. A video decoder comprising a processor and a memory configured to store a computer program capable of running on the processor, wherein the processor is configured to: determine whether to apply a combined prediction mode to a current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block; and Obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, in, obtaining the first prediction block based on motion information of a merge candidate of the current block and obtaining the second prediction block based on an intra prediction mode of the current block, wherein, when the combined prediction mode is applied to the current block, triangulation is disabled for the current block, The intra prediction mode of the current block is planar mode. Wherein, when performing the weighted sum operation, the weighted values applied to the first prediction block and the second prediction block are determined based on the prediction modes of the neighboring blocks adjacent to the current block.

6. A video encoder comprising a processor and a memory configured to store a computer program capable of running on the processor, wherein the processor is configured to: determine whether to apply a combined prediction mode to a current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block; and Obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, in, obtaining the first prediction block based on motion information of a merge candidate of the current block and obtaining the second prediction block based on an intra prediction mode of the current block, wherein, when the combined prediction mode is applied to the current block, triangulation is disabled for the current block, The intra prediction mode of the current block is planar mode. Wherein, when performing the weighted sum operation, the weighted values applied to the first prediction block and the second prediction block are determined based on the prediction modes of the neighboring blocks adjacent to the current block.

7. A storage medium storing an executable program, wherein: When the executable program is executed by a processor, the method according to claim 1 is performed.

8. A storage medium storing an executable program, wherein: When the executable program is executed by a processor, the method according to claim 2 is performed.

Images