Video signal encoding / decoding method and apparatus therefor

Abstract

A video decoding method according to the present invention comprises the steps of: determining whether to apply a combined prediction mode to a current block; when the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block; and obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block.

Description

Video signal encoding / decoding method and apparatus for said method

[0001] Divisional Description This application is a divisional application of application filed on November 7, 2019, with application number 201980070621.8 and invention title "Video Signal Encoding / Decoding Method and Apparatus for the Method". Technical Field

[0002] This invention relates to a video signal encoding / decoding method and an apparatus for the method. Background Technology

[0003] With the trend of increasingly larger display panels, there is a growing need for higher-quality video services. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, research is actively underway to improve video compression rates. As a representative example, in 2009, the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013, and its compression performance is approximately twice that of H.264 / AVC. However, with the rapid development of high-definition video services, the limitations of HEVC have gradually become apparent. Summary of the Invention

[0004] The technical problem to be solved: The purpose of this invention is to provide a combined prediction method that combines multiple prediction methods when encoding / decoding video signals, and an apparatus for performing the method.

[0005] The purpose of this invention is to provide a method for dividing a coding block into multiple prediction units when encoding / decoding a video signal, and an apparatus for performing the method.

[0006] The technical problems to be solved by the present invention are not limited to those mentioned above, and those skilled in the art to which the present invention pertains will clearly understand other technical problems not mentioned through the following description.

[0007] The technical solution of the video signal decoding / encoding method according to the present invention includes the following steps: determining whether to apply a combined prediction mode to the current block; when the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block; and obtaining a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block. In this case, the first prediction block can be obtained based on the motion information of the merging candidates of the current block, and the second prediction block can be obtained based on the intra-frame prediction mode of the current block.

[0008] In the video signal decoding / encoding method according to the present invention, when the combined prediction mode is applied to the current block, it can be configured to disable triangle partitioning for the current block.

[0009] In the video signal decoding / encoding method according to the present invention, the intra-frame prediction mode of the current block can be set to a planar mode.

[0010] In the video signal decoding / encoding method according to the present invention, the second prediction block can be obtained based on the reference sample lines included in adjacent reference sample lines.

[0011] In the video signal decoding / encoding method according to the present invention, when performing the weighted sum operation, the weighting value applied to the first prediction block and the second prediction block can be determined based on the prediction mode of the neighboring blocks adjacent to the current block.

[0012] In the video signal decoding / encoding method according to the present invention, when at least one of the width or height of the current block is greater than a threshold, the combined prediction mode may not be applied to the current block.

[0013] In the video signal decoding / encoding method according to the present invention, when the flag indicating that a merging mode should be applied to the current block is true, it can be configured to apply the combined prediction mode to the current block.

[0014] The features briefly outlined above are merely exemplary embodiments of the invention as described in the detailed description to follow, and do not limit the scope of the invention.

[0015] According to the present invention, by providing a combined prediction method that combines multiple prediction methods, the efficiency of inter-frame prediction can be improved.

[0016] According to the present invention, inter-frame prediction efficiency can be improved by providing a method that divides a coded block into multiple prediction blocks and derives motion information of each prediction block in the multiple prediction blocks.

[0017] The effects that can be obtained in this invention are not limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art through the following description. Attached Figure Description

[0018] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0019] Figure 2 is a block diagram of the video decoder according to an embodiment of the present invention.

[0020] Figure 3 is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0021] Figure 4 is a diagram showing the various partitioning types of coded blocks.

[0022] Figure 5 is a diagram showing an example of the partitioning of a coding tree unit.

[0023] Figure 6 is a diagram showing the shape of the basic data unit.

[0024] Figures 7 and 8 are diagrams illustrating examples of dividing a coded block into multiple sub-blocks.

[0025] Figure 9 is a flowchart of an inter-frame prediction method according to an embodiment of the present invention.

[0026] Figure 10 is a flowchart of the process of exporting motion information of the current block in merge mode.

[0027] Figure 11 is a diagram showing an example of candidate blocks used to derive merge candidates.

[0028] Figure 12 is a diagram illustrating an example of dividing a coding block into multiple prediction units using diagonals.

[0029] Figure 13 is a diagram illustrating an example of dividing a coding block into two prediction units.

[0030] Figure 14 shows an example of dividing a coded block into multiple prediction blocks of different sizes.

[0031] Figure 15 is a diagram illustrating an example of applying diagonal division to a transformation unit.

[0032] Figure 16 is a diagram showing adjacent blocks used to derive triangle merging candidates.

[0033] Figure 17 is a diagram illustrating an example of determining the availability of neighboring blocks for each triangular prediction unit.

[0034] Figures 18 and 19 are examples illustrating how to derive a prediction sample based on a weighted sum of a first and a second prediction sample.

[0035] Figure 20 is a flowchart illustrating an intra-frame prediction method according to an embodiment of the present invention.

[0036] Figure 21 is a diagram illustrating the intra-frame prediction mode.

[0037] Figures 22 and 23 are diagrams illustrating an example of a one-dimensional array in which reference samples are arranged in a row.

[0038] Figure 24 is a diagram showing the angle formed between the intra-angle prediction mode and a straight line parallel to the x-axis.

[0039] Figure 25 is a diagram illustrating an example of obtaining a predicted sample when the current block is not a square.

[0040] Figure 26 is a diagram illustrating the wide-angle intra-frame prediction mode.

[0041] Figure 27 is a diagram illustrating an embodiment of the application of PDPC.

[0042] Figure 28 is a diagram illustrating an example of applying weighted values.

[0043] Figure 29 is a diagram illustrating an example of assigning different weights to sub-blocks based on whether the merging candidate has bidirectional motion information.

[0044] Figure 30 is a diagram illustrating an example of assigning different weights to sub-blocks based on the intra-prediction mode of the current block.

[0045] Figure 31 is a diagram illustrating an example of specifying a second merge candidate by considering the search order of candidate blocks.

[0046] Figure 32 is a diagram illustrating an example of selecting a first and second merge candidate from merge candidates derived from non-adjacent blocks.

[0047] Figure 33 is a diagram illustrating an example of determining a second merger candidate.

[0048] Figure 34 is a diagram illustrating an example of determining the weighting applied to the prediction block based on the shape of the candidate block.

[0049] Figure 35 is a diagram illustrating an example of a replacement merge candidate.

[0050] Figures 36 and 37 are diagrams showing the sub-blocks to which a second transformation will be performed.

[0051] Figure 38 is a diagram illustrating an example of determining the transformation type of the current block.

[0052] Figure 39 is a flowchart illustrating the process of determining block strength.

[0053] Figure 40 is a diagram showing predefined filter candidates. Detailed Implementation

[0054] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0055] Video encoding and decoding are performed on a block-by-block basis. For example, encoding / decoding processes such as transform, quantization, prediction, loop filtering, or reconstruction can be performed on encoded blocks, transform blocks, or prediction blocks.

[0056] Hereinafter, the block to be encoded / decoded will be referred to as the "current block". For example, depending on the current encoding / decoding process step, the current block can represent an encoded block, a transform block, or a prediction block.

[0057] Additionally, as used herein, the term "unit" refers to a basic unit used to perform a specific encoding / decoding process, and "block" can be understood as representing a sample array of a predetermined size. Unless otherwise stated, "block" and "unit" are used interchangeably. For example, in the embodiments described later, encoding block and encoding unit can be understood to have the same meaning.

[0058] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0059] Referring to FIG1, the video encoding device 100 may include an image segmentation unit 110, a prediction unit 120, 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy coding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0060] The components shown in Figure 1 are illustrated individually to represent the distinct functionalities of the video encoding device and do not imply that each component is composed of separate hardware or a single software component. That is, for ease of explanation, the components are arranged such that at least two components are combined into one component or one component is divided into multiple components to perform functions. Such embodiments of integrated components and embodiments of separated components are also within the scope of this invention, provided they do not depart from its spirit.

[0061] Furthermore, some structural elements are not essential structural elements for performing the essential functions of this invention, but rather optional structural elements used only to improve performance. This invention can be implemented by including only the components necessary for realizing the essence of the invention, excluding the structural elements used only to improve performance, and structures including only the essential structural elements, excluding the optional structural elements used only to improve performance, are also within the scope of this invention.

[0062] The image partitioning unit 110 can divide the input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transformation unit (TU), or a coding unit (CU). The image partitioning unit 110 divides an image into a combination of multiple coding units, prediction units, and transformation units, and can select a combination of coding units, prediction units, and transformation units to encode the image based on a predetermined criterion (e.g., a cost function).

[0063] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quadtree structure can be used. A video or the largest coding unit can be used as the root, and the coding unit can be divided into additional coding units with a number of child nodes equivalent to the number of coding units in the division. Coding units that are no longer divided according to certain constraints become leaf nodes. That is, when it is assumed that a coding unit can only be divided into squares, a coding unit can be divided into a maximum of four other coding units.

[0064] In the embodiments of the present invention, the encoding unit may mean a unit that performs encoding, or it may mean a unit that performs decoding.

[0065] Prediction units within a coding unit can be divided into at least one shape of the same size, such as squares or rectangles, or a prediction unit within a coding unit can be divided into units with different shapes and / or sizes than another prediction unit.

[0066] Intra-prediction can be performed when the prediction unit for intra-prediction based on the coding unit is not the smallest coding unit, without having to divide it into multiple prediction units N×N.

[0067] Prediction units 120 and 125 may include an inter-frame prediction unit 120 that performs inter-frame prediction and an intra-frame prediction unit 125 that performs intra-frame prediction. It can be determined whether inter-frame prediction or intra-frame prediction is used for the prediction unit, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) is determined based on each prediction method. In this case, the processing unit performing the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction unit may determine the prediction method and prediction mode, and the transformation unit may perform the prediction. The residual value (residual block) between the generated prediction block and the original block can be input to the transformation unit 130. Furthermore, the prediction mode information, motion vector information, etc., used for prediction, along with the residual value, can be encoded in the entropy coding unit 165 and transmitted to the decoder. When using a specific coding mode, the original block can also be directly encoded and transmitted to the decoder without generating a prediction block through the prediction units 120 and 125.

[0068] The inter-frame prediction unit 120 can predict prediction units based on information from at least one image preceding or following the current image. In some cases, it can also predict prediction units based on information from a portion of the encoded region within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0069] The reference image interpolation unit receives reference image information from memory 155 and can generate pixel information of integer pixels or fractional pixels from the reference image. For luminance pixels, in order to generate pixel information of fractional pixels in 1 / 4 pixel units, an 8-tap DCT-based interpolation filter with different filter coefficients can be used. For chrominance signals, in order to generate pixel information of fractional pixels in 1 / 8 pixel units, a 4-tap DCT-based interpolation filter with different filter coefficients can be used.

[0070] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as the Full Search-based Block Matching Algorithm (FBMA), the Three-Step Search (TSS), and the New Three-Step Search Algorithm (NTS). Motion vectors can have values ​​in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. Different motion prediction methods can be used in the motion prediction unit to predict the current prediction unit. These methods include skipping, merging, Advanced Motion Vector Prediction (AMVP), and Intra Block Copying.

[0071] The intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block, which serves as pixel information within the current image. When the neighboring block of the current prediction unit is a block that has already undergone inter-frame prediction, and the reference pixel is a pixel that has undergone inter-frame prediction, the reference pixel included in the block that has undergone inter-frame prediction can be used as the reference pixel information for the surrounding block that has undergone intra-frame prediction. That is, when a reference pixel is unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0072] In intra-frame prediction, the prediction mode can have an angular prediction mode that uses reference pixel information in the prediction direction and a non-angular mode that does not use direction information when performing prediction. The mode used to predict luminance information and the mode used to predict chrominance information can be different. To predict chrominance information, either the intra-frame prediction mode information used for predicting luminance information or the predicted luminance signal information can be applied.

[0073] When performing intra-frame prediction, if the size of the prediction unit is the same as the size of the transform unit, intra-frame prediction can be performed based on pixels to the left, upper left, and upper right of the prediction unit. However, when performing intra-frame prediction, if the size of the prediction unit is different from the size of the transform unit, intra-frame prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-frame prediction using an N×N partition only for the smallest coding unit can be applied.

[0074] Intra-prediction methods can generate prediction blocks after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel based on the prediction mode. The type of AIS filter used for the reference pixel may vary. To perform intra-prediction, the intra-prediction mode of the current prediction unit can be predicted from the intra-prediction modes of prediction units existing in the vicinity of the current prediction unit. When using mode information predicted from surrounding prediction units to predict the prediction mode of the current prediction unit, if the intra-prediction modes of the current prediction unit and those of the surrounding prediction units are the same, predetermined flag information can be used to convey information indicating that the prediction modes of the current prediction unit and those of the surrounding prediction units are the same. If the prediction modes of the current prediction unit and those of the surrounding prediction units are different, entropy coding can be performed to encode the prediction mode information of the current block.

[0075] Furthermore, residual blocks including residual information can be generated, the residual information being the difference between the prediction unit that performs prediction based on the prediction unit generated in prediction units 120 and 125 and the original block of the prediction unit. The generated residual blocks can be input to the transformation unit 130.

[0076] In the transform unit 130, the residual block can be transformed using a transform method such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST). The residual block includes residual information between the original block and the prediction units generated by the prediction units 120 and 125. The DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on the intra-frame prediction mode information of the prediction units used to generate the residual block. Transformation of the residual block can also be skipped. A flag indicating whether to skip the transformation of the residual block can be encoded. Transformation skipping is allowed for residual blocks, luma components, or chroma components (4:4:4 format or below) with a size below a threshold.

[0077] The quantization unit 135 can quantize the values ​​that have been transformed into the frequency domain in the transformation unit 130. The quantization coefficients can be changed according to the importance of the block or video. The values ​​calculated in the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0078] The rearrangement unit 160 can rearrange the coefficient values ​​of the quantized residual values.

[0079] The rearrangement unit 160 can transform 2D block shape coefficients into 1D vector form using a coefficient scanning method. For example, the rearrangement unit 160 can use a zig-zag scan method to scan the DC coefficients and even the coefficients in the high-frequency domain, and transform them into 1D vector form. Depending on the size of the transform unit and the intra-frame prediction mode, instead of zig-zag scanning, vertical scanning along the column direction and horizontal scanning along the row direction can also be used to scan the 2D block shape coefficients. That is, the choice between zig-zag scanning, vertical scanning, and horizontal scanning can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0080] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. For example, entropy coding can use various coding methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0081] The entropy coding unit 165 can encode various information such as residual coefficient information, block type information, prediction mode information, partitioning unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding units originating from the rearrangement unit 160 and the prediction units 120 and 125.

[0082] The coefficient values ​​of the coding units input from the rearrangement unit 160 can be entropy encoded in the entropy coding unit 165.

[0083] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the multiple values ​​quantized in the quantization unit 135, and perform inverse transform on the values ​​transformed in the transform unit 130. The residual values ​​generated in the inverse quantization unit 140 and the inverse transform unit 145 can be merged with the prediction units predicted by the motion prediction unit, motion compensation unit and intra-frame prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0084] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0085] Deblocking filters remove block distortion generated in the reconstructed image due to the boundaries between blocks. To determine whether to perform deblocking, the number of pixels in the columns or rows included in the block can be used to decide whether to apply a deblocking filter to the current block. When applying a deblocking filter to a block, a strong or weak filter can be applied depending on the desired deblocking intensity. Furthermore, during the use of deblocking filters, horizontal and vertical filtering can be processed simultaneously when performing vertical and horizontal filtering.

[0086] The offset correction unit can correct the offset between the video being deblocked and the original video on a pixel-by-pixel basis. To perform offset correction on a specific image, the following methods can be used: after dividing the pixels included in the video into a predetermined number of regions, determine the region to be offset and apply the offset to the corresponding region, or apply the offset by taking into account the edge information of each pixel.

[0087] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered reconstructed image and the original video. After dividing the pixels in the video into predetermined groups, filtering can be performed differently for each group by determining a filter to be used for the corresponding group. Information related to whether adaptive loop filtering is applied, along with luminance signals, can be transmitted per coding unit (CU). The shape and filter coefficients of the adaptive loop filter to be applied can vary depending on the block. Furthermore, it is possible to apply the same type (fixed type) of adaptive loop filter regardless of the characteristics of the block to which it is applied.

[0088] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the prediction units 120 and 125 when performing inter-frame prediction.

[0089] Figure 2 is a block diagram of the video decoder according to an embodiment of the present invention.

[0090] Referring to FIG2, the video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0091] When inputting a video stream from a video encoder, the input stream can be decoded by following the reverse steps of the video encoder.

[0092] The entropy decoding unit 210 can perform entropy decoding in the reverse order of the entropy encoding steps performed in the entropy encoding unit of the video encoder. For example, corresponding to the methods performed in the video encoder, various methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied.

[0093] The entropy decoding unit 210 can decode information related to intra-frame prediction and inter-frame prediction performed by the encoder.

[0094] The rearrangement unit 215 can perform rearrangement based on a method used in the encoding unit to rearrange the bitstream that has been entropily decoded by the entropy decoding unit 210. Rearrangement can be performed by reconstructing multiple coefficients represented in 1D vector form into 2D block-shaped coefficients. The rearrangement unit 215 receives information related to the coefficient scan performed in the encoding unit and can perform rearrangement by performing a reverse scan based on the scan order performed in the corresponding encoding unit.

[0095] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged blocks.

[0096] The inverse transform unit 225 can perform inverse discrete cosine transform and inverse discrete sine transform on the quantization result performed by the video encoder. These inverse discrete cosine transforms and inverse discrete sine transforms are inverse transforms of the transforms performed in the transform unit, i.e., inverse transforms of the discrete cosine transform and discrete sine transform. The DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform unit 225 may not perform the inverse transform. The inverse transform can be performed based on the transmission unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, a transform method (e.g., DCT or DST) can be selectively performed based on multiple pieces of information such as the prediction method, the size of the current block, and the prediction direction.

[0097] Prediction units 230 and 235 can generate prediction blocks based on information related to prediction block generation provided by entropy decoding unit 210 and previously decoded block or image information provided by memory 245.

[0098] As described above, when intra-prediction is performed in the same manner as in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra-prediction is performed on the prediction unit based on the pixels to its left, the pixels to its upper left, and the pixels above it. If the size of the prediction unit is different from the size of the transform unit, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using only N×N partitioning for the smallest coding unit can also be applied.

[0099] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit receives various information input from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method. It classifies prediction units according to the current coding unit and determines whether the prediction unit is performing inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 can use the information required for inter-frame prediction of the current prediction unit provided by the video encoder and perform inter-frame prediction on the current prediction unit based on information included in at least one of the previous or subsequent images of the current image to which the current prediction unit belongs. Alternatively, inter-frame prediction can also be performed based on information from a portion of the reconstructed region within the current image to which the current prediction unit belongs.

[0100] In order to perform inter-frame prediction, it is possible to determine, based on the coding unit, which of the following modes of motion prediction method is used for the prediction units included in the corresponding coding unit: Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), or Intra-Block Copy Mode.

[0101] The intra-prediction unit 235 can generate prediction blocks based on pixel information within the current image. When the prediction unit is one that has already performed intra-prediction, intra-prediction can be performed based on the intra-prediction mode information of the prediction unit provided by the video encoder. The intra-prediction unit 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra-smoothing filter is the part that performs filtering on the reference pixels of the current block, and whether to apply the filter can be determined according to the prediction mode of the current prediction unit. Adaptive intra-smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided by the video encoder and the adaptive intra-smoothing filter information. If the prediction mode of the current block is a mode that does not perform adaptive intra-smoothing filtering, then the adaptive intra-smoothing filter may not be applied.

[0102] For the reference pixel interpolation unit, if the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on pixel values ​​interpolated from reference pixels, then reference pixels with integer or fractional pixel units can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, then interpolation of reference pixels is not required. If the prediction mode of the current block is DC mode, then the DC filter can generate prediction blocks by filtering.

[0103] The reconstructed blocks or images can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0104] Information related to whether to apply a deblocking filter to a corresponding block or image can be received from the video encoder, as well as information regarding whether to apply strong or weak filtering when applying the deblocking filter. Information related to the deblocking filter provided by the video encoder can be received from the video decoder's deblocking filter, and deblocking filtering can be performed on the corresponding block at the video decoder.

[0105] The offset correction unit can perform offset correction on the reconstructed video based on the type and amount of offset correction used during video encoding.

[0106] The ALF can be applied to the coding unit based on information provided by the encoder, such as whether the ALF is applied and ALF coefficient information. This ALF information can be provided by including it in a specific parameter set.

[0107] The memory 245 stores the reconstructed image or block, such that the image or block can be used as a reference image or reference block, and the reconstructed image can be provided to the output unit.

[0108] Figure 3 is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0109] The largest coding block can be defined as a coding tree block. An image can be divided into multiple coding tree units (CTUs). A coding tree unit is the largest coding unit and can also be called the largest coding unit (LCU). Figure 3 shows an example of dividing an image into multiple coding tree units.

[0110] The size of a coding tree unit can be defined at the image level or the sequence level. Therefore, information representing the size of a coding tree unit can be transmitted via signals using either an image parameter set or a sequence parameter set.

[0111] For example, the size of the coding tree unit for the entire image within the sequence can be set to 128 × 128. Alternatively, either 128 × 128 or 256 × 256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image can be set to 128 × 128, and the size of the coding tree unit in the second image can be set to 256 × 256.

[0112] Coded blocks can be generated by dividing the coding tree into units. A coded block represents a basic unit used for encoding / decoding processing. For example, prediction or transformation can be performed on different coded blocks, or prediction modes can be determined on different coded blocks. The prediction mode represents the method for generating the predicted image. For example, prediction modes can include intraprediction, interprediction, current picture referencing (CPR, or intra block copy (IBC)), or combined prediction. For a coded block, at least one of the prediction modes—intraprediction, interprediction, current picture referencing, or combined prediction—can be used to generate a prediction block associated with the coded block.

[0113] Information indicating the prediction mode of the current block can be transmitted via signals in the bitstream. For example, this information could be a 1-bit flag indicating whether the prediction mode is intra-frame or inter-frame. Current image reference or combined prediction can be used only if the prediction mode of the current block is determined to be inter-frame.

[0114] The current image reference is used to set the current image as the reference image and obtain the prediction block of the current block from the encoded / decoded region within the current image. Here, the current image means the image that includes the current block. Information indicating whether the current image reference is applied to the current block can be sent via signals in the bitstream. For example, this information could be a 1-bit flag. When the flag is true, the prediction mode of the current block can be determined as the current image reference; when the flag is false, the prediction mode of the current block can be determined as inter-frame prediction.

[0115] Alternatively, the prediction mode for the current block can be determined based on a reference image index. For example, when the reference image index points to the current image, the prediction mode for the current block can be determined as current image reference. When the reference image index points to another image instead of the current image, the prediction mode for the current block can be determined as inter-frame prediction. That is, current image reference is a prediction method that uses information from encoded / decoded regions within the current image, and inter-frame prediction is a prediction method that uses information from other encoded / decoded images.

[0116] Combinatorial prediction represents a coding mode composed of two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, when applying combinatorial prediction, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or the current image reference, and a second prediction block can be generated based on another. If a first and a second prediction block are generated, a final prediction block can be generated by averaging or weighted summing the first and second prediction blocks. Information indicating whether combinatorial prediction is applied can be transmitted via a signal in the bitstream. This information can be a 1-bit flag.

[0117] Figure 4 is a diagram showing the various partitioning types of coded blocks.

[0118] A coded block can be divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. Furthermore, the divided coded blocks can be further divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0119] Quadtree partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block can be divided into four square partitions (refer to 'SPLIT_QT' in Figure 4).

[0120] Binary tree partitioning refers to a partitioning technique that divides the current block into two blocks. The process of partitioning the current block along a vertical direction (i.e., using a vertical line crossing the current block) is called vertical binary tree partitioning, and the process of partitioning the current block along a horizontal direction (i.e., using a horizontal line crossing the current block) is called horizontal binary tree partitioning. After binary tree partitioning, the current block can be divided into two non-square partitions. In Figure 4, "SPLIT_BT_VER" represents the result of vertical binary tree partitioning, and "SPLIT_BT_HOR" represents the result of horizontal binary tree partitioning.

[0121] Ternary tree partitioning refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along a vertical direction (i.e., using two vertical lines crossing the current block) is called vertical ternary tree partitioning, and the process of dividing the current block into three blocks along a horizontal direction (i.e., using two horizontal lines crossing the current block) is called horizontal ternary tree partitioning. After ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. "SPLIT_TT_VER" in Figure 4 represents the result of vertical ternary tree partitioning, and "SPLIT_TT_HOR" in Figure 4 represents the result of horizontal ternary tree partitioning.

[0122] The number of partitions in a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or image level. Therefore, the maximum partitioning depth of a coding tree unit can vary depending on different sequences or images.

[0123] Alternatively, the maximum partitioning depth can be determined individually for each of the multiple partitioning techniques. For example, the maximum partitioning depth allowed for quadtree partitioning can be different from the maximum partitioning depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0124] The encoder can transmit information via signals from the bitstream representing at least one of the partition shape or partition depth of the current block. The decoder can determine the partition shape and partition depth of the coding tree unit based on the information parsed from the bitstream.

[0125] Figure 5 is a diagram showing an example of the partitioning of a coding tree unit.

[0126] The process of dividing coding blocks using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning is called multitree partitioning.

[0127] The coded blocks generated by applying a multi-way tree partitioning to the coded block can be called multiple downstream coded blocks. When the partitioning depth of the coded block is k, the partitioning depth of the multiple downstream coded blocks is set to k+1.

[0128] On the other hand, for multiple coding blocks with a partitioning depth of k+1, the coding block with a partitioning depth of k can be called the upstream coding block.

[0129] The partition type of the current coding block can be determined based on at least one of the partition shape of the upstream coding block or the partition type of the adjacent coding blocks. The adjacent coding blocks are adjacent to the current coding block and can include at least one of the current coding block's upper adjacent block, left adjacent block, or adjacent block to its upper left corner. The partition type can include at least one of whether to partition into a quadtree, whether to partition into a binary tree, the binary tree partition direction, whether to partition into a ternary tree, or the ternary tree partition direction.

[0130] To determine the shape of the coded block partition, information indicating whether the coded block has been partitioned can be sent via a signal in the bitstream. This information is a 1-bit flag "split_cu_flag," and when the flag is true, it indicates that the coded block has been partitioned using a multi-way tree partitioning technique.

[0131] When "split_cu_flag" is true, information indicating whether the coded block has been partitioned by a quadtree can be sent via a signal in the bitstream. This information is a 1-bit flag "split_qt_flag". When this flag is true, the coded block can be divided into 4 blocks.

[0132] For example, the example shown in Figure 5 illustrates the generation of four coded blocks with a partition depth of 1 by dividing the coding tree unit into quadtrees. Furthermore, it illustrates applying quadtree partitioning again to the first and fourth coded blocks generated as a result of the quadtree partitioning. Ultimately, four coded blocks with a partition depth of 2 can be generated.

[0133] Furthermore, a coded block with a partition depth of 3 can be generated by applying a quadtree partition to the coded block with a partition depth of 2 again.

[0134] When a quadtree partitioning is not applied to the coded block, it can be determined whether to perform a binary tree partitioning or a ternary tree partitioning on the coded block by considering at least one of the following: the size of the coded block, whether the coded block is located at an image boundary, the maximum partitioning depth, or the partitioning shape of adjacent blocks. When it is determined whether to perform a binary tree partitioning or a ternary tree partitioning on the coded block, information indicating the partitioning direction can be transmitted via a signal in the bitstream. This information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partitioning direction (vertical or horizontal) can be determined based on this flag. Additionally, information indicating whether a binary tree partitioning or a ternary tree partitioning is applied to the coded block can be transmitted via a signal in the bitstream. This information can be a 1-bit flag "mtt_split_cu_binary_flag". The binary tree partitioning or ternary tree partitioning can be determined based on this flag.

[0135] For example, the example shown in Figure 5 illustrates the application of a vertical binary tree partition to a coded block with a partition depth of 1, the application of a vertical ternary tree partition to the left coded block in the coded block generated as a result of the partition, and the application of a vertical binary tree partition to the right coded block.

[0136] When implementing devices for encoding or decoding video, there is a problem that hardware performance may be insufficient to handle regions larger than a threshold. For example, there may be a problem where, when hardware performance allows for processing up to 4096 samples simultaneously, 64 × 64 data units should be accessed and processed redundantly, and data in regions with more than 4096 samples cannot be processed simultaneously. As mentioned above, the basic unit of data processing can be defined as a pipelined basic data unit (virtual processing data unit, VPDU, hereinafter referred to as basic data unit).

[0137] Basic data units can be categorized into square, non-square, or non-rectangular types.

[0138] Figure 6 is a diagram showing the shape of the basic data unit.

[0139] The basic data unit can include samples that are equal to or less than the maximum number of samples that can be processed simultaneously. For example, as shown in the example of Figure 6(a), a 64 × 64 square block can be set as the basic data unit. Alternatively, a non-square block can be set as the basic data unit. For example, as shown in the examples of Figure 6(b) or 6(c), a 32 × 128 block or a 64 × 32 block can be set as the basic data unit.

[0140] Although not shown, basic data units such as triangles, L-shapes, or polygons can be defined.

[0141] Information for determining basic data units can be transmitted via a bitstream using signals. This information can be used to determine at least one of the size or shape of the basic data unit. Based on this information, it can be determined whether non-square basic data units are permitted.

[0142] Alternatively, at least one of the sizes or shapes of the basic data units can be predefined in the encoder and decoder.

[0143] The size of the basic data unit can be considered to determine whether the type of coded block partitioning is allowed. For example, when the resulting coded block is larger than the basic data unit, such partitioning may not be allowed. Alternatively, when the resulting non-square coded block is larger than the basic data unit, such partitioning may not be allowed. For example, binary or ternary tree partitioning may not be allowed when the width or height of the coded block exceeds a threshold, or when the number of samples included in the coded block exceeds a threshold. Therefore, the encoding of information related to binary or ternary tree partitioning can be omitted.

[0144] Alternatively, it can be configured to require that coded blocks larger than the basic data unit be partitioned. Alternatively, it can be configured to require that coded blocks larger than the basic data unit be partitioned into a binary tree or a ternary tree. Therefore, for coded blocks larger than the basic data unit, although the flag split_flag indicating whether the coded block has been partitioned is not encoded, the value of the flag can be derived as 1.

[0145] As another example, a coding block larger than a basic data unit can be divided into multiple sub-blocks. Here, the sub-blocks can be set as prediction units for basic units used for prediction, or as transformation units for basic units used for transformation and / or quantization. In this case, dividing the coding block into multiple prediction units can be defined as VPDU prediction unit partitioning, and dividing the coding block into multiple transformation units can be defined as VPDU transformation unit partitioning.

[0146] At least one of VPDU prediction unit partitioning and VPDU transform unit partitioning can be applied to a coding block. The partitioning type of a coding block partitioned according to the application of VPDU prediction units can be set to the same as the partitioning type of a coding block partitioned according to the application of VPDU transform units.

[0147] When VPDU prediction unit partitioning is applied only to coding blocks, prediction is performed for each sub-block, but transforms and / or quantization can be performed on the coding block. In this case, prediction modes such as prediction modes, intra-frame prediction modes, or inter-frame prediction modes can be determined for the coding block.

[0148] When only VPDU transform units are applied to a coding block, prediction is performed on the sub-blocks, but transform and / or quantization can be performed on each sub-block.

[0149] Figures 7 and 8 are diagrams illustrating examples of dividing a coded block into multiple sub-blocks.

[0150] Figure 7 is a diagram showing the partitioning pattern when only square basic data units are allowed, and Figure 8 is a diagram showing the partitioning pattern when both square and non-square basic data units are allowed.

[0151] Assuming only square basic data units are allowed, in Figures 7(a) and 7(b), CU0 and CU2 are defined as two different VPDUs, and CU1 is defined as four different VPDUs. Therefore, CU0 and CU2 can be divided into two sub-blocks, and CU1 can be divided into four sub-blocks.

[0152] Assuming that square and non-square basic data units are allowed, in Figures 8(a) and 8(b), CU0 and CU2 can be defined as one VPDU, and CU1 can be defined as using two different VPDUs. Therefore, CU0 and CU2 are not divided into sub-blocks, while CU1 can be divided into two sub-blocks.

[0153] In this case, CU1 can be divided into square sub-blocks or non-square sub-blocks. For example, based on the horizontal lines dividing CU1 vertically, CU1 can be divided into two square sub-blocks. Alternatively, CU1 can be divided into two non-square sub-blocks based on the vertical lines dividing CU1 horizontally.

[0154] When multiple partitioning type candidates exist for a coded block, information indicating any one of the multiple partitioning type candidates can be transmitted via signals in the bitstream. For example, the information can indicate whether the coded block is divided into square sub-blocks or whether the coded block is divided into non-square sub-blocks.

[0155] Alternatively, dividing a coded block into square sub-blocks can be set to have a higher priority than dividing a coded block into non-square sub-blocks. For example, dividing a coded block into non-square sub-blocks can be allowed when the coded block cannot be divided into square sub-blocks.

[0156] Alternatively, the partitioning type of the coding block can be determined based on the partitioning type of the parent node coding block. For example, when partitioning the parent node coding block based on a ternary tree, it can be set to partition the coding block into square sub-blocks. On the other hand, when partitioning the parent node coding block based on a binary tree or ternary tree, it can be set to partition the coding block into non-square sub-blocks.

[0157] Inter-frame prediction refers to using information from previous images to predict the prediction pattern of the current block. For example, a block in a previous image that is at the same position as the current block (hereinafter referred to as a collocated block) can be set as the prediction block for the current block. Hereinafter, the prediction block generated based on the block at the same position as the current block is called a collocated prediction block.

[0158] On the other hand, if an object that existed in a previous image has moved to a different location in the current image, the object's motion can be used to effectively predict the current block. For example, if the direction and size of the object's movement can be known by comparing the previous and current images, the object's motion information can be considered to generate a predicted block (or predicted image) for the current block. Hereinafter, the predicted block generated using motion information can be referred to as a motion prediction block.

[0159] A residual block can be generated by subtracting a prediction block from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using a motion prediction block instead of a corresponding prediction block, thereby improving the compression performance of the residual block.

[0160] As mentioned above, the process of generating prediction blocks using motion information can be called motion-compensated prediction. In most inter-frame predictions, prediction blocks can be generated based on motion-compensated prediction.

[0161] Motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weighted index. The motion vector represents the direction and magnitude of the object's movement. The reference image index specifies the reference image for the current block among a list of reference images. The prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 and L1 prediction). Motion information in either the L0 or L1 direction can be used based on the prediction direction of the current block. The bidirectional weighted index specifies the weights applied to the L0 prediction block and the weights applied to the L1 prediction block.

[0162] Figure 9 is a flowchart of an inter-frame prediction method according to an embodiment of the present invention.

[0163] Referring to Figure 9, the inter-frame prediction method includes: determining the inter-frame prediction mode of the current block (S901); obtaining motion information of the current block according to the determined inter-frame prediction mode (S902); and performing motion compensation prediction of the current block based on the obtained motion information (S903).

[0164] Inter-frame prediction modes represent various techniques used to determine the motion information of the current block, and can include inter-frame prediction modes using translational motion information and inter-frame prediction modes using affine motion information. For example, inter-frame prediction modes using translational motion information can include merging mode and advanced motion vector prediction mode, while inter-frame prediction modes using affine motion information can include affine merging mode and affine motion vector prediction mode. Based on the inter-frame prediction mode, the motion information of the current block can be determined based on neighboring blocks adjacent to the current block or information parsed from the bitstream.

[0165] Motion information for the current block can be derived from the motion information of other blocks. These other blocks can be those that are prioritized for inter-frame prediction encoding / decoding compared to the current block. Setting the motion information of the current block to be the same as that of other blocks is defined as a merging mode. Furthermore, setting the motion vectors of other blocks to the predicted values ​​of the motion vectors of the current block is defined as a motion vector prediction mode.

[0166] Figure 10 is a flowchart of the process of exporting motion information of the current block in merge mode.

[0167] Merging candidates for the current block can be exported (S1001). Merging candidates for the current block can be exported from blocks that were encoded / decoded using inter-frame prediction before the current block.

[0168] Figure 11 is a diagram showing an example of candidate blocks used to derive merge candidates.

[0169] Candidate blocks can include at least one of the following: neighboring blocks containing samples adjacent to the current block, or non-neighboring blocks containing samples not adjacent to the current block. Hereinafter, the samples used to determine candidate blocks will be designated as reference samples. Furthermore, reference samples adjacent to the current block will be referred to as neighboring reference samples, and reference samples not adjacent to the current block will be referred to as non-neighboring reference samples.

[0170] Adjacent reference samples can be included in the adjacent column of the leftmost column of the current block or the adjacent row of the topmost row of the current block. For example, if the coordinates of the top-left sample of the current block are (0, 0), then at least one of the following blocks—a block including a reference sample at position (-1, H-1), a block including a reference sample at position (W-1, -1), a block including a reference sample at position (W, -1), a block including a reference sample at position (-1, H), or a block including a reference sample at position (-1, -1)—can be used as candidate blocks. Referring to the accompanying drawings, adjacent blocks with indices 0 to 4 can be used as candidate blocks.

[0171] A non-adjacent reference sample refers to a sample whose x-axis distance or y-axis distance to the reference sample adjacent to the current block has a predefined value. For example, a block containing a reference sample whose x-axis distance to the left reference sample is a predefined value, a block containing a non-adjacent sample whose y-axis distance to the upper reference sample is a predefined value, or a block containing non-adjacent samples whose x-axis and y-axis distances to the upper-left reference sample are both predefined values ​​can be used as a candidate block. The predefined value can be an integer such as 4, 8, 12, 16, etc. Referring to the accompanying drawings, at least one of the blocks with indices from 5 to 26 can be used as a candidate block.

[0172] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block can be set as unsuitable for merging. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, candidate blocks that include the reference sample can be set as unsuitable for merging.

[0173] Merge candidates can also be derived from temporally adjacent blocks included in images different from the current block. For example, merge candidates can be derived from co-located blocks included in a co-located image. Any one of multiple reference images included in a list of reference images can be set as a co-located image. Index information for identifying co-located images among multiple reference images can be sent via signaling in the bitstream. Alternatively, a reference image with a predefined index among multiple reference images can be identified as a co-located image.

[0174] The motion information of the merged candidate can be set to be the same as that of the candidate block. For example, at least one of the motion vector, reference image index, prediction direction, or bidirectional weighted index of the candidate block can be set as the motion information of the merged candidate.

[0175] A list of merge candidates, including merge candidates, can be generated (S1002).

[0176] The indices of multiple merge candidates in the merge candidate list can be assigned in a predetermined order. For example, the indices can be assigned in the order of merge candidates derived from the left adjacent block, merge candidates derived from the top adjacent block, merge candidates derived from the top right adjacent block, merge candidates derived from the bottom left adjacent block, merge candidates derived from the top left adjacent block, and merge candidates derived from the time adjacent block.

[0177] When the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates can be selected (S1003). Specifically, information specifying any one of the multiple merge candidates can be sent via a signal in the bitstream. For example, information merge_idx representing the index of any one of the multiple merge candidates included in the merge candidate list can be sent via a signal in the bitstream.

[0178] When the number of merge candidates in the merge candidate list is less than a threshold, merge candidates included in the inter-frame motion information list can be added to the merge candidate list. The threshold can be a value calculated from the maximum number of merge candidates the merge candidate list can include, or the maximum number of merge candidates minus an offset. The offset can be an integer such as 1 or 2. The inter-frame motion information list can include merge candidates derived based on blocks encoded / decoded prior to the current block.

[0179] The inter-frame motion information list includes merging candidates derived from blocks encoded / decoded based on inter-frame prediction within the current image. For example, the motion information of the merging candidates included in the inter-frame motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on inter-frame prediction. The motion information may include at least one of motion vectors, reference image indexes, prediction directions, or bidirectional weighted indexes.

[0180] The coded block can be divided into multiple prediction units, and prediction can be performed on each of the divided prediction units. Here, a prediction unit represents the basic unit used for prediction.

[0181] A coded block can be divided using at least one of vertical lines, horizontal lines, diagonal lines, or diagonal lines. Information for determining at least one of the number, angle, or position of lines dividing the coded block can be transmitted via signals in the bitstream. For example, information indicating any of the candidate partition types for the coded block can be transmitted via signals in the bitstream, or information specifying any of a plurality of line candidates for dividing the coded block can be transmitted via signals in the bitstream. Alternatively, information for determining the number or type of line candidates for dividing the coded block can be transmitted via signals in the bitstream. For example, using a 1-bit flag, it can be determined whether diagonal lines with angles greater than the diagonal and / or diagonal lines with angles less than the diagonal are suitable as line candidates.

[0182] Alternatively, at least one of the following can be adaptively determined based on at least one of the intra-frame prediction mode, inter-frame prediction mode, available merge candidate location, or adjacent block partitioning type of the coding block: the number, angle, or location of the lines that partition the coding block.

[0183] If a coding block is divided into multiple prediction units, then intra-frame prediction or inter-frame prediction can be performed on each prediction unit.

[0184] Figure 12 is a diagram illustrating an example of dividing a coding block into multiple prediction units using diagonals.

[0185] As shown in the examples in Figures 12(a) and 12(b), the coded block can be divided into two triangular prediction units using diagonals.

[0186] Figures 12(a) and 12(b) show how a coded block can be divided into two prediction units using a diagonal line connecting the two vertices of the coded block. However, a coded block can also be divided into two prediction units using a diagonal line whose at least one end does not cross a vertex of the coded block.

[0187] Figure 13 is a diagram illustrating an example of dividing a coding block into two prediction units.

[0188] As shown in the examples in Figures 13(a) and 13(b), the coding block can be divided into two prediction units using a slanted line whose two ends touch the upper and lower boundaries of the coding block, respectively.

[0189] Alternatively, as shown in the examples of Figures 13(c) and 13(d), the coded block can be divided into two prediction units using a diagonal line whose two ends contact the left and right boundaries of the coded block, respectively.

[0190] Alternatively, the coded block can be divided into two prediction blocks of different sizes. For example, the dividing line of the coded block can be set to contact the two boundary surfaces that form a vertex, thereby dividing the coded block into two prediction units of different sizes.

[0191] Figure 14 shows an example of dividing a coded block into multiple prediction blocks of different sizes.

[0192] As shown in the examples in Figures 14(a) and 14(b), the coding block can be divided into two prediction units of different sizes by setting the diagonal connecting the top-left or bottom-right corner of the coding block to pass through the left, right, top, or bottom boundary of the coding block, instead of passing through the top-left or bottom-right corner of the coding block.

[0193] Alternatively, as in the examples shown in Figures 14(c) and 14(d), the coding block can be divided into two prediction units of different sizes by setting the diagonal connecting the top right or bottom left corner of the coding block to pass through the left, right, top, or bottom boundary of the coding block, rather than through the top left or bottom right corner of the coding block.

[0194] Each prediction unit generated by dividing the coding block is called the "Nth prediction unit". For example, in the examples shown in Figures 12 to 14, PU1 can be defined as the first prediction unit, and PU2 can be defined as the second prediction unit. The first prediction unit can refer to the prediction unit that includes samples located in the lower left or upper left of the coding block, and the second prediction unit can refer to the prediction unit that includes samples located in the upper right or lower right of the coding block.

[0195] Conversely, a prediction unit that includes samples located in the upper right or lower right of the coding block can be defined as a first prediction unit, and a prediction unit that includes samples located in the lower left or upper left of the coding block can be defined as a second prediction unit.

[0196] The embodiments described later primarily illustrate examples of partitioning using diagonals. Specifically, the process of dividing a coding block into two prediction units using diagonals is called diagonal partitioning or triangular partitioning, and the prediction units generated based on diagonal partitioning are called triangular prediction units. However, it is also possible, in the embodiments described later, to use examples of partitioning using oblique lines at angles different from vertical lines, horizontal lines, or diagonals.

[0197] Whether to apply diagonal partitioning to a coding block can be determined based on at least one of the following: stripe type, maximum number of merging candidates that may be included in the merge candidate list, size of the coding block, shape of the coding block, prediction mode of the coding block, or partitioning type of the parent node.

[0198] For example, whether to apply diagonal partitioning to a coded block can be determined based on whether the current stripe is a B-strip. Diagonal partitioning is only allowed when the current stripe is a B-strip.

[0199] Alternatively, the decision to apply diagonal partitioning to the coded block can be based on whether the maximum number of merge candidates included in the merge candidate list is two or more. Diagonal partitioning is only permitted when the maximum number of merge candidates included in the merge candidate list is two or more.

[0200] Alternatively, when at least one of the width or height of the hardware is greater than 64, there is a drawback that 64 × 64 data processing units are redundantly accessed. Therefore, when at least one of the width or height of the coded block is greater than a threshold, it may be impossible to divide the coded block into multiple prediction blocks. For example, when at least one of the height and width of the coded block is greater than 64 (e.g., when at least one of the width and height is 128), diagonal division may not be used.

[0201] Alternatively, considering the maximum number of samples that can be processed simultaneously in a hardware implementation, diagonal partitioning may not be allowed for coded blocks with a sample count greater than a threshold. For example, diagonal partitioning may not be allowed for coded tree blocks with a sample count greater than 4096.

[0202] Alternatively, diagonal splits may not be allowed for coding blocks containing fewer than a threshold number of samples. For example, when a coding block contains fewer than 64 samples, it can be configured not to apply diagonal splits to the coding block.

[0203] Alternatively, whether to apply diagonal division to the coding block can be determined based on whether the width-to-height ratio of the coding block is less than a first threshold or whether the width-to-height ratio of the coding block is greater than a second threshold. Here, the width-to-height ratio whRatio of the coding block can be determined as the ratio of the width CbW to the height CbH of the coding block, as shown in Equation 1 below.

[0204] Equation 1

[0205] The second threshold can be the reciprocal of the first threshold. For example, if the first threshold is k, the second threshold can be 1 / k.

[0206] Diagonal division can only be applied to a coding block if the width-to-height ratio of the coding block is between the first threshold and the second threshold.

[0207] Alternatively, triangular partitioning can only be used if the width-to-height ratio of the coded block is less than a first threshold or greater than a second threshold. For example, when the first threshold is 16, diagonal partitioning is not allowed for coded blocks of size 64 × 4 or 4 × 64.

[0208] Alternatively, the permission to split diagonally can be determined based on the partitioning type of the parent node. For example, when the coded block serving as the parent node is partitioned based on a quadtree, diagonal partitioning can be applied to the coded blocks serving as leaf nodes. On the other hand, when the coded block serving as the parent node is partitioned based on a binary or ternary tree, the coded blocks serving as leaf nodes are set to not allow diagonal partitioning.

[0209] Alternatively, the permission to split diagonally can be determined based on the prediction mode of the coded block. For example, when a coded block is coded using intra-frame prediction, diagonal splitting is permitted only if the coded block is coded using inter-frame prediction or if the coded block is coded using a predefined inter-frame prediction mode. The predefined inter-frame prediction mode can represent at least one of a merging mode, an advanced motion vector prediction mode, an affine merging mode, or an affine motion vector prediction mode.

[0210] Alternatively, the size of the parallel processing region can be used to determine whether diagonal partitioning is allowed. For example, if the size of the coded block is larger than the size of the parallel processing region, diagonal partitioning may not be used.

[0211] You can also consider two or more of the listed conditions to determine whether to apply diagonal partitioning to the coded block.

[0212] As another example, information indicating whether diagonal partitioning is applied to a coded block can be signaled via the bitstream. This information can be signaled at the sequence level, image level, stripe level, or block level. For example, a flag indicating whether a triangle partition has been applied to a coded block can be signaled at the coded block level.

[0213] When determining whether to apply diagonal division to a coding block, information indicating the number or position of the lines dividing the coding block can be sent via signals in the bitstream.

[0214] For example, when a coded block is divided by a diagonal, information indicating the direction of the diagonal dividing the coded block can be sent via signals in the bitstream. For example, a flag indicating the direction of the diagonal, `triangle_partition_type_flag`, can be sent via signals in the bitstream. This flag indicates whether the coded block is divided by a diagonal connecting the top left and bottom right, or by a diagonal connecting the top right and bottom left. Dividing the coded block by a diagonal connecting the top left and bottom right is called a left triangle partition type, and dividing the coded block by a diagonal connecting the top right and bottom left is called a right triangle partition type. For example, a value of 0 indicates a left triangle partition type, and a value of 1 indicates a right triangle partition type.

[0215] Additionally, the bitstream can be used to signal information indicating whether prediction units have the same size or information indicating the position of the diagonal used to divide the coding block. For example, if the information indicating the size of the prediction units indicates that the prediction units are the same size, the encoding of the information indicating the diagonal position is omitted, and the coding block can be divided into two prediction units using a diagonal line passing through the two vertices of the coding block. On the other hand, when the information indicating the size of the prediction units indicates that the prediction units are not the same size, the position of the diagonal used to divide the coding block can be determined based on the information indicating the position of the diagonal. For example, when a left triangle partitioning type is applied to a coding block, the position information can indicate whether the diagonal touches the left and lower boundaries or the upper and right boundaries of the coding block. Alternatively, when a right triangle partitioning type is applied to a coding block, the position information can indicate whether the diagonal touches the right and lower boundaries or the upper and left boundaries of the coding block.

[0216] Information indicating the partition type of a coded block can be sent at the coded block level using signals. Therefore, the partition type can be determined for different coded blocks that are partitioned diagonally.

[0217] As another example, for sequence, image, strip, tile, or coding tree units, information indicating the partition type can be sent using signals. In this case, the partition type of the coding block with diagonal partitioning can be set to be the same within the sequence, image, strip, tile, or coding tree unit.

[0218] Alternatively, for the first coding unit within the coding tree unit that applies diagonal partitioning, the information used to determine the partitioning type is encoded and transmitted by a signal, and the second and subsequent coding units that apply diagonal partitioning are set to use the same partitioning type as the first coding unit.

[0219] As another example, the partitioning type of a coded block can be determined based on the partitioning type of adjacent blocks. Adjacent blocks can include at least one of the following: adjacent blocks at the top-left corner, adjacent blocks at the top-right corner, adjacent blocks at the bottom-left corner, adjacent blocks above, or adjacent blocks to the left. For example, the partitioning type of the current block can be set to the same type as that of its adjacent blocks. Alternatively, the partitioning type of the current block can be determined based on whether the top-left adjacent block applies a left triangle partitioning type, or whether the top-right or bottom-left adjacent block applies a right triangle partitioning type.

[0220] The size of the prediction unit can be set to be no larger than the size of the transform unit. Additionally, it can be set so that a predictive unit must be included within a transform unit. Therefore, when applying VPDU transform unit partitioning to a coding block, diagonal partitioning can be applied to the transform unit instead of diagonal partitioning.

[0221] Figure 15 is a diagram illustrating an example of applying diagonal division to a transformation unit.

[0222] As shown in the example in Figure 15, when a coded block is divided into multiple transform units, it can be determined whether to apply a diagonal partition to each transform unit. Alternatively, it can be determined whether to apply a diagonal partition in units smaller than the size of a transform unit.

[0223] Alternatively, when applying VPDU transform unit partitioning to a coding block, it is determined whether diagonal partitioning should be applied to the coding block, and all transform units included in the coding block can be set to follow this determination. For example, when it is determined that diagonal partitioning should be applied to the coding block, all transform units included in the coding block can be partitioned into two prediction units.

[0224] To perform motion prediction compensation on the first and second triangle prediction units, motion information for each unit can be derived. In this case, the motion information of the first and second triangle prediction units can be derived from the merging candidates included in the merging candidate list. To distinguish between a general merging candidate list and the merging candidate list used to derive motion information of triangle prediction units, the merging candidate list used to derive motion information of triangle prediction units is referred to as the triangle merging candidate list, and the merging candidates included in the triangle merging candidate list are referred to as triangle merging candidates. However, applying the aforementioned merging candidate deriving method and merging candidate list construction method to triangle merging candidates and triangle merging candidate list construction methods is also included within the spirit of this invention.

[0225] Information for determining the maximum number of triangle merging candidates that the candidate list can include can be transmitted via a bitstream signal. This information can represent the difference between the maximum number of merge candidates that the candidate list can include and the maximum number of triangle merging candidates that the candidate list can include.

[0226] Triangle merging candidates can be derived from spatially adjacent blocks and temporally adjacent blocks of the coded block.

[0227] Figure 16 is a diagram showing adjacent blocks used to derive triangle merging candidates.

[0228] Triangle merging candidates can be derived using at least one of the following: an upper adjacent block, a left adjacent block, or a co-position block included in an image different from the coded block. An upper adjacent block can include at least one of the following: a block containing samples (xCb+CbW-1, yCb-1) above the coded block; a block containing samples (xCb+CbW, yCb-1) above the coded block; or a block containing samples (xCb-1, yCb-1) above the coded block. A left adjacent block can include at least one of the following: a block containing samples (xCb-1, yCb+CbH-1) to the left of the coded block; or a block containing samples (xCb-1, yCb+CbH) to the left of the coded block. A co-occurring block can be defined as either a block that includes samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coded block within the co-occurring image, or a block that includes samples (xCb / 2, yCb / 2) located at the center of the coded block.

[0229] Neighboring blocks can be searched in a predefined order, and triangle merge candidates can be constructed into a triangle merge candidate list in a predefined order. For example, triangle merge candidates can be searched in the order of B1, A1, B0, A0, C0, B2, and C1 to construct the triangle merge candidate list.

[0230] The motion information of the triangle prediction units can be derived based on the triangle merging candidate list. That is, triangle prediction units can share a single triangle merging candidate list.

[0231] To derive the motion information of the triangle merging unit, information specifying at least one of the triangle merging candidates included in the triangle merging candidate list can be transmitted via a signal in the bitstream. For example, index information merge_triangle_idx specifying at least one of the triangle merging candidates can be transmitted via a signal in the bitstream.

[0232] The index information can specify a combination of merge candidates for the first triangular prediction unit and merge candidates for the second triangular prediction unit. For example, Table 1 below shows an example of a combination of merge candidates based on the index information merge_triangle_idx.

[0233] Table 1

[0234] A value of 1 in the index information `merge_triangle_idx` indicates that the motion information of the first triangular prediction unit is derived from the merge candidate at index 1, and the motion information of the second triangular prediction unit is derived from the merge candidate at index 0. The merge candidate triangles used to derive the motion information of the first triangular prediction unit and the merge candidate triangles used to derive the motion information of the second triangular prediction unit can be determined through the index information `merge_triangle_idx`.

[0235] The partitioning type of the coding block using diagonal partitioning can also be determined based on the index information. That is, the index information can specify a combination of the merging candidates for the first triangular prediction unit, the merging candidates for the second triangular prediction unit, and the partitioning direction of the coding block. When determining the partitioning type of the coding block based on the index information, the information indicating the diagonal direction of the partitioned coding block, `triangle_partition_type_flag`, does not need to be encoded. Table 2 shows the partitioning types of the coding block based on the index information `merge_triangle_idx`.

[0236] Table 2

[0237] When the variable TriangleDir is 0, it indicates that the coding block uses the left triangle partitioning type; when TriangleDir is 1, it indicates that the coding block uses the right triangle partitioning type. By combining Tables 1 and 2, it can be set to specify the merging candidates for the first triangle prediction unit, the merging candidates for the second triangle prediction unit, and the partitioning direction of the coding block based on the index information merge_triangle_idx.

[0238] As another example, index information can be sent using signals only for either the first or second triangle prediction unit, and the index of the triangle merging candidate for the other of the first and second triangle prediction units can be determined based on this index information. For example, the triangle merging candidate for the first triangle prediction unit can be determined based on the index information merge_triangle_idx representing the index of either of the triangle merging candidates. Furthermore, the triangle merging candidate for the second triangle prediction unit can be specified based on the merge_triangle_idx. For example, the triangle merging candidate for the second triangle prediction unit can be derived by adding or subtracting an offset from the index information merge_triangle_idx. The offset can be an integer such as 1 or 2. For example, the triangle merging candidate for the second triangle prediction unit can be determined as the triangle merging candidate with an index of merge_traingle_idx plus 1. When merge_triangle_idx indicates the triangle merging candidate with the largest index value among the triangle merging candidates, the motion information of the second triangle prediction unit can be derived from the triangle merging candidate with index 0 or the triangle merging candidate with an index obtained by subtracting 1 from merge_triangle_idx.

[0239] Alternatively, motion information for the second triangle prediction unit can be derived from a triangle merging candidate having the same reference image as the triangle merging candidate of the first triangle prediction unit specified according to the index information. The triangle merging candidate having the same reference image as the triangle merging candidate of the first triangle prediction unit can represent at least one of an L0 reference image or an L1 reference image having the same triangle merging candidate as the first triangle prediction unit. When multiple triangle merging candidates exist that have the same reference image as the triangle merging candidate of the first triangle prediction unit, any one can be selected based on at least one of whether the merging candidate includes bidirectional motion information or the difference between the index of the merging candidate and the index information.

[0240] As another example, index information can be transmitted via signals for the first triangle prediction unit and the second triangle prediction unit, respectively. For instance, a first index information 1st_merge_idx for determining triangle merging candidates for the first triangle prediction unit and a second index information 2nd_merge_idx for determining triangle merging candidates for the second triangle prediction unit can be transmitted via signals through the bitstream. The motion information of the first triangle prediction unit can be derived from the triangle merging candidates determined based on the first index information 1st_merge_idx, and the motion information of the second triangle prediction unit can be derived from the triangle merging candidates determined based on the second index information 2nd_merge_idx.

[0241] The first index information 1st_merge_idx can represent any index among the triangle merging candidates included in the triangle merging candidate list. The triangle merging candidate of the first triangle prediction unit can be determined as the triangle merging candidate pointed to by the first index information 1st_merge_idx.

[0242] The triangle merging candidate pointed to by the first index information 1st_merge_idx is set as a triangle merging candidate that cannot be used as a second triangle prediction unit. Therefore, the second index information 2nd_merge_idx of the second triangle prediction unit can indicate the index of any of the remaining triangle merging candidates other than the triangle merging candidate pointed to by the first index information. When the value of the second index information 2nd_merge_idx is less than the value of the first index information 1st_merge_idx, the triangle merging candidate of the second triangle prediction unit can be determined as a triangle merging candidate with the index information represented by the second index information 2nd_merge_idx. On the other hand, when the value of the second index information 2nd_merge_idx is the same as or greater than the value of the first index information 1st_merge_idx, the triangle merging candidate of the second triangle prediction unit can be determined as a triangle merging candidate with the value of the second index information 2nd_merge_idx plus 1 as its index.

[0243] Alternatively, the decision to signal the second index information can be determined based on the number of triangle merging candidates included in the triangle merging candidate list. For example, if the maximum number of triangle merging candidates that the triangle merging candidate list can include is no more than 2, signaling the second index information can be omitted. When signaling the second index information is omitted, the second triangle merging candidate can be derived by adding or subtracting an offset from the first index information. For example, when the maximum number of triangle merging candidates that the triangle merging candidate list can include is 2 and the first index information is index 0, the second triangle merging candidate can be derived by adding 1 to the first index information. Alternatively, when the maximum number of triangle merging candidates that the triangle merging candidate list can include is 2 and the first index information is 1, the second triangle merging candidate can be derived by subtracting 1 from the first index information.

[0244] Alternatively, when omitting the signal transmission of the second index information, the second index information can be set to a default value. The default value can be 0. By comparing the first and second index information, second triangle merging candidates can be derived. For example, when the second index information is less than the first index information, the merging candidate for index 0 is set as the second triangle merging candidate; when the second index information is the same as or greater than the first index information, the merging candidate for index 1 is set as the second triangle merging candidate.

[0245] When a triangle merging candidate has unidirectional motion information, that unidirectional motion information is set as the motion information of the triangle prediction unit. Conversely, when a triangle merging candidate has bidirectional motion information, only either L0 motion information or L1 motion information is set as the motion information of the triangle prediction unit. The choice between L0 and L1 motion information can be determined based on the index of the triangle merging candidate or the motion information of another triangle prediction unit.

[0246] For example, when the index of a triangle merging candidate is even, the L0 motion information of the triangle prediction unit is set to 0, and the L1 motion information of the triangle merging candidate is set to the L1 motion information of the triangle prediction unit. Conversely, when the index of a triangle merging candidate is odd, the L1 motion information of the triangle prediction unit is set to 0, and the L0 motion information of the triangle merging candidate is also set to 0. Conversely, when the index of a triangle merging candidate is even, the L0 motion information of the triangle merging candidate is set to the L0 motion information of the triangle prediction unit, and when the index of a triangle merging candidate is odd, the L1 motion information of the triangle merging candidate can also be set to the L1 motion information of the triangle prediction unit. Alternatively, for the first triangle prediction unit, when the number of triangle merging candidates is even, the L0 motion information of the triangle merging candidate is set to the L0 motion information of the first triangle prediction unit; on the other hand, for the second triangle prediction unit, when the number of triangle merging candidates is odd, the L1 motion information of the triangle merging candidate is set to the L1 motion information of the second triangle prediction unit.

[0247] Alternatively, when the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit is set to 0, and the L1 motion information of the triangle merging candidate is set to the L1 information of the second triangle prediction unit. On the other hand, when the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit is set to 0, and the L0 motion information of the triangle merging candidate is set to the L0 motion signal of the second triangle prediction unit.

[0248] The triangle merging candidate list used to derive motion information of the first triangle prediction unit and the triangle merging candidate list used to derive motion information of the second triangle prediction unit can also be set to be different.

[0249] For example, when specifying triangle merging candidates for deriving motion information of the first triangle prediction unit within the triangle merging candidate list based on index information associated with the first triangle prediction unit, the motion information of the second triangle prediction unit can be derived using a triangle merging list that includes the remaining triangle merging candidates other than those indicated by the index information. Specifically, the motion information of the second triangle prediction unit can be derived from any of the remaining triangle merging candidates.

[0250] Therefore, the maximum number of triangle merging candidates included in the triangle merging candidate list of the first triangle prediction unit and the maximum number of triangle merging candidates included in the triangle merging candidate list of the second triangle prediction unit will be different. For example, when the triangle merging candidate list of the first triangle prediction unit includes M merging candidates, the triangle merging candidate list of the second triangle prediction unit may include M-1 merging candidates other than those indicated by the index information of the first triangle prediction unit.

[0251] As another example, merging candidates for each triangular prediction unit can be derived based on neighboring blocks adjacent to the coded block, and the availability of neighboring blocks can be determined by taking into account the shape or position of the triangular prediction unit.

[0252] Figure 17 is a diagram illustrating an example of determining the availability of neighboring blocks for each triangular prediction unit.

[0253] Neighboring blocks that are not adjacent to the first triangle prediction unit can be set as neighboring blocks that are unavailable to the first triangle prediction unit, and neighboring blocks that are not adjacent to the second triangle prediction unit can be set as neighboring blocks that are unavailable to the second triangle prediction unit.

[0254] For example, as shown in the example of Figure 17(a), when the left triangle partitioning type is applied to a coding block, it can be determined that blocks A1, A0, and A2, which are adjacent to the first triangle prediction unit in the adjacent blocks of the coding block, can be used for the first triangle prediction unit, while blocks B0 and B1 cannot be used for the first triangle prediction unit. Therefore, the triangle merging candidate list associated with the first triangle prediction unit includes triangle merging candidates derived from blocks A1, A0, and A2, but excludes triangle merging candidates derived from blocks B0 and B1.

[0255] As shown in the example in Figure 17(b), when the left triangle partitioning type is applied to the coding block, it can be determined that blocks B0 and B1 adjacent to the second triangle prediction unit can be used for the second triangle prediction unit, while blocks A1, A0, and A2 cannot be used for the second triangle prediction unit. Therefore, the triangle merging candidate list associated with the second triangle prediction unit includes triangle merging candidates derived from blocks B0 and B1, but excludes triangle merging candidates derived from blocks A1, A0, and A2.

[0256] Therefore, the number or range of triangle merging candidates that a triangle prediction unit can use can be determined based on at least one of the location of the triangle prediction unit or the partitioning type of the coding block.

[0257] As another example, the merging mode can be applied to only either the first triangle prediction unit or the second triangle prediction unit. Furthermore, another piece of motion information in the first and second triangle prediction units can be set to be the same as the motion information of the triangle prediction unit to which the merging mode is applied, or the motion information of the triangle prediction unit to which the merging mode is applied can be refined to derive it.

[0258] For example, the motion vector and reference image index of the first triangle prediction unit can be derived based on the triangle merging candidate. The motion vector of the first triangle prediction unit can be refined to derive the motion vector of the second triangle prediction unit. For example, the motion vector of the second triangle prediction unit can be derived by adding or subtracting the refined motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangle prediction unit. The reference image index of the second triangle prediction unit can be set to be the same as that of the first triangle prediction unit.

[0259] Information for determining the fine motion vector representing the difference between the motion vector of the first triangle prediction unit and the motion vector of the second triangle prediction unit can be transmitted via a code stream. This information may include at least one of information representing the magnitude of the fine motion vector or information representing the sign of the fine motion vector.

[0260] Alternatively, the symbol of the fine motion vector can be derived based on at least one of the position, index, or partition type applied to the coding block of the triangular prediction unit.

[0261] As another example, the motion vector and reference image index of either the first triangle prediction unit or the second triangle prediction unit can be transmitted via signaling. The other motion vector in the first triangle prediction unit or the second triangle prediction unit can be derived by refining the motion vector transmitted via signaling.

[0262] For example, based on information transmitted via signals from the bitstream, the motion vector and reference image index of the first triangle prediction unit can be determined. Furthermore, the motion vector of the second triangle prediction unit can be derived by refining the motion vector of the first triangle prediction unit. For instance, the motion vector of the second triangle prediction unit can be derived by adding or subtracting the refined motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangle prediction unit. The reference image index of the second triangle prediction unit can be set to be the same as that of the first triangle prediction unit.

[0263] Motion prediction compensation can be performed on the coded block based on the motion information of the first and second triangular prediction units. In this case, image quality degradation may occur at the boundaries of the first and second triangular prediction units. For example, the presence of edges on the boundaries of the first and second triangular prediction units can lead to a deterioration in image quality continuity. To reduce image quality degradation at the boundaries, prediction samples can be derived through smoothing filtering or weighted prediction.

[0264] Predictive samples applying diagonal division within a coding block can be derived by weighted summing of a first predictive sample obtained based on motion information of a first triangular predictive unit and a second predictive sample obtained based on motion information of a second triangular predictive unit. Alternatively, predictive samples of the first triangular predictive unit can be derived from a first prediction block determined based on motion information of the first triangular predictive unit, and predictive samples of the second triangular predictive unit can be derived from a second prediction block determined based on motion information of the second triangular predictive unit. Furthermore, predictive samples located in the boundary region between the first and second triangular predictive units can be derived by weighted summing of the first predictive samples included in the first prediction block and the second predictive samples included in the second prediction block. For example, Equation 2 below shows an example of deriving predictive samples of the first and second triangular predictive units.

[0265] Equation 2

[0266] In Equation 2, P1 represents the first predicted sample, and P2 represents the second predicted sample. w1 represents the weight applied to the first predicted sample, and (1-w1) represents the weight applied to the second predicted sample. As shown in the example in Equation 2, the weight applied to the second predicted sample can be derived by subtracting the weight applied to the first predicted sample from the constant.

[0267] When applying the left triangle partitioning type to a coding block, the boundary region may include predicted samples with the same x-axis and y-axis coordinates. On the other hand, when applying the right triangle partitioning type to a coding block, the boundary region may include predicted samples whose sum of x-axis and y-axis coordinates is above a first threshold and below a second threshold.

[0268] The size of the boundary region can be determined based on at least one of the following: the size of the coding block, the shape of the coding block, the motion information of the triangular prediction unit, the motion vector difference of the triangular prediction unit, the output order of the reference image, or the difference between the first and second prediction samples in the diagonal boundary.

[0269] Figures 18 and 19 are diagrams illustrating examples of deriving prediction samples based on a weighted sum operation of a first prediction sample and a second prediction sample. Figure 18 shows an example of applying a left triangle partitioning type to a coding block, and Figure 19 shows an example of applying a right triangle partitioning type to a coding block. Additionally, Figures 18(a) and 19(a) are diagrams illustrating prediction patterns for the luminance component, and Figures 18(b) and 19(b) are diagrams illustrating prediction patterns for the chrominance component.

[0270] In the diagram shown, the numbers written in the prediction samples near the boundaries of the first and second prediction units represent the weighting values ​​applied to the first prediction sample. For example, when the number written in the prediction sample is N, a weighting value of N / 8 is applied to the first prediction sample, and a weighting value of (1-(N / 8)) is applied to the second prediction sample, thereby deriving the prediction sample.

[0271] In non-boundary regions, either the first or second predicted sample can be identified as a predicted sample. Referring to the example in Figure 18, in regions where the absolute value of the difference between the x-axis and y-axis coordinates is greater than a threshold and belongs to the region of the first triangular prediction unit, the first predicted sample derived from the motion information of the first triangular prediction unit can be identified as a predicted sample. On the other hand, in regions where the difference between the x-axis and y-axis coordinates is greater than a threshold and belongs to the region of the second triangular prediction unit, the second predicted sample derived from the motion information of the second triangular prediction unit can be identified as a predicted sample.

[0272] Referring to the example in Figure 19, in regions where the sum of the x-axis and y-axis coordinates is less than a first threshold, a first prediction sample derived from the motion information of the first triangle prediction unit can be determined as a prediction sample. On the other hand, in regions where the sum of the x-axis and y-axis coordinates is greater than a second threshold, a second prediction sample derived from the motion information of the second triangle prediction unit can be determined as a prediction sample.

[0273] The threshold for identifying non-boundary regions can be determined based on at least one of the size of the coded block, the shape of the coded block, or the color components. For example, when the threshold associated with the luminance component is set to N, the threshold associated with the chrominance component can be set to N / 2.

[0274] The predicted samples included in the boundary region can be derived based on a weighted sum of the first and second predicted samples. In this case, the weighting value applied to the first and second predicted samples can be determined based on at least one of the position of the predicted sample, the size of the coding block, the shape of the coding block, or the color component of the coding block.

[0275] For example, as shown in the example in Figure 18(a), prediction samples with the same x-axis and y-axis coordinates can be derived by applying the same weighting values ​​to the first and second prediction samples. A prediction sample with an absolute value of 1 for the difference between the x-axis and y-axis coordinates can be derived by setting the weighting ratio applied to the first and second prediction samples to (3:1) or (1:3). Furthermore, a prediction sample with an absolute value of 2 for the difference between the x-axis and y-axis coordinates can be derived by setting the weighting ratio applied to the first and second prediction samples to (7:1) or (1:7).

[0276] Alternatively, as shown in the example of Figure 18(b), prediction samples at locations with the same x-axis and y-axis coordinates can be derived by applying the same weighting to the first and second prediction samples, and prediction samples with an absolute value of 1 for the difference between the x-axis and y-axis coordinates can be derived by setting the weighting ratio applied to the first and second prediction samples to (7:1) or (1:7).

[0277] For example, as shown in the example in Figure 19(a), a prediction sample whose sum of x-axis and y-axis coordinates is 1 less than the width or height of the coded block can be derived by applying the same weighting to the first and second prediction samples. A prediction sample whose sum of x-axis and y-axis coordinates is the same as or 2 less than the width or height of the coded block can be derived by setting the weighting ratio applied to the first and second prediction samples to (3:1) or (1:3). A prediction sample whose sum of x-axis and y-axis coordinates is 1 greater than or 3 less than the width or height of the coded block can be derived by setting the weighting ratio applied to the first and second prediction samples to (7:1) or (1:7).

[0278] Alternatively, as in the example shown in Figure 19(b), prediction samples whose sum of x-axis and y-axis coordinates is 1 less than the width or height of the coded block can be derived by applying the same weighting to both the first and second prediction samples. The weighting ratio applied to the first and second prediction samples can be set to (7:1) or (1:7) to derive prediction samples whose sum of x-axis and y-axis coordinates is the same as or 2 less than the width or height of the coded block.

[0279] As another example, the location of the predicted sample or the shape of the coding block can be considered to determine the weighting value. Equations 3 through 5 show examples of deriving the weighting value when a left-triangle partitioning type is applied to the coding block. Equation 3 shows an example of deriving the weighting value applied to the first predicted sample when the coding block is a square.

[0280] Equation 3

[0281] In Equation 3, x and y represent the positions of the predicted samples. When the coded block is not square, the weights applied to the first predicted sample can be derived as shown in Equation 4 or 5. Equation 4 shows the case where the width of the coded block is greater than its height, and Equation 5 shows the case where the width of the coded block is less than its height.

[0282] Equation 4

[0283] Equation 5

[0284] When the right triangle partitioning type is applied to a coding block, the weights applied to the first predicted sample can be determined as shown in Equations 6 through 8. Equation 6 shows an example of deriving the weights applied to the first predicted sample when the coding block is a square.

[0285] Equation 6

[0286] In Equation 6, CbW represents the width of the coded block. When the coded block is not square, the weighting applied to the first predicted sample can be derived as shown in Equations 7 or 8. Equation 7 shows the case where the width of the coded block is greater than its height, and Equation 8 shows the case where the width of the coded block is less than its height.

[0287] Equation 7

[0288] Equation 8

[0289] In Equation 7, CbH represents the height of the coded block.

[0290] As shown in the example, for the predicted samples within the boundary region, the samples included in the first triangular prediction unit can be derived by assigning a larger weighting value to the first predicted sample than to the second predicted sample, and the samples included in the second triangular prediction unit can be derived by assigning a larger weighting value to the second predicted sample than to the first predicted sample.

[0291] When applying diagonal partitioning to a coding block, the coding block can be set to a combined prediction mode that does not apply the combination of intra-prediction mode and merging mode.

[0292] Intra-frame prediction uses encoded / decoded reconstructed samples from the surrounding blocks to predict the current block. In this case, intra-frame prediction of the current block can use reconstructed samples before the loop filter is applied.

[0293] Intra-prediction techniques include matrix-based intra-prediction and general intra-prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra-prediction technique for the current block can be signaled via the bitstream. This information may be a 1-bit flag. Alternatively, the intra-prediction technique for the current block can be determined based on at least one of the intra-prediction techniques of the current block's position, size, shape, or neighboring blocks. For example, when the current block crosses an image boundary, the current block is set not to apply matrix-based intra-prediction.

[0294] Matrix-based intra-frame prediction is a method that obtains the predicted block for the current block by performing matrix multiplication between the matrices stored in the encoder and decoder and the reconstructed samples surrounding the current block. Information specifying any one of a plurality of stored matrices can be transmitted via a signal in the bitstream. The decoder can then determine the matrix used for intra-frame prediction of the current block based on this information and the size of the current block.

[0295] General intra-frame prediction is a method for obtaining the prediction block associated with the current block based on non-angular intra-frame prediction mode or angular intra-frame prediction mode. The process of performing intra-frame prediction based on general intra-frame prediction is described in more detail below with reference to the accompanying drawings.

[0296] Figure 20 is a flowchart illustrating an intra-frame prediction method according to an embodiment of the present invention.

[0297] The reference sample line for the current block can be determined (S2001). The reference sample line refers to the set of reference samples included in the Kth row above and / or to the left of the current block. The reference samples can be derived from the reconstructed samples that have been encoded / decoded around the current block.

[0298] Index information of reference sample lines identifying the current block among multiple reference sample lines can be transmitted via signals in the bitstream. For example, the index information intra_luma_ref_idx for specifying the reference sample line of the current block can be transmitted via signals in the bitstream. This index information can be transmitted via signals on a block-by-block basis.

[0299] Multiple reference sample lines may include at least one of the first, second, third, or fourth lines above and / or to the left of the current block. A reference sample line consisting of the row adjacent to the top of the current block and the column adjacent to the left of the current block can be called an adjacent reference sample line, while other reference sample lines can be called non-adjacent reference sample lines.

[0300] Only some of the multiple reference sample lines can be selected as reference sample lines for the current block. For example, the remaining reference sample lines, excluding the third non-adjacent reference sample line, can be set as candidate reference sample lines. Table 3 shows the indices assigned to each candidate reference sample line.

[0301] Table 3

[0302] More candidate reference sample lines than described can be set, or fewer can be set. Furthermore, the number or location of non-adjacent reference sample lines set as candidate reference sample lines is not limited to the described example. For example, the first and third non-adjacent reference sample lines can be set as candidate reference sample lines, or the second and third non-adjacent reference sample lines can be set as candidate reference sample lines. Alternatively, the first, second, and third non-adjacent reference sample lines can all be set as candidate reference sample lines.

[0303] The number or type of candidate reference sample lines can be determined based on at least one of the following: the size, shape, position, whether or not the current block is divided into sub-blocks, or the intra-frame prediction mode.

[0304] The reference sample line for the current block can also be determined based on at least one of the following: the current block's position, size, shape, or the prediction patterns of neighboring blocks. For example, when the current block is in contact with the boundary of an image, tile, strip, or coding tree unit, the first reference sample line can be determined as the reference sample line for the current block.

[0305] Alternatively, when the current block is not square, adjacent reference sample lines can be set as reference sample lines for the current block. Alternatively, when the width-to-height ratio of the current block is above or below a threshold, adjacent reference sample lines can be determined as reference sample lines for the current block.

[0306] The reference sample line can include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper and left reference samples can be derived from the reconstructed samples surrounding the current block. The reconstructed samples can be in the state before the application of the loop filter.

[0307] Next, the intra-prediction mode of the current block can be determined (S2002). For the intra-prediction mode of the current block, at least one of a non-angular intra-prediction mode or an angular intra-prediction mode can be determined as the intra-prediction mode of the current block. Non-angular intra-prediction modes include planar and DC (divergent focus), and angular intra-prediction modes include 33 or 65 modes from the lower left diagonal to the upper right diagonal.

[0308] Figure 21 is a diagram illustrating the intra-frame prediction mode.

[0309] Figure 21(a) shows 35 intra-prediction modes, and Figure 21(b) shows 67 intra-prediction modes.

[0310] More or fewer intra-frame prediction modes can be defined compared to those shown in Figure 21.

[0311] The Most Probable Mode (MPM) can be set based on the intra-prediction modes of neighboring blocks adjacent to the current block. Neighboring blocks can include the left neighboring block to the left of the current block and the upper neighboring block above the current block.

[0312] The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs can be sent via a signal in the bitstream. Alternatively, the number of MPMs can be determined based on at least one of the prediction modes of neighboring blocks, the size of the current block, or its shape. For example, N MPMs can be used when an adjacent reference sample line is determined as the reference sample line of the current block, while M MPMs can be used when a non-adjacent reference sample line is determined as the reference sample line of the current block. M is an integer less than N; for example, N can be 6, and M can be 5, 4, or 3. Thus, when the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra-prediction mode of the current block can be determined as any one of 6 candidate intra-prediction modes, while when the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra-prediction mode of the current block can be determined as any one of 5 candidate intra-prediction modes.

[0313] Alternatively, a fixed number (e.g., 6 or 5) of MPM candidates can be used, regardless of the index of the reference sample line of the current block.

[0314] An MPM list comprising multiple MPMs can be generated, and information indicating whether an MPM with the same intra-prediction mode as the current block is included in the MPM list can be transmitted via the bitstream signal. This information is a 1-bit flag, referred to as the MPM flag. When the MPM flag indicates that an MPM with the same intra-prediction mode as the current block is included in the MPM list, index information identifying one of the MPMs can be transmitted via the bitstream signal. For example, index information `mpm_idx` specifying any one of multiple MPMs can be transmitted via the bitstream signal. The MPM specified by the index information can be set as the intra-prediction mode of the current block. When the MPM flag indicates that an MPM with the same intra-prediction mode as the current block is not included in the MPM list, remaining mode information indicating any of the other intra-prediction modes besides the MPM can be transmitted via the bitstream signal. The remaining mode information represents the index value corresponding to the intra-prediction mode of the current block when the index is reallocated to the other intra-prediction modes besides the MPM. The decoder can sort the MPMs in ascending order and determine the intra-prediction mode of the current block by comparing the remaining mode information with the MPMs. For example, when the remaining mode information is the same as or smaller than the MPM, the intra-prediction mode of the current block can be derived by adding 1 to the remaining mode information.

[0315] When deriving the intra-prediction mode for the current block, some comparisons with other mode information in the MPM can be omitted. For example, the MPM for non-angular intra-prediction modes can be excluded from the comparison. When a non-angular intra-prediction mode is set to MPM, since the other mode information explicitly indicates angular intra-prediction modes, the intra-prediction mode for the current block can be derived by comparing the remaining MPMs (excluding non-angular intra-prediction modes) with the other mode information. Instead of excluding non-angular intra-prediction modes from the comparison, the result of adding the number of non-angular intra-prediction modes to the remaining mode information can be compared with the MPM.

[0316] Instead of setting the default mode to MPM, information indicating whether the intra-prediction mode of the current block is the default mode can be signaled via the bitstream. This information is a 1-bit flag, which may be called the default mode flag. The default mode flag is signaled only if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As mentioned above, the default mode can include at least one of planar, DC, vertical, or horizontal modes. For example, when planar is set as the default mode, the default mode flag can indicate whether the intra-prediction mode of the current block is planar. When the default mode flag indicates that the intra-prediction mode of the current block is not the default mode, one of the MPMs indicated by index information can be set as the intra-prediction mode of the current block.

[0317] When using the default mode flag, the intra prediction mode that is the same as the default mode may not be set to MPM. For example, when the default mode flag indicates whether the intra prediction mode of the current block is planar, the intra prediction mode of the current block can be derived by using 5 MPMs other than the MPM that is equivalent to planar.

[0318] When multiple intra-prediction modes are set as the default mode, index information indicating any of the default modes can be further sent using a signal. The intra-prediction mode of the current block can be set to the default mode indicated by the index information.

[0319] When the index of the reference sample line of the current block is not 0, the default mode is set not to be used. For example, when a non-adjacent reference sample line is determined to be the reference sample line of the current block, it can be set not to use non-angular intra-frame prediction modes such as DC mode or planar mode. Thus, when the index of the reference sample line is not 0, the default mode flag is not sent by signal, and the value of the default mode flag can be set to a predefined value (i.e., false).

[0320] When the intra-prediction mode of the current block is determined, the prediction sample of the current block can be obtained based on the determined intra-prediction mode (S2003).

[0321] When DC mode is selected, predicted samples related to the current block can be generated based on the average value of reference samples. Specifically, the value of the overall sample within the predicted block can be generated based on the average value of reference samples. The average value can be derived using at least one of the upper reference sample located above the current block and the left reference sample located to the left of the current block.

[0322] The number or range of reference samples used to derive the average value will vary depending on the shape of the current block. For example, when the current block is a non-square block with a width greater than its height, the average value can be calculated using only the top reference sample. On the other hand, when the current block is a non-square block with a width less than its height, the average value can be calculated using only the left reference sample. That is, when the width and height of the current block are different, the average value can be calculated using only the reference sample adjacent to the longer side. Alternatively, the choice between using only the top reference sample or only the left reference sample can be determined based on the width-to-height ratio of the current block.

[0323] When the planar mode is selected, prediction samples can be obtained using horizontal and vertical prediction samples. Specifically, the horizontal prediction sample is obtained based on the left and right reference samples located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on the upper and lower reference samples located on the same vertical line as the prediction sample. The right reference sample can be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample can be generated by copying the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample can be obtained based on a weighted sum of the left and right reference samples, and the vertical prediction sample can be obtained based on a weighted sum of the upper and lower reference samples. In this case, the weighting value assigned to each reference sample can be determined based on the position of the prediction sample. The prediction sample can also be obtained based on the average or weighted sum of the horizontal and vertical prediction samples. When performing a weighted sum operation, the weighting value assigned to the horizontal and vertical prediction samples can be determined based on the position of the prediction sample.

[0324] When an angle prediction mode is selected, parameters representing the prediction direction (or prediction angle) of the selected angle prediction mode can be determined. Table 4 below shows the intrapredAng parameter for each intrapredangling prediction mode.

[0325] Table 4

[0326] Table 4 shows the intra-direction parameters for each intra-prediction mode with an index of any one of 2 to 34 when 35 intra-prediction modes are defined. When more than 33 angular intra-prediction modes are defined, Table 4 is further subdivided into setting the intra-direction parameters for each angular intra-prediction mode.

[0327] After arranging the upper and left reference samples of the current block into a column, the predicted sample can be obtained based on the value of the intra-frame orientation parameter. In this case, when the value of the intra-frame orientation parameter is negative, the left and upper reference samples can be arranged into a column.

[0328] Figures 22 and 23 are diagrams illustrating an example of a one-dimensional array in which reference samples are arranged in a row.

[0329] Figure 22 shows an example of a vertical one-dimensional array of reference samples arranged in the vertical direction, and Figure 23 shows an example of a horizontal one-dimensional array of reference samples arranged in the horizontal direction. Embodiments of Figures 22 and 23 will be described under the assumption of defining 35 intra-frame prediction modes.

[0330] When the intra-prediction mode index is any one of 11 to 18, a one-dimensional horizontal arrangement of the upper reference sample can be applied, rotating counterclockwise. When the intra-prediction mode index is any one of 19 to 25, a one-dimensional vertical arrangement of the left reference sample can be applied, rotating clockwise. When arranging the reference samples in a column, the intra-prediction mode angle can be considered.

[0331] Reference sample determination parameters can be determined based on intra-frame orientation parameters. These parameters may include a reference sample index for specifying the reference sample and weighting parameters for determining the weights applied to the reference sample.

[0332] The reference sample index iIdx and the weighted parameter ifact can be obtained through Equations 9 and 10, respectively.

[0333] Equation 9

[0334] Equation 10

[0335] In equations 9 and 10, P ang This represents the intra-frame orientation parameter. The reference sample specified by the reference sample index iIdx is equivalent to an integer pixel (Integer pel).

[0336] To derive predicted samples, more than one reference sample can be specified. Specifically, considering the slope of the prediction pattern, the position of the reference sample used when deriving predicted samples can be specified. For example, using the reference sample index iIdx, the reference sample used when deriving predicted samples can be specified.

[0337] In this scenario, when the slope of the intra-prediction mode is not represented by a single reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, when the slope of the intra-prediction mode is the value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample can be obtained by interpolating the first and second reference samples. That is, when the angular line following the intra-prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample can be obtained by interpolating the reference samples that are adjacent to the left, right, or top and bottom of the position through which the angular line passes.

[0338] Equation 11 below shows an example of obtaining a predicted sample based on a reference sample.

[0339] Equation 11

[0340] In Equation 11, P represents the predicted sample, and Ref_1D represents any one of the reference samples in a one-dimensional arrangement. In this case, the position of the reference sample can be determined based on the position (x, y) of the predicted sample and the index iIdx of the reference sample.

[0341] When the slope of the intra-frame prediction mode can be represented by a reference sample, the weighting parameter i can be... fact Set it to 0. Therefore, Equation 11 can be simplified to Equation 12 as shown below.

[0342] Equation 12

[0343] Intra-prediction can also be performed on the current block based on multiple intra-prediction modes. For example, intra-prediction modes can be derived for different prediction samples, and prediction samples can be derived based on the intra-prediction modes assigned to each prediction sample.

[0344] Alternatively, intra-prediction modes can be derived for different regions, and intra-prediction can be performed on each region based on the intra-prediction modes assigned to each region. Each region may include at least one sample. The size or shape of the region can be adaptively determined based on at least one of the size, shape, or intra-prediction mode of the current block. Alternatively, in the encoder and decoder, it is possible to predefine at least one of the size or shape of the region, independent of the size or shape of the current block.

[0345] Alternatively, intra-prediction can be performed based on multiple intra-prediction methods, and the final prediction sample can be derived based on the average or weighted sum of multiple prediction samples obtained through multiple intra-prediction operations. For example, a first prediction sample can be obtained by performing intra-prediction based on a first intra-prediction mode, and a second prediction sample can be obtained by performing intra-prediction based on a second intra-prediction mode. Then, the final prediction sample can be obtained based on the average or weighted sum of the first and second prediction samples. In this case, the weighting values ​​assigned to the first and second prediction samples can be determined by considering at least one of whether the first intra-prediction mode is a non-angle / angle prediction mode, whether the second intra-prediction mode is a non-angle / angle prediction mode, or the intra-prediction modes of adjacent blocks.

[0346] Multiple intra-frame prediction modes can be a combination of non-angle intra-frame prediction modes and angle prediction modes, a combination of angle prediction modes, or a combination of non-angle prediction modes.

[0347] Figure 24 is a diagram showing the angle formed between the intra-angle prediction mode and a straight line parallel to the x-axis.

[0348] In the example shown in Figure 24, the angle prediction pattern can exist between the lower left diagonal direction and the upper right diagonal direction. When described as the angle formed by the x-axis and the angle prediction pattern, the angle prediction pattern can exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0349] When the current block is not square, the following will happen: depending on the intra-prediction mode of the current block, the prediction sample is derived by using the reference sample that is farther away from the prediction sample from the reference samples located on the corner that follows the intra-prediction angle, rather than the reference sample that is closer to the prediction sample.

[0350] Figure 25 is a diagram illustrating an example of obtaining a predicted sample when the current block is not a square.

[0351] For example, as shown in the example of Figure 25(a), suppose the current block is a non-square with a width greater than its height, and the intra-frame prediction mode of the current block is an angular intra-frame prediction mode with an angle between 0 and 45 degrees. In this case, when deriving the prediction sample A near the right column of the current block, a left reference sample L, which is far from the prediction sample in the angular mode located at the angle, is used instead of the upper reference sample T, which is close to the prediction sample.

[0352] As another example, as shown in the example of Figure 25(b), suppose the current block is a non-square with a height greater than its width, and the intra-frame prediction mode of the current block is an angular intra-frame prediction mode with an angle between -90 degrees and -135 degrees. In the above case, when deriving the prediction sample A near the lower row of the current block, a situation occurs where an upper reference sample T, which is far from the prediction sample in the angular mode located at the angle, is used instead of a left reference sample L that is close to the prediction sample.

[0353] To address the aforementioned issue, when the current block is not square, the intra-prediction mode of the current block can be replaced with an intra-prediction mode in the opposite direction. Therefore, for non-square blocks, an angle prediction mode with an angle larger or smaller than the angle prediction mode shown in Figure 21 can be used. This angle intra-prediction mode can be defined as a wide-angle intra-prediction mode. A wide-angle intra-prediction mode refers to an angle intra-prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0354] Figure 26 is a diagram illustrating the wide-angle intra-frame prediction mode.

[0355] In the example shown in Figure 26, the intra-prediction modes with indices -1 to -14 and the intra-prediction modes with indices 67 to 80 represent wide-angle intra-prediction modes.

[0356] Although Figure 26 shows 14 wide-angle intra-prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra-prediction modes (67 to 80) with angles less than -135 degrees, more or fewer wide-angle intra-prediction modes can be defined.

[0357] When using the wide-angle intra-frame prediction mode, the length of the upper reference sample is set to 2W+1, and the length of the left reference sample is set to 2H+1.

[0358] When using the wide-angle intra-frame prediction mode, the reference sample T can be used to predict sample A shown in Figure 25(a), and the reference sample L can be used to predict sample A shown in Figure 25(b).

[0359] By adding the existing intra-prediction modes to N wide-angle intra-prediction modes, a total of 67 + N intra-prediction modes can be used. For example, Table 5 shows the intra-direction parameters of the intra-prediction modes when 20 wide-angle intra-prediction modes are defined.

[0360] Table 5

[0361] When the current block is non-square and the intra-prediction mode of the current block obtained in step S2002 falls within the transformation range, the intra-prediction mode of the current block can be transformed into a wide-angle intra-prediction mode. The transformation range can be determined based on at least one of the size, shape, or ratio of the current block. The ratio can represent the ratio between the width and height of the current block.

[0362] When the current block is a non-square with a width greater than its height, the transformation range can be set from the intra-prediction mode index in the upper-right diagonal direction (e.g., 66) to (the index of the intra-prediction mode in the upper-right diagonal direction - N). Here, N can be determined based on the ratio of the current block. When the intra-prediction mode of the current block falls within the transformation range, the intra-prediction mode can be transformed into a wide-angle intra-prediction mode. The transformation can be performed by subtracting a predefined value from the intra-prediction mode; the predefined value can be the total number of intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 67).

[0363] According to the embodiment, the intra-frame prediction modes between the 66th and 53rd frames can be transformed into wide-angle intra-frame prediction modes between the -1st and -14th frames, respectively.

[0364] When the current block is a non-square with a height greater than its width, the transformation range can be set from the intra-prediction mode index (e.g., 2) along the lower left diagonal to (the index of the intra-prediction mode along the lower left diagonal + M). Here, M can be determined based on the ratio of the current block. When the intra-prediction mode of the current block falls within the transformation range, the intra-prediction mode can be transformed into a wide-angle intra-prediction mode. This transformation can be performed by adding a predefined value to the intra-prediction mode; the predefined value can be the total number of angular intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 65).

[0365] According to the embodiment, the intra-frame prediction modes between the 2nd and 15th frames are transformed into wide-angle intra-frame prediction modes between the 67th and 80th frames, respectively.

[0366] Hereinafter, the intra-frame prediction modes that fall within the transform range will be referred to as wide-angle intra-frame replacement prediction modes.

[0367] The transform range can be determined based on the ratio of the current block. For example, Tables 6 and 7 show the transform range when 35 intra-prediction modes and 67 intra-prediction modes, excluding the wide-angle intra-prediction mode, are defined, respectively.

[0368] Table 6

[0369] Table 7

[0370] As shown in the examples in Tables 6 and 7, the number of wide-angle intra-frame replacement prediction modes falling within the transform range can vary depending on the ratio of the current block.

[0371] The transformation range can be set as shown in Table 8 by subdividing the current block by a certain ratio.

[0372] Table 8

[0373] When a non-adjacent reference sample line is determined as the reference sample line for the current block, or when a multi-line intra-prediction coding method is used to select any one of multiple reference sample lines, it can be configured not to use the wide-angle intra-prediction mode. That is, even if the current block is not square and the intra-prediction mode of the current block falls within the transform range, it is possible not to transform the intra-prediction mode of the current block to the wide-angle intra-prediction mode.

[0374] Alternatively, when the intra-prediction mode for the current block is determined to be wide-angle intra-prediction mode, non-adjacent reference sample lines can be set to be unusable as reference sample lines for the current block, or the multi-line intra-prediction coding method for selecting any one of the multiple reference sample lines can be disabled. When the multi-line intra-prediction coding method is not used, adjacent reference sample lines can be set as reference sample lines for the current block.

[0375] When the wide-angle intra-frame prediction mode is not used, refW and refH are set to the sum of nTbW and nTbH. Therefore, besides the top-left reference sample, non-adjacent reference samples at a distance i from the current block can include (nTbW + nTbH + offsetX[i]) above reference samples and (nTbW + nTbH + offsetY[i]) left reference samples. That is, non-adjacent reference samples at a distance i from the current block can include (2nTbW + 2nTbH + offsetX[i] + offsetY[i] + 1) reference samples. For example, when the value of whRatio is greater than 1, the value of offsetX is set to be greater than the value of offsetY. For example, the value of offsetX is set to 1, and the value of offsetY is set to 0. On the other hand, when the value of whRatio is less than 1, the value of offsetY is set to be greater than the value of offsetX. For example, the value of offsetX is set to 0, and the value of offsetY is set to 1.

[0376] With the use of a wide-angle intra prediction mode in addition to the existing intra prediction modes, the resources required for encoding the wide-angle intra prediction mode increase, potentially reducing coding efficiency. Therefore, instead of directly encoding the wide-angle intra prediction mode, encoding an alternative intra prediction mode associated with it can improve coding efficiency.

[0377] For example, when encoding the current block using the 67th wide-angle intra-prediction mode, the 67th wide-angle replacement intra-prediction mode (number 2) can be encoded as the intra-prediction mode for the current block. Similarly, when encoding the current block using the -1st wide-angle intra-prediction mode, the -1st wide-angle replacement intra-prediction mode (number 66) can be encoded as the intra-prediction mode for the current block.

[0378] The decoder can decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode is included in the transform range. When the decoded intra-prediction mode is a wide-angle replacement intra-prediction mode, it can transform the intra-prediction mode into a wide-angle intra-prediction mode.

[0379] Alternatively, when encoding the current block in wide-angle intra-prediction mode, wide-angle intra-prediction mode can also be encoded directly.

[0380] Encoding of the intra-prediction mode can be implemented based on the MPM list. Specifically, when encoding adjacent blocks in the wide-angle intra-prediction mode, the MPM can be set based on the wide-angle replacement intra-prediction mode corresponding to the wide-angle intra-prediction mode.

[0381] If a prediction block is generated from the results of intra-frame prediction, the prediction samples can be updated based on the position of each prediction sample included in the prediction block. The update method described above can be called an intra-frame weighted prediction method based on sample position (or position-dependent prediction combination (PDPC)).

[0382] Whether to use PDPC can be determined by considering the intra-prediction mode of the current block, the reference sample line of the current block, the size of the current block, or the color component. For example, PDPC can be used if the intra-prediction mode of the current block is at least one of planar mode, DC mode, vertical mode, horizontal mode, a mode with an index value smaller than the vertical mode, or a mode with an index value larger than the horizontal mode. Alternatively, PDPC can be used only if at least one of the width or height of the current block is greater than 4. Alternatively, PDPC can be used only if the index of the reference image line of the current block is 0. Alternatively, PDPC can be used only if the index of the reference image line of the current block is greater than or equal to a predefined value. Alternatively, PDPC can be used only for the luma component. Alternatively, whether to use PDPC can be determined based on whether two or more of the listed conditions are met.

[0383] As another example, information indicating whether PDPC is applied can be sent via signaling through the bitstream.

[0384] If a prediction sample is obtained through intra-frame prediction, a reference sample for correcting the prediction sample can be determined based on the position of the obtained prediction sample. For ease of explanation, in the following embodiments, the reference sample used to correct the prediction sample will be referred to as the PDPC reference sample. Furthermore, the prediction sample obtained through intra-frame prediction will be referred to as the first prediction sample, and the prediction sample obtained by correcting the first prediction sample will be referred to as the second prediction sample.

[0385] Figure 27 is a diagram illustrating an embodiment of the application of PDPC.

[0386] The first prediction sample can be corrected using at least one PDPC reference sample. The PDPC reference sample may include at least one of the following: a reference sample adjacent to the top-left corner of the current block, an upper reference sample above the current block, or a left reference sample to the left of the current block.

[0387] At least one of the reference samples belonging to the reference sample line of the current block can be set as a PDPC reference sample. Alternatively, regardless of the reference sample line of the current block, at least one of the reference samples belonging to the reference sample line with index 0 can be set as a PDPC reference sample. For example, even if the first prediction sample is obtained using reference samples included in the reference sample line with index 1 or index 2, the second prediction sample can also be obtained using reference samples included in the reference sample line with index 0.

[0388] The number or location of PDPC reference samples used to correct the first prediction sample can be determined by considering at least one of the intra-prediction mode of the current block, the size of the current block, the shape of the current block, or the location of the first prediction sample.

[0389] For example, when the intra-prediction mode of the current block is planar mode or DC mode, a second prediction sample can be obtained using the upper reference sample and the left reference sample. In this case, the upper reference sample can be a reference sample perpendicular to the first prediction sample (e.g., a reference sample with the same x-coordinate), and the left reference sample can be a reference sample horizontal to the first prediction sample (e.g., a reference sample with the same y-coordinate).

[0390] When the intra-prediction mode of the current block is horizontal intra-prediction mode, a second prediction sample can be obtained using the upper reference sample. In this case, the upper reference sample can be a reference sample perpendicular to the first prediction sample.

[0391] When the intra-prediction mode of the current block is vertical intra-prediction mode, a second prediction sample can be obtained using the left reference sample. In this case, the left reference sample can be a reference sample at the same level as the first prediction sample.

[0392] When the intra-prediction mode of the current block is either the bottom-left diagonal intra-prediction mode or the top-right diagonal intra-prediction mode, a second prediction sample can be obtained based on the top-left reference sample, the top reference sample, and the left reference sample. The top-left reference sample can be a reference sample adjacent to the top-left corner of the current block (e.g., the reference sample at position (-1, -1)). The top reference sample can be a reference sample located in the top-right diagonal direction of the first prediction sample, and the left reference sample can be a reference sample located in the bottom-left diagonal direction of the first prediction sample.

[0393] In summary, when the position of the first predicted sample is (x, y), R(-1, -1) can be set as the top-left reference sample, and R(x+y+1, -1) or R(x, -1) can be set as the top reference sample. Alternatively, R(-1, x+y+1) or R(-1, y) can be set as the left-side reference sample.

[0394] A prediction mode can be applied to the current block multiple times, or multiple prediction modes can be applied to the current block repeatedly. Thus, prediction methods that use the same or different prediction modes can be called combined prediction modes (or multi-hypothesis prediction modes).

[0395] The combined prediction mode may include at least one of the following: a mode that combines a merging mode and a merging mode, a mode that combines inter-frame prediction and intra-frame prediction, a mode that combines a merging mode and an advanced motion vector prediction mode, and a mode that combines a merging mode and intra-frame prediction.

[0396] In the combined prediction mode, a first prediction block can be generated based on a first prediction mode, and a second prediction block can be generated based on a second prediction mode. Then, a third prediction block can be generated based on a weighted sum of the first and second prediction blocks. The third prediction block can be set as the final prediction block for the current block.

[0397] Whether to use a combined prediction mode can be determined based on the size or shape of the current block. For example, it can be determined based on at least one of the following: the size of the coding block, the number of samples included in the coding block, the width of the coding block, and the height of the coding block. For example, when at least one of the width or height of the coding block is greater than or equal to 128, or when the number of samples included in the coding block is less than or equal to 64, the combined prediction mode may not be applied to the coding block.

[0398] Additionally, information indicating whether a combined prediction mode has been applied to the current block can be transmitted via signaling in the bitstream. For example, this information could be a 1-bit flag. For instance, a flag `mh_intra_flag` indicating whether a combined prediction mode (combining merging and intra-frame prediction) is used can be transmitted via signaling in the bitstream. `mh_intra_flag` of 1 indicates that a combined prediction mode (combining merging and intra-frame prediction) is used, and `mh_intra_flag` of 0 indicates that a combined prediction mode (combining merging and intra-frame prediction) is not used.

[0399] A combined prediction mode, which combines merging and intra-frame prediction, can only be applied to the current block if the current block's prediction mode is inter-frame prediction mode and merging mode is applied to the current block. That is, the mh_intra_flag signal can be sent when the merge_flag flag, which indicates whether to apply merging mode to the current block, is set to 1.

[0400] In a combined prediction mode that combines merging mode and intra-prediction, the intra-prediction mode of the current block can be set to a predefined intra-prediction mode. For example, when using a combined prediction mode, the intra-prediction mode of the current block can be set to planar mode.

[0401] As another example, when using a combined prediction mode that combines merging mode and intra-prediction, one of the candidate intra-prediction modes can be determined as the intra-prediction mode for the current block. Here, the candidate intra-prediction mode can include at least one of a non-angular intra-prediction mode and an angular intra-prediction mode in a specific direction. Here, the non-angular intra-prediction mode includes at least one of a DC mode and a planar mode, and the angular intra-prediction mode includes at least one of a horizontal intra-prediction mode, a vertical intra-prediction mode, and a diagonal intra-prediction mode. For example, when using a combined prediction mode, only the DC mode, the planar mode, the horizontal intra-prediction mode, or the vertical intra-prediction mode can be set as the intra-prediction mode for the current block. Alternatively, when using a combined prediction mode, only the planar mode, the horizontal intra-prediction mode, or the vertical intra-prediction mode can be set as the intra-prediction mode for the current block. Alternatively, when using a combined prediction mode, only the DC mode, the planar mode, or the vertical intra-prediction mode can be set as the intra-prediction mode for the current block. Alternatively, when using combined prediction modes, only the DC mode, planar mode, or horizontal intra-prediction mode can be set to the intra-prediction mode of the current block.

[0402] Index information specifying any of the candidate intra-prediction modes can be transmitted via signals in the bitstream. For example, the index mh_intra_idx specifying any of the candidate intra-prediction modes can be transmitted via signals in the bitstream. Tables 9 and 10 show the intra-prediction modes based on the value of mh_intra_idx. The intra-prediction mode indicated by mh_intra_idx can be determined as the intra-prediction mode for the current block.

[0403] Table 9

[0404] Table 10

[0405] In both the encoder and decoder, the number of candidate intra-prediction modes can be a fixed value. Alternatively, the number or type of candidate intra-prediction modes can vary depending on at least one of the size of the current block, the shape of the current block, or whether adjacent blocks are encoded via intra-prediction.

[0406] As another example, when applying a combined prediction mode that combines merging mode and intra-prediction, the intra-prediction mode for the current block can be determined based on the MPM. In this case, the number of MPMs included in the MPM list when applying general intra-prediction can differ from the number of MPMs included when applying the combined prediction mode. For example, the MPM list may include 6 or 5 MPMs when not applying the combined prediction mode, while it may include 4 or 3 MPMs when applying the combined prediction mode. That is, N MPMs are used when performing general intra-prediction, while fewer than N MPMs can be used when applying the combined prediction mode.

[0407] For example, suppose that when applying a general intra-prediction mode, the MPMs derived from the intra-prediction modes of neighboring blocks adjacent to the current block are Planar Mode, DC Mode, INTRA_MODE32, INTRA_MODE31, INTRA_MODE33, and INTRA_MODE30. When applying a combined prediction mode to the current block, the intra-prediction mode of the current block can be determined using only some of the six MPMs. For example, the intra-prediction mode of the current block can be determined using a list of MPMs including the three MPMs with the lowest indices (i.e., Planar Mode, DC Mode, and INTRA_MODE32). Alternatively, any MPM with a predetermined index value can be determined as the intra-prediction mode of the current block.

[0408] As another example, the number of MPMs used to determine the intra-prediction mode for the current block when applying general intra-prediction can be set to the same number of MPMs used to determine the intra-prediction mode for the current block when applying combined prediction mode.

[0409] When applying the combined prediction mode, the encoding of the MPM flag can be omitted, and the MPM flag can be treated as true. That is, when applying the combined prediction mode, any one of the MPMs can be determined as the intra-prediction mode for the current block.

[0410] When applying a combined prediction mode that combines merging mode and intra-prediction to the current block, it can be configured not to use multi-line intra-prediction coding. When not using multi-line intra-prediction coding, adjacent reference sample lines can be used for intra-prediction of the current block.

[0411] Alternatively, when a combined prediction mode that combines merging mode and intra-prediction is applied to the current block, it can be configured to use a multi-line intra-prediction coding method. Information for determining the reference sample lines of the current block among multiple reference sample lines can be transmitted via signaling in the bitstream. Alternatively, the reference sample lines of the current block can be determined by considering at least one of the following: the size of the current block, the shape of the current block, the intra-prediction mode of the current block, or the reference sample lines of adjacent blocks.

[0412] The number of available candidate intra-prediction modes can be set differently depending on the reference sample lines of the current block. For example, when using adjacent reference sample lines, the candidate intra-prediction modes can include at least one of the following: planar mode, DC mode, horizontal intra-prediction mode, vertical intra-prediction mode, lower left diagonal intra-prediction mode, and upper right diagonal intra-prediction mode. On the other hand, when using non-adjacent reference sample lines, the candidate intra-prediction modes can include at least one of the following: horizontal intra-prediction mode, vertical intra-prediction mode, lower left diagonal intra-prediction mode, and upper right diagonal intra-prediction mode.

[0413] The first prediction block can be generated using the merge candidate selected in the merge mode, and the second prediction block can be generated using the intra-prediction mode. In this case, when generating the first prediction block, it can be configured not to apply triangle partitioning to the current block. That is, in the combined prediction mode that combines the merge mode and intra-prediction, triangle partitioning may not be allowed. Additionally, when generating the second prediction block, it can be configured not to apply PDPC to the current block. That is, in the combined prediction mode that combines the merge mode and intra-prediction, PDPC may not be allowed.

[0414] A third prediction block can be generated based on a weighted sum of the first and second prediction blocks. For example, Equation 13 shows a diagram illustrating an example of generating a third prediction block based on a weighted sum of the first and second prediction blocks.

[0415] Equation 13

[0416] In equation 13, P merge This represents the first predicted block obtained based on the merging pattern, and P intra This represents the second prediction block obtained based on intra-frame prediction. P combThis represents the third prediction block obtained by combining the first and second prediction blocks. `w` represents the first weighting value applied to the first prediction block. The second weighting value applied to the second prediction block can be derived by subtracting the first weighting value `w` from the constant `N`. Here, `N` can have a value predefined by the encoder and decoder. For example, `N` can be 4 or 8. Alternatively, the constant `N` can be derived based on at least one of the current block size, shape, and intra-prediction mode.

[0417] Unlike the example shown in Equation 13, w can be set as the second weighting value, and the value obtained by subtracting the second weighting value from a predetermined constant N can be set as the first weighting value.

[0418] The weights applied to the first and second prediction blocks can be determined based on at least one of the intra-prediction mode and the merging mode of the current block, specifically whether they possess bidirectional motion information. For example, a first weighted value w when the intra-prediction mode of the current block is an angle prediction mode can be greater than a first weighted value w when the intra-prediction mode of the current block is a non-angle prediction mode. Alternatively, a first weighted value w when performing bidirectional prediction through a merging candidate can be greater than a first weighted value w when performing unidirectional prediction through a merging candidate.

[0419] As another example, the prediction patterns of neighboring blocks adjacent to the current block can be considered to determine the weighting value. Here, neighboring blocks can include at least one of the upper neighboring block above the current block, the left neighboring block to the left of the current block, and the upper-left neighboring block to the upper-left corner of the current block. The weighting value can be determined based on the number of neighboring blocks encoded via intra-frame prediction in the neighboring blocks adjacent to the current block. For example, as the number of neighboring blocks encoded via intra-frame prediction increases, the first weighting value can have a smaller value, and the second weighting value can have a larger value. On the other hand, as the number of neighboring blocks encoded via non-intra-frame prediction decreases, the first weighting value can have a larger value, and the second weighting value can have a smaller value. For example, when both the left and upper neighboring blocks are encoded via intra-frame prediction, the second weighting value can be set to n. When only one of the left and upper neighboring blocks is encoded via intra-frame prediction, the second weighting value can be set to n / 2. When neither the left-neighboring block nor the top-neighboring block is encoded via intra-frame prediction, the second weighting value can be set to n / 4. Here, n represents an integer equal to or less than N.

[0420] Weighting values ​​can be determined on a sample or sub-block basis within the current block. For example, the current block can be divided into multiple sub-blocks, and the weighting value applied to each sub-block can be set to a different value. In this case, the number of sub-blocks can have a fixed value in both the encoder and decoder. Alternatively, the number of sub-blocks can be adaptively determined based on at least one of the following: the size of the current block, the shape of the current block, the intra-prediction mode of the current block, and the prediction modes of neighboring blocks.

[0421] Figure 28 is a diagram illustrating an example of applying weighted values.

[0422] The current block can be divided into multiple sub-blocks, and the weighting value applied to each sub-block can be set to a different value. For example, as shown in the examples in Figures 28(a) and 28(b), for a sub-block that includes a sample located in the upper left corner of the current block, the second weighting value is set to be greater than the first weighting value. On the other hand, for a sub-block that includes a sample located in the lower right corner of the current block, the first weighting value can be set to be greater than the second weighting value.

[0423] The third predicted sub-block of a sub-block can be obtained by weighted prediction of the first and second predicted sub-blocks corresponding to that sub-block. In this case, as in the example shown in Figure 28, the weighting values ​​used for weighted prediction can be set differently for each sub-block.

[0424] Depending on whether the merge candidate has bidirectional motion information, the weighting values ​​assigned to the sub-blocks can be determined differently.

[0425] Figure 29 is a diagram illustrating an example of assigning different weights to sub-blocks based on whether the merging candidate has bidirectional motion information.

[0426] When a merge candidate has bidirectional motion information, the first weighting value w assigned to at least one sub-block can be set to a larger value compared to when the merge candidate has unidirectional motion information. For example, Figures 29(a) and 29(b) show that the weighting value w applied when bidirectional prediction is applied has a greater value than the weighting value w applied when unidirectional prediction is applied for all sub-blocks.

[0427] Alternatively, the weighting applied to the sub-block can be determined based on the intra-prediction mode of the current block.

[0428] Figure 30 is a diagram illustrating an example of assigning different weights to sub-blocks based on the intra-prediction mode of the current block.

[0429] When the intra-prediction mode of the current block is vertical, the second weighting value of the sub-block above the current block can be set to be greater than the second weighting value of the sub-block below the current block.

[0430] For example, for a child block above the current block, the second weighting value can be set to be greater than the first weighting value, while for a child block below the current block, the first weighting value and the second weighting value can be set to be equal, or the first weighting value can be set to be greater than the second weighting value.

[0431] When the intra-prediction mode of the current block is horizontal, the second weighting value of the sub-block to the left of the current block can be set to be greater than the second weighting value of the sub-block to the right of the current block.

[0432] For example, for a child block to the left of the current block, the second weighting value can be set to be greater than the first weighting value, while for a child block to the right of the current block, the first weighting value and the second weighting value can be set to be equal, or the first weighting value can be set to be greater than the second weighting value.

[0433] As another example, when the intra-prediction mode of the current block is vertical, the first weighting value of the sub-block above the current block is set to be greater than the first weighting value of the sub-block below the current block; or when the intra-prediction mode of the block is horizontal, the first weighting value of the sub-block to the left of the current block is set to be greater than the first weighting value of the sub-block to the right of the current block.

[0434] As another example, the weighting value assigned to each sub-block can be determined based on the prediction modes of neighboring blocks adjacent to the current block. For instance, when the upper neighboring block of the current block is encoded via intra-prediction while the left neighboring block is not, the second weighting value of the sub-block above the current block can be set to be greater than the second weighting value of the sub-block below the current block. Conversely, when the upper neighboring block of the current block is not encoded via intra-prediction while the left neighboring block is encoded via intra-prediction, the second weighting value of the sub-block to the left of the current block can be set to be greater than the second weighting value of the sub-block to the right of the current block.

[0435] For at least one of the multiple sub-blocks, either the first weighting value or the second weighting value can be set to 0. That is, for at least one of the multiple sub-blocks, either the first predicted sub-block or the second predicted sub-block can be set as the third predicted sub-block. Therefore, combined prediction can be omitted from at least one of the multiple sub-blocks. For example, for at least one of the multiple sub-blocks, the first weighting value w can be set to 0, or the first weighting value w can be set to a constant N.

[0436] A block to which a combined prediction mode, which combines merging mode and intra-frame prediction, is applied can be considered as coded via inter-frame prediction. Therefore, the intra-frame prediction mode of a block encoded with the combined prediction mode can be set to an intra-frame prediction mode that is not available for exporting blocks that will be encoded / decoded later. When exporting the MPM, the intra-frame prediction mode of an unavailable block can be treated as a planar mode.

[0437] On the other hand, the motion information of blocks encoded in a combined prediction mode can be set as motion information that can be used to derive blocks that will be encoded / decoded later.

[0438] In a mode that combines merging patterns, multiple merging candidates can be used to perform motion compensation prediction. Specifically, a first merging candidate can be used to generate a first prediction block, and a second merging candidate can be used to generate a second prediction block. A third prediction block can be generated based on a weighted sum of the first and second prediction blocks.

[0439] Information specifying the first and second merge candidates can be transmitted separately via signals in the bitstream. For example, index information `merge_idx` for specifying the first merge candidate and index information `merge_2nd_idx` for specifying the second merge candidate can be transmitted via signals in the bitstream. The second merge candidate can be determined based on the index information `merge_2nd_idx` and `merge_idx`.

[0440] The index information merge_idx specifies any one of the merge candidates included in the merge candidate list.

[0441] The index information `merge_2nd_idx` can specify any of the remaining merge candidates besides the one specified by `merge_idx`. Therefore, when the value of `merge_2nd_idx` is less than the value of `merge_idx`, the merge candidate with the index `merge_2nd_idx` can be set as the second merge candidate. When the value of `merge_2nd_idx` is equal to or greater than the value of `merge_idx`, the merge candidate with the index `merge_2nd_idx` plus 1 can be set as the second merge candidate.

[0442] Alternatively, the search order of candidate blocks can be considered to specify a second merge candidate.

[0443] Figure 31 is a diagram illustrating an example of specifying a second merge candidate by considering the search order of candidate blocks.

[0444] In the example shown in Figure 31, the indices marked on adjacent and non-adjacent samples indicate the search order of candidate blocks. For example, candidate blocks can be searched sequentially from position A0 to position A14.

[0445] When block A4 is selected as the first merge candidate, a merge candidate derived from a candidate block searched after A4 can be designated as the second merge candidate. For example, a merge candidate derived from A5 can be selected as the second merge candidate. If the candidate block at position A5 cannot be used as a merge candidate, a merge candidate derived from the next candidate block can be selected as the second merge candidate.

[0446] You can also select a first merge candidate and a second merge candidate from the merge candidates derived from non-adjacent blocks.

[0447] Figure 32 is a diagram illustrating an example of selecting a first and second merge candidate from merge candidates derived from non-adjacent blocks.

[0448] As shown in the example in Figure 32, merge candidates derived from a first candidate block and a second candidate block that are not adjacent to the current block can be selected as the first merge candidate and the second merge candidate, respectively. In this case, the block rows to which the first candidate block belongs and the second candidate block belong can be different. For example, the first merge candidate can be derived from any of the candidate blocks A5 to A10, and the second merge candidate can be derived from any of the candidate blocks A11 to A15.

[0449] Alternatively, it can be configured such that the first candidate block and the second candidate block are not included in the same line (e.g., row or column).

[0450] As another example, a second merge candidate can be specified based on a first merge candidate. In this case, the first merge candidate can be specified by the index information `merge_idx` sent from the bitstream via signaling. For example, a merge candidate adjacent to the first merge candidate can be designated as the second merge candidate. Here, a merge candidate adjacent to the first merge candidate can mean a merge candidate whose index difference from the first merge candidate is 1. For example, a merge candidate with an index value of `merge_idx+1` can be set as the second merge candidate. In this case, when the value of `merge_idx+1` is greater than the maximum index value (or when the index value of the first merge candidate is the maximum index), a merge candidate with an index value of `merge_idx-1` or a merge candidate with an index value of a predefined value (e.g., 0) can be set as the second merge candidate.

[0451] Alternatively, a merge candidate adjacent to the first merge candidate may mean a merge candidate derived from a candidate block that is spatially adjacent to the candidate block used to derive the first merge candidate. Here, an adjacent candidate block may mean a block to the left, right, above, below, or diagonally adjacent to the candidate block.

[0452] As another example, a second merging candidate can be specified based on the motion information of a first merging candidate. For instance, a merging candidate with the same reference image as the first merging candidate can be selected as the second merging candidate. When multiple merging candidates share the same reference image as the first merging candidate, the merging candidate with the smallest index, the merging candidate with the smallest index difference from the first merging candidate, or the merging candidate with the smallest motion vector difference from the first merging candidate can be selected as the second merging candidate. Alternatively, the second merging candidate can be selected based on the index information of any one of the multiple merging candidates.

[0453] Alternatively, when the first merging candidate is a unidirectional prediction in the first direction, a merging candidate including motion information in the second direction can be set as the second merging candidate. For example, when the first merging candidate has motion information in direction L0, a merging candidate with motion information in direction L1 can be set as the second merging candidate. When multiple merging candidates with motion information in direction L1 exist, the merging candidate with the smallest index or the merging candidate with the smallest index difference from the first merging candidate is set as the second merging candidate. Alternatively, the second merging candidate can be selected based on the index information of any one of the multiple merging candidates.

[0454] As another example, one of the merge candidates derived from the adjacent blocks that are adjacent to the current block can be set as the first merge candidate, and one of the merge candidates derived from the non-adjacent blocks that are not adjacent to the current block can be set as the second merge candidate.

[0455] As another example, one of the merge candidates exported from the candidate block above the current block can be set as the first merge candidate, and one of the merge candidates exported from the candidate block to the left can be set as the second merge candidate.

[0456] In the combined prediction mode that combines the above-mentioned merging modes, the merging mode can refer to a merging mode based on a translational motion model (hereinafter referred to as the translational merging mode) or a merging mode based on an affine motion model (hereinafter referred to as the affine merging mode). That is, motion compensation prediction can be performed by combining the translational merging mode and the translational merging mode, or by combining the affine merging mode and the affine merging mode.

[0457] As another example, when neighboring blocks used to derive a first merge candidate are encoded based on affine motion information, a merge candidate derived from the neighboring blocks encoded based on affine motion information can be set as a second merge candidate. For example, when the first merge candidate is an affine merge candidate, or when the first merge candidate is derived from the motion information of a sub-block in a coded block encoded based on affine motion information, the affine merge candidate or a merge candidate derived from the motion information of a sub-block in a coded block encoded based on affine motion information can be selected as the second merge candidate. Here, the second merge candidate may include at least one of the following: the merge candidate closest to the first merge candidate in search order, the merge candidate with the smallest index difference from the first merge candidate, the merge candidate with the smallest index, and the merge candidate with the smallest motion vector difference from the first merge candidate.

[0458] Conversely, when neighboring blocks used to derive a first merge candidate are encoded based on translational motion information (i.e., non-affine motion information), merge candidates derived from neighboring blocks encoded based on translational motion information can be set as second merge candidates. For example, when the first merge candidate is a non-affine merge candidate, the non-affine merge candidate can be selected as the second merge candidate. Here, the second merge candidate may include at least one of the following: the non-affine merge candidate that is closest to the first merge candidate in search order; the merge candidate with the smallest index difference from the first merge candidate; the merge candidate with the smallest index; and the merge candidate with the smallest motion vector difference from the first merge candidate.

[0459] Alternatively, the second merge candidate can be derived by adding or subtracting an offset from the index of the first merge candidate, and the second merge candidate can be reset to another merge candidate when the encoding methods used to derive the first and second merge candidates are different.

[0460] Figure 33 is a diagram illustrating an example of determining a second merger candidate.

[0461] When a first merge candidate is selected, a merge candidate with an index derived by adding or subtracting an offset from the index of the first merge candidate can be selected as a second merge candidate. For example, a merge candidate with an index indicated by merge_idx sent from the bitstream as a signal can be selected as the first merge candidate, and a merge candidate with an index of merge_idx + 1 can be selected as the second merge candidate.

[0462] In this scenario, when the first merge candidate is an affine merge candidate or is derived from a sub-block of a coded block encoded using an affine motion model, the affine merge candidate or the merge candidate derived from a sub-block of a coded block encoded using an affine motion model should be set as the second merge candidate. When the merge candidate with merge_idx + 1 as its index is not an affine merge candidate or is not derived from a sub-block of a coded block encoded using an affine motion model, other merge candidates can be set as the second merge candidate. Here, other merge candidates can be the merge candidates with the smallest difference from merge_idx + 1 among the first merge candidate, affine merge candidates, or merge candidates derived from a sub-block of a coded block encoded using an affine motion model.

[0463] On the other hand, when the first merge candidate is a non-affine merge candidate, it should be set as the second merge candidate. When the merge candidate with the index merge_idx + 1 is an affine merge candidate, or is derived from a sub-block of a coded block encoded using an affine motion model, other merge candidates can be set as the second merge candidate. Here, other merge candidates can be the first merge candidate or the non-affine merge candidate with the smallest difference from merge_idx + 1. For example, in the example shown in Figure 33, merge candidate A3 with the index merge_idx + 2 is set as the second merge candidate.

[0464] As another example, motion compensation prediction can be performed by combining a translational merging pattern with an affine merging pattern. That is, either the first or second merging candidate can be an affine merging candidate, while the other can be a non-affine merging candidate.

[0465] A combined prediction block can be obtained by weighting and summing a first prediction block derived from a first merge candidate and a second prediction block derived from a second merge candidate. In this case, the weighting value applied to the first prediction block can be set to be greater than the weighting value applied to the second prediction block.

[0466] Alternatively, the weighting values ​​can be determined based on the motion information of the first and second merged candidates. For example, the weighting values ​​applied to the first and second prediction blocks can be determined based on the difference in the output order between the reference image and the current image. Specifically, the greater the difference in the output order between the reference image and the current image, the smaller the weighting value applied to the prediction block can be.

[0467] Alternatively, the weighting values ​​applied to the first and second prediction blocks can be determined by considering the size or shape of the candidate block used to derive the first merge candidate (hereinafter referred to as the first candidate block) and the candidate block used to derive the second merge candidate (hereinafter referred to as the second candidate block). For example, the weighting value applied to the prediction block derived from a candidate block in the first or second candidate block whose shape is similar to the current block can be set to a larger value. On the other hand, the weighting value applied to the prediction block derived from a candidate block in the first or second candidate block whose shape is not similar to the current block can be set to a smaller value.

[0468] Figure 34 is a diagram illustrating an example of determining the weighting applied to the prediction block based on the shape of the candidate block.

[0469] Assume the current block is a non-square with a width greater than its height.

[0470] A first prediction block and a second prediction block can be derived based on a first merge candidate and a second merge candidate, and a combined prediction block can be generated based on a weighted sum of the first prediction block and the second prediction block. In this case, the weighting values ​​applied to the first prediction block and the second prediction block can be determined based on the shapes of the first candidate block and the second candidate block.

[0471] For example, in the example shown in Figure 34, the first candidate block is a square, and the second candidate block is a non-square with a width greater than its height. Since the shape of the second candidate block is the same as the current block, the weighting applied to the second prediction block can be set to be greater than the weighting applied to the first prediction block. For example, 5 / 8 of the weighting can be applied to the second prediction block, and 3 / 8 of the weighting can be applied to the first prediction block. Equation 14 shows an example of deriving a combined prediction block based on the weighted sum of the first and second prediction blocks.

[0472] Equation 14

[0473] P(x, y) represents the combined prediction block, P1(x, y) represents the first prediction block, and P2(x, y) represents the second prediction block.

[0474] As another example, the weighting values ​​applied to the first and second prediction blocks can be determined based on the shape of the current block. For instance, when the current block is a non-square with a width greater than its height, a larger weighting value can be applied to the prediction block generated based on the merge candidate derived from the candidate block located above the current block among the first and second merge candidates. When both the first and second merge candidates are derived from the candidate block located above the current block, the weighting values ​​applied to the first and second prediction blocks can be set in the same way. On the other hand, when the current block is a non-square with a height greater than its width, a larger weighting value can be applied to the prediction block generated based on the merge candidate derived from the candidate block located to the left of the current block among the first and second merge candidates. When both the first and second merge candidates are derived from the candidate block located to the left of the current block, the weighting values ​​applied to the first and second prediction blocks can be set in the same way. When the current block is a square, the weighting values ​​applied to the first and second prediction blocks can be set in the same way.

[0475] As another example, the weighting applied to each prediction block can be determined based on the distance between the current block and candidate blocks. Here, the distance can be derived based on the difference in x-axis coordinates, the difference in y-axis coordinates, or the minimum of both from the current block. The weighting applied to prediction blocks derived from merge candidates that are closer to the current block can be set to be greater than the weighting applied to prediction blocks derived from merge candidates that are farther from the current block. For example, in the example shown in Figure 31, the first merge candidate is derived from an adjacent block adjacent to the current block, and the second merge candidate is derived from a non-adjacent block not adjacent to the current block. In this case, since the x-axis distance between the first candidate block and the current block is less than the x-axis distance between the second candidate block and the current block, the weighting applied to the first prediction block can be set to be greater than the weighting applied to the second prediction block.

[0476] Alternatively, when both the first and second merge candidates are derived from non-adjacent blocks, a larger weighting value can be assigned to the predicted block derived from a non-adjacent block that is closer to the current block. For example, in the example shown in Figure 32, since the y-axis distance between the first candidate block and the current block is smaller than the y-axis distance between the second candidate block and the current block, the weighting value applied to the first predicted block can be set to be greater than the weighting value applied to the second predicted block.

[0477] The integrated motion information can be derived based on the first and second merging candidates, and motion compensation prediction can be performed on the current block based on the integrated motion information. For example, the motion vector of the current block can be derived based on the average or weighted sum of the motion vectors of the first and second merging candidates. In this case, the weighted values ​​applied to the motion vectors of the first and second merging candidates can be determined using the embodiments described above.

[0478] When the first merge candidate is a non-affine merge candidate and the second affine merge candidate is an affine merge candidate, the motion vector of the current block can be derived by scaling the motion vector of the second merge candidate. Equation 15 shows an example of deriving the motion vector of the current block.

[0479] Equation 15

[0480] In Equation 15, (mvX, mvY) represent the motion vector of the current block, (mv0x, mv0y) represent the motion vector of the first merging candidate, and (mv1x, mv1y) represent the motion vector of the second merging candidate. M represents the scaling parameter. M can be predefined in the encoder and decoder. Alternatively, the value of the scaling parameter M can be determined based on the size of the current block or the candidate block. For example, M can be set to 3 when the width or height of the second candidate block is greater than 32, and otherwise M can be set to 2.

[0481] Motion information for the first or second merge candidate can be stored as motion information for a block to which a merge prediction mode combining the merge mode and the merge mode is applied. The stored motion information can be used to derive motion information for blocks that will later be encoded / decoded.

[0482] Alternatively, the block can be divided into multiple sub-blocks, and the motion information of the first merging candidate and the motion information of the second merging candidate can be stored in the motion information of each sub-block. In this case, the motion information of some sub-blocks among the multiple sub-blocks can be set as the motion information of the first merging candidate, and the motion information of other sub-blocks can be set as the motion information of the second merging candidate.

[0483] Alternatively, the integrated motion information derived from the motion information of the first and second merge candidates can be stored as motion information of a block to which a merge prediction mode combining the merge pattern and the merge pattern is applied.

[0484] In a prediction mode that combines merging mode and advanced motion vector prediction mode, a first prediction block can be generated using motion information derived from merging candidates, and a second prediction block can be generated using motion vectors derived from motion vector prediction candidates.

[0485] In advanced motion vector prediction mode, motion vector prediction candidates can be derived from neighboring blocks adjacent to the current block or from co-located blocks within the same image. Then, any one of several motion vector prediction candidates can be specified, and the specified candidate can be set as the motion vector prediction result for the current block. The motion vector for the current block can then be derived by adding the current block's motion vector prediction result to the motion vector difference.

[0486] In prediction modes that combine merging mode and advanced motion vector prediction mode, merge candidates and motion vector prediction candidates can be derived from the same candidate block. For example, if a merge candidate is specified via `merge_idx`, the motion vectors of the candidate block used to derive the specified merge candidate can be set as the motion vector prediction result. Alternatively, if a motion vector prediction candidate is specified via `mvp_flag`, the merge candidate can be selected from the candidate block used to derive the specified merge candidate.

[0487] Alternatively, the candidate blocks used to derive merge candidates and the candidate blocks used to derive motion vector prediction candidates can be different. For example, if a merge candidate is selected from a candidate block located above the current block, it can be set to select a motion vector prediction candidate derived from a candidate block located to the left of the current block.

[0488] Alternatively, when the merge candidate selected by index information and the motion vector prediction candidate selected by index information are derived from the same candidate block, the motion vector prediction candidate can be replaced by a motion vector prediction candidate derived from an adjacent candidate block adjacent to the candidate block, or the merge candidate can be replaced by a merge candidate derived from an adjacent candidate block.

[0489] Figure 35 is a diagram illustrating an example of a replacement merge candidate.

[0490] In the example shown in Figure 35, a merge candidate and a motion vector prediction candidate derived from the candidate block at position A2 are illustrated. As shown, when the merge candidate and the motion vector prediction candidate are derived from the same candidate block, a merge candidate or motion vector prediction candidate derived from a candidate block adjacent to that candidate block can be used instead of the merge candidate or the motion vector prediction candidate itself. For example, as shown in Figure 35, the merge candidate at position A1 can be used instead of the merge candidate at position A2.

[0491] A first prediction block can be derived based on the merging candidates of the current block, and a second prediction block can be derived based on the motion vector prediction candidates. A combined prediction block can then be derived by a weighted sum of the first and second prediction blocks. In this case, the weighting applied to the second prediction block generated using the advanced motion vector prediction mode can be set to be greater than the weighting applied to the first prediction block generated using the merging mode.

[0492] The residual image can be derived by subtracting the predicted image from the original image. In this case, when the residual image is transformed into the frequency domain, even if high-frequency components are removed, the subjective image quality of the video is not significantly degraded. Therefore, reducing the value of high-frequency components or setting the value of high-frequency components to 0 can improve compression efficiency without causing significant visual distortion. Reflecting these characteristics, the current block can be transformed to decompose the residual image into 2D frequency components. This transformation can be performed using transformation techniques such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST).

[0493] DCT uses cosine transform to decompose (or transform) the residual image into 2D frequency components, while DST uses sine transform to decompose (or transform) the residual image into 2D frequency components. As the transformed result of the residual image, the frequency components can be represented as the basic image. For example, performing a DCT transform on a block of size N×N yields N... 2 There are several basic pattern components. The magnitudes of each basic pattern component within an N×N block can be obtained through transformation. Depending on the transformation technique used, the magnitudes of the basic pattern components can be referred to as DCT coefficients or DST coefficients.

[0494] The Direct Transformation Technique (DCT) is primarily used to transform images with a high proportion of low-frequency non-zero components. The Direct Transformation Technique (DST) is primarily used for images with a high proportion of high-frequency components.

[0495] Transformation techniques other than DCT or DST can also be used to transform residual images.

[0496] The process of transforming the residual image into two-dimensional frequency components is referred to as two-dimensional image transformation. Furthermore, the magnitudes of the fundamental pattern components obtained from the transformation are called transformation coefficients. For example, transformation coefficients can refer to DCT coefficients or DST coefficients. When the primary and secondary transformations, which will be described later, are applied simultaneously, the transformation coefficients can represent the magnitudes of the fundamental pattern components generated by the secondary transformation.

[0497] Transform techniques can be determined on a block-by-block basis. A transform technique can be determined based on at least one of the prediction mode, the size of the current block, or the shape of the current block. For example, when the current block is encoded in intra-prediction mode and the size of the current block is less than N×N, the transform technique DST can be used to perform the transform. On the other hand, when the aforementioned conditions cannot be met, the transform technique DCT can be used to perform the transform.

[0498] In the residual image, a portion of the block may not undergo 2D image transformation. This omission of 2D image transformation is called transform skipping. When transform skipping is applied, quantization can be applied to the residual values ​​for which no transformation was performed.

[0499] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the DCT- or DST-based transformation can be defined as the primary transformation, and the process of transforming the block again using the primary transformation can be called the secondary transformation.

[0500] The main transform can be performed using any of a number of transform kernel candidates. For example, the main transform can be performed using any of DCT2, DCT8, or DCT7.

[0501] Different transform kernels can be used for the horizontal and vertical directions. Information representing combinations of horizontal and vertical transform kernels can also be transmitted via a bitstream signal.

[0502] The execution units for the primary and secondary transformations will differ. For example, the primary transformation can be performed on an 8 × 8 block, and the secondary transformation can be performed on a 4 × 4 sub-block within the transformed 8 × 8 block. In this case, the transformation coefficients of the remaining region where the secondary transformation is not performed can also be set to 0.

[0503] Alternatively, a primary transformation can be performed on a 4×4 block, and a secondary transformation can be performed on an 8×8 region of size that includes the 4×4 block of the transformation.

[0504] Information indicating whether a secondary transformation is to be performed can be sent via signals through the bitstream.

[0505] Alternatively, the decision to perform a secondary transformation can be based on whether the horizontal and vertical transformation kernels are the same. For example, a secondary transformation can only be performed if the horizontal and vertical transformation kernels are the same. Alternatively, a secondary transformation can only be performed if the horizontal and vertical transformation kernels are different.

[0506] Alternatively, quadratic transformations are permitted only when the horizontal and vertical transformations utilize predefined transformation kernels. For example, quadratic transformations are permitted when the DCT2 transformation kernel is used for both horizontal and vertical transformations.

[0507] Alternatively, the decision to perform a quadratic transform can be based on the number of non-zero transform coefficients in the current block. For example, if the number of non-zero transform coefficients in the current block is less than or equal to a threshold, the quadratic transform can be set to not be used, and if the number of non-zero transform coefficients in the current block is greater than the threshold, the quadratic transform can be used. Alternatively, the quadratic transform can be set to be used only if the current block is coded with intra-frame prediction.

[0508] Based on the shape of the current block, the size or shape of the sub-block to be subjected to the secondary transformation can be determined.

[0509] Figures 36 and 37 are diagrams showing the sub-blocks to which a second transformation will be performed.

[0510] When the current block is square, after performing the main transformation, a secondary transformation can be performed on the N×N sub-block in the upper left corner of the current block. For example, when the current block is an 8×8 coded block, a secondary transformation can be performed on the 4×4 sub-block in the upper left corner of the current block after performing the main transformation on the current block (see Figure 36).

[0511] When the current block is a non-square with a width greater than 4 times its height, after performing the primary transformation, a secondary transformation can be performed on the sub-block of size (kN) × (4 kN) at the top left of the current block. For example, when the current block is a non-square with a size of 16 × 4, a secondary transformation can be performed on the 2 × 8 sub-block at the top left corner of the current block after performing the primary transformation on the current block (see Figure 37(a)).

[0512] When the current block is a non-square with a height greater than 4 times its width, after performing the primary transformation, a secondary transformation can be performed on the sub-block of size (4kN) × (kN) at the top left of the current block. For example, when the current block is a non-square with a size of 16 × 4, a secondary transformation can be performed on the 2 × 8 sub-block at the top left corner of the current block after performing the primary transformation on the current block (see Figure 37(b)).

[0513] The decoder can perform the inverse of the second inverse transform (second inverse transform), and can perform the inverse of the main transform (first inverse transform) on its result. The residual signal of the current block can be obtained as the result of the second inverse transform and the first inverse transform.

[0514] Information indicating the transformation type of the current block can be transmitted via signals through the bitstream. This information may be an index, tu_mts_idx, indicating a combination of horizontal and vertical transformation types.

[0515] The vertical and horizontal transform kernels can be determined based on the transform type candidates specified by the index information tu_mts_idx. Tables 11 and 12 show the combinations of transform types based on tu_mts_idx.

[0516] Table 11

[0517] Table 12

[0518] The transform type can be determined as DCT2, DST7, DCT8, or a skip transform. Alternatively, in addition to transform skipping, transform type combination candidates can be constructed using only the transform kernel.

[0519] When using Table 11, if tu_mts_idx is 0, transformation skipping can be applied in both the horizontal and vertical directions. If tu_mts_idx is 1, DCT2 can be applied in both the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 can be applied in the horizontal direction and DCT7 in the vertical direction.

[0520] When using Table 12, DCT2 can be applied in both the horizontal and vertical directions when tu_mts_idx is 0. If tu_mts_idx is 1, transform skipping can be applied in both the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 can be applied in the horizontal direction and DCT7 in the vertical direction.

[0521] Whether to encode index information can be determined based on at least one of the following: the size, shape, or number of non-zero coefficients of the current block. For example, if the number of non-zero coefficients is equal to or less than a threshold, index information is not sent, and a default transform type can be applied to the current block. The default transform type could be DST7. Alternatively, the default transform type may differ depending on the size, shape, or intra-prediction mode of the current block.

[0522] The threshold can be determined based on the size or shape of the current block. For example, the threshold can be set to 2 when the size of the current block is less than or equal to 32 × 32, and to 4 when the current block is greater than 32 × 32 (e.g., when the current block is a coded block of size 32 × 64 or 64 × 32).

[0523] Multiple lookup tables can be pre-stored in the encoder / decoder. In these multiple lookup tables, at least one of the following can be different: the index value assigned to a transform type combination candidate, the type of transform type combination candidate, or the number of transform type combination candidates.

[0524] The lookup table for the current block can be selected based on at least one of the following: the size, shape, prediction mode, intra-frame prediction mode, whether a second transformation is applied, or whether the transformation is skipped and applied to adjacent blocks.

[0525] For example, when the current block size is 4 × 4 or smaller, or when the current block is encoded by inter-frame prediction, the lookup table in Table 11 can be used, and when the current block size is greater than 4 × 4, or when the current block is encoded by intra-block prediction, the lookup table in Table 12 can be used.

[0526] Alternatively, information indicating any of the multiple lookup tables can be sent via signaling in the bitstream. The decoder can then select the lookup table for the current block based on this information.

[0527] As another example, the index assigned to a transform type combination candidate can be adaptively determined based on at least one of the following: the size and shape of the current block, the prediction mode, the intra-prediction mode, whether a second transform is applied, or whether transform skipping is applied to adjacent blocks. For example, when the current block size is 4 × 4, the index assigned to transform skipping can have a smaller value than the index assigned to transform skipping when the current block size is greater than 4 × 4. Specifically, when the current block size is 4 × 4, an index 0 can be assigned to transform skipping; when the current block size is greater than 4 × 4 but less than 16 × 16, an index greater than 0 (e.g., index 1) can be assigned to transform skipping; when the current block size is greater than 16 × 16, a maximum value (e.g., 5) can be assigned as the index for transform skipping.

[0528] Alternatively, when the current block is coded with inter-frame prediction, index 0 can be assigned to the transform skip. When the current block is coded with intra-frame prediction, an index greater than 0 (e.g., index 1) can be assigned to the transform skip.

[0529] Alternatively, when the current block is a 4 × 4 block coded with inter-frame prediction, index 0 can be skipped in the transform. On the other hand, when the current block is not coded with inter-frame prediction, or when the current block is larger than 4 × 4, an index with a value greater than 0 (e.g., index 1) can be skipped in the transform.

[0530] Transform type combination candidates that differ from those listed in Tables 11 and 12 can be defined and used. For example, a transform type combination candidate with a transform kernel such as DCT7, DCT8, or DST2 can be used to apply transform skipping to horizontal or vertical transforms, as well as other transforms. In this case, it can be determined whether to use transform skipping as a transform type candidate for the horizontal or vertical direction based on at least one of the current block size (e.g., width and / or height), shape, prediction mode, or intra-frame prediction mode.

[0531] Alternatively, information indicating whether a specified transform type candidate is available can be transmitted via signaling in the bitstream. For example, flags indicating whether a transform can be skipped as a transform type candidate for the horizontal and vertical directions can be transmitted via signaling. Based on these flags, it can be determined whether a specified transform type combination candidate is included among a plurality of transform type combination candidates.

[0532] Alternatively, information indicating whether to apply a transform type candidate to the current block can be sent via signaling in the bitstream. For example, a flag cu_mts_flag indicating whether to apply DCT2 to the horizontal and vertical directions can be sent via signaling. When cu_mts_flag is 1, DCT2 can be set as the transform kernel for both the vertical and horizontal directions. When cu_mts_flag is 0, DCT8 or DST7 can be set as the transform kernel for both the vertical and horizontal directions. Alternatively, when cu_mts_flag is 0, information tu_mts_idx can be sent via signaling to specify any one of several transform type combination candidates.

[0533] When the current block is a non-square with a width greater than its height or a non-square with a height greater than its width, the encoding of cu_mts_flag can be omitted, and the value of cu_mts_flag can be regarded as 0.

[0534] The number of available transform type combination candidates can be set differently depending on the size, shape, or intra-prediction mode of the current block. For example, more than three transform type combination candidates can be used when the current block is square, while two transform type combination candidates can be used when the current block is not square. Alternatively, when the current block is square, only transform type combination candidates with different transform types in the horizontal and vertical directions can be utilized.

[0535] When there are three or more transform type combination candidates available for the current block, an index information tu_mts_idx indicating one of the transform type combination candidates can be sent using a signal. On the other hand, when there are two transform type combination candidates available for the current block, a flag mts_flag indicating either of the transform type combination candidates can be sent using a signal. Table 13 below illustrates the process for encoding information for a specified transform type combination candidate based on the shape of the current block.

[0536] Table 13

[0537] Based on the shape of the current block, the indices of the candidate transformation type combinations are rearranged (or reordered). For example, the indices assigned to candidates of transformation type combinations when the current block is a square can be different from those assigned when the current block is not a square. For example, when the current block is a square, transformation type combinations can be selected based on Table 14 below, and when the current block is not a square, transformation type combinations can be selected based on Table 15 below.

[0538] Table 14

[0539] Table 15

[0540] The transform type can be determined based on the number of non-zero coefficients in the horizontal or vertical direction of the current block. The number of non-zero coefficients in the horizontal direction represents the number of non-zero coefficients included in 1×N (where N is the width of the current block), and the number of non-zero coefficients in the vertical direction represents the number of non-zero coefficients included in N×1 (where N is the height of the current block). When the maximum value of the non-zero coefficients in the horizontal direction is less than or equal to a threshold, a primary transform type can be applied in the horizontal direction; when the maximum value of the non-zero coefficients in the horizontal direction is greater than the threshold, a secondary transform type can be applied in the horizontal direction. Similarly, when the maximum value of the non-zero coefficients in the vertical direction is less than or equal to a threshold, a primary transform type can be applied in the vertical direction; when the maximum value of the non-zero coefficients in the vertical direction is greater than the threshold, a secondary transform type can be applied in the vertical direction.

[0541] Figure 38 is a diagram illustrating an example of determining the transformation type of the current block.

[0542] For example, when the current block is encoded by intra-frame prediction and the maximum value of the non-zero coefficients in the horizontal direction of the current block is 2 or less (see Figure 38(a)), the transformation type in the horizontal direction can be determined as DST7.

[0543] When the current block is encoded by intra-frame prediction and the maximum value of the non-zero coefficients in the vertical direction of the current block is greater than 2 (see Figure 38(b)), the transform type in the vertical direction can be determined as DCT2 or DCT8.

[0544] Information indicating whether the transform type of the current block can be explicitly determined based on the information signaled from the bitstream can be transmitted via signaling. For example, information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for blocks coded via intra-frame predictive coding and / or information sps_explicit_inter_mts_flag indicating whether explicit transform type determination is allowed for blocks coded via inter-frame predictive coding can be transmitted via signaling at the sequence level.

[0545] When explicit transform type determination is allowed, the transform type of the current block can be determined based on the index information tu_mts_idx transmitted from the bitstream using signals. Conversely, when explicit transform type determination is not allowed, the transform type can be determined based on at least one of the following: the size of the current block, the shape of the current block, whether transforms of sub-block units are allowed, and the position of sub-blocks including non-zero transform coefficients. For example, the horizontal transform type of the current block can be determined based on its width, and the vertical transform type can be determined based on its height. For example, when the width of the current block is less than 4 or greater than 16, the horizontal transform type can be determined as DCT2. Otherwise, the horizontal transform type can be determined as DST7. When the height of the current block is less than 4 or greater than 16, the vertical transform type can be determined as DCT2. Otherwise, the vertical transform type can be determined as DST7. Here, to determine the horizontal and vertical transform types, a threshold compared to the width and height can be determined based on at least one of the current block's size, shape, and intra-prediction mode.

[0546] Alternatively, when the current block is a square with the same height and width, the horizontal and vertical transformation types can be set to the same value. When the current block is a non-square with different heights and widths, the horizontal and vertical transformation types can be set to different values. For example, when the width of the current block is greater than its height, the horizontal transformation type can be set to DST7, and the vertical transformation type to DCT2. When the height of the current block is greater than its width, the vertical transformation type can be set to DST7, and the horizontal transformation type to DCT2.

[0547] Depending on whether explicit transform type determination is allowed, the number and / or types of transform type candidates, or the number and / or types of combinations of transform type candidates, can vary. For example, when explicit transform type determination is allowed, DCT2, DST7, and DCT8 can be used as transform type candidates. Therefore, each of the horizontal and vertical transform types can be set to DCT2, DST8, or DCT8. If explicit transform type determination is not allowed, only DCT2 and DST7 can be used as transform type candidates. Therefore, each of the horizontal and vertical transform types can be determined as DCT2 or DST7.

[0548] If transform and quantization are performed in the encoder, the decoder can obtain the residual block through inverse quantization and inverse transform. The decoder then adds the predicted block and the residual block together to obtain the reconstructed block of the current block.

[0549] If the reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss during quantization and encoding. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter will be referred to as the first reconstructed block, and the reconstructed block after applying the in-loop filter will be referred to as the second reconstructed block.

[0550] A second reconstructed block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block. In this case, SAO or ALF can be applied after the deblocking filter.

[0551] Deblocking filters are used to mitigate the image quality degradation (blocking artifact) that occurs at block boundaries when quantization is performed on a block-by-block basis. To apply a deblocking filter, the block strength (BS) between the first reconstructed block and its adjacent reconstructed blocks can be determined.

[0552] Figure 39 is a flowchart illustrating the process of determining block strength.

[0553] In the example shown in Figure 39, P represents the first reconstructed block, and Q represents the adjacent reconstructed block. The adjacent reconstructed block can be adjacent to the left or top of the current block.

[0554] The example shown in Figure 39 illustrates how the prediction mode considering P and Q, whether non-zero transform coefficients are included, whether the same reference image is used to perform inter-frame prediction, and whether the difference in motion vectors is greater than or equal to a threshold are used to determine the block strength.

[0555] Based on the block strength, it can be determined whether a deblocking filter has been applied. For example, if the block strength is 0, filtering may not be performed.

[0556] SAO (Sound Analysis and Offset) is used to mitigate the ringing artifact that occurs when performing quantization in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. Methods for determining the offset include Edge Offset (EO) or Band Offset (BO). EO indicates a method of determining the offset of the current sample based on the pattern of surrounding pixels. BO indicates a method of applying a common offset to a set of pixels with similar brightness values ​​within a region. Specifically, pixel brightness is divided into 32 equal intervals, and pixels with similar brightness values ​​are grouped together. For example, four adjacent bands out of the 32 bands are grouped together, and samples belonging to those four bands can have the same offset applied.

[0557] ALF is a method for generating a second reconstructed image by applying a predefined filter of size / shape to a first reconstructed image or a reconstructed image with a deblocking filter applied. Equation 16 below shows an example of applying ALF.

[0558] Equation 16

[0559] You can select any of the predefined filter candidates at the image, coding tree unit, coding block, prediction block, or transform block level. Each filter candidate may have a different size or shape.

[0560] Figure 40 is a diagram showing predefined filter candidates.

[0561] As shown in the example in Figure 40, at least one of the following rhombuses can be selected: 5 × 5, 7 × 7, and 9 × 9.

[0562] Only rhombuses of size 5 × 5 can be used for chromaticity components.

[0563] Embodiments described with a focus on the decoding or encoding process are also included within the scope of this invention. Variations of multiple embodiments described in a predetermined order, in a different order than those described, are also included within the scope of this invention.

[0564] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention, and they can be performed simultaneously or in a different order as needed. Furthermore, in the above embodiments, the structural elements constituting the block diagrams (e.g., units, modules, etc.) can also be implemented as hardware devices or software, and multiple structural elements can be combined to implement a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can individually or in combination include program instructions, data files, data structures, etc. Examples of computer-readable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy optical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. The hardware device can be configured to operate as one or more software modules to perform the processing according to the invention, and vice versa.

[0565] Industrial applicability: This invention can be applied to electronic devices that encode / decode video.

Claims

1. A video decoding method, comprising the following steps: Determine whether to apply the combined prediction mode to the current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block, wherein the first prediction block is obtained based on the motion information of the merging candidate of the current block and the second prediction block is obtained based on the intra-frame prediction mode of the current block; and obtain a third prediction block of the current block based on the weighted sum of the first prediction block and the second prediction block, wherein when the combined prediction mode is applied to the current block, triangle partitioning is disabled for the current block, wherein the second prediction block is obtained based on the reference sample lines included in the adjacent reference sample lines.

2. A video encoding method, comprising the following steps: Determine whether to apply the combined prediction mode to the current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block, wherein the first prediction block is obtained based on the motion information of the merging candidate of the current block and the second prediction block is obtained based on the intra-frame prediction mode of the current block; and obtain a third prediction block of the current block based on the weighted sum of the first prediction block and the second prediction block, wherein when the combined prediction mode is applied to the current block, triangle partitioning is disabled for the current block, wherein the second prediction block is obtained based on the reference sample lines included in the adjacent reference sample lines.

3. A video decoding apparatus, comprising a prediction unit, the prediction unit being configured to: determine whether to apply a combined prediction mode to a current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block; and obtain a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, wherein, The first prediction block is obtained based on the motion information of the merging candidates of the current block, and the second prediction block is obtained based on the intra-frame prediction mode of the current block, wherein when the combined prediction mode is applied to the current block, triangle partitioning is disabled for the current block, and the second prediction block is obtained based on the reference sample lines included in the adjacent reference sample lines.

4. A video encoding apparatus, comprising a prediction unit, the prediction unit being configured to: determine whether to apply a combined prediction mode to a current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block; and obtain a third prediction block of the current block based on a weighted sum operation of the first prediction block and the second prediction block, wherein, The first prediction block is obtained based on the motion information of the merging candidates of the current block, and the second prediction block is obtained based on the intra-frame prediction mode of the current block, wherein when the combined prediction mode is applied to the current block, triangle partitioning is disabled for the current block, and the second prediction block is obtained based on the reference sample lines included in the adjacent reference sample lines.

5. A video decoder, comprising a processor and a memory configured to store a computer program executable on the processor, wherein the processor is configured to: determine whether to apply a combined prediction mode to a current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block; and obtain a third prediction block of the current block based on a weighted sum of the first prediction block and the second prediction block, wherein... The first prediction block is obtained based on the motion information of the merging candidates of the current block, and the second prediction block is obtained based on the intra-frame prediction mode of the current block, wherein when the combined prediction mode is applied to the current block, triangle partitioning is disabled for the current block, and the second prediction block is obtained based on the reference sample lines included in the adjacent reference sample lines.

6. A video encoder, comprising a processor and a memory configured to store a computer program executable on the processor, wherein the processor is configured to: determine whether to apply a combined prediction mode to a current block; when the combined prediction mode is applied to the current block, obtain a first prediction block and a second prediction block of the current block; and obtain a third prediction block of the current block based on a weighted sum of the first prediction block and the second prediction block, wherein... The first prediction block is obtained based on the motion information of the merging candidates of the current block, and the second prediction block is obtained based on the intra-frame prediction mode of the current block, wherein when the combined prediction mode is applied to the current block, triangle partitioning is disabled for the current block, and the second prediction block is obtained based on the reference sample lines included in the adjacent reference sample lines.

7. A storage medium having an executable program stored thereon, wherein, When the executable program is executed by the processor, the method as described in claim 1 is performed.

8. A storage medium having an executable program stored thereon, wherein, When the executable program is executed by the processor, the method as described in claim 2 is performed.

9. A method for transmitting a code stream, characterized in that, The video encoding method as described in claim 2 is used to generate a bitstream; and the bitstream is transmitted.

10. A method for receiving a code stream, characterized in that, Receive the bitstream; and perform the video decoding method as described in claim 1 to decode the bitstream to generate a video or image.

11. A storage medium storing a bitstream generated by a processor performing the video encoding method as described in claim 2.

Images