Video signal encoding / decoding method and apparatus therefor

Abstract

The video decoding method of the present application comprises the following steps: determining whether to divide a coding block into a first prediction unit and a second prediction unit; when determining to divide the coding block, determining a division type of the coding block; deriving first motion information of the first prediction unit and second motion information of the second prediction unit of the coding block; and obtaining a prediction sample in the coding block based on the first motion information and the second motion information.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese patent application No. 201980070594.4, entitled "Video Signal Encoding / Decoding Method and Apparatus for the Method", which entered the Chinese national phase of PCT international patent application PCT / KR2019 / 015096 filed on November 7, 2019.

[0003] Cross-reference to related applications

[0004] This application is based on and claims priority to Korean Patent Application No. 10-2018-0136255, filed on November 8, 2018, the entire contents of which are incorporated herein by reference.

[0005] This application is based on and claims priority to Korean Patent Application No. 10-2018-0148890, filed on November 27, 2018, the entire contents of which are incorporated herein by reference.

[0006] This application is based on and claims priority to Korean Patent Application No. 10-2018-0149064, filed on November 27, 2018, the entire contents of which are incorporated herein by reference. Technical Field

[0007] This invention relates to a video signal encoding / decoding method and an apparatus for the method. Background Technology

[0008] As display panels become larger, there is a growing need for higher-quality video services. The biggest challenge with high-definition video services is the dramatic increase in data volume. To address this, research is actively underway to improve video compression rates. As a representative example, in 2009, the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013, boasting compression performance approximately twice that of H.264 / AVC. However, with the rapid development of high-definition video services, the limitations of HEVC have gradually become apparent. Summary of the Invention

[0009] Technical problems to be solved

[0010] The purpose of this invention is to provide a method for dividing a coding block into multiple prediction blocks when encoding / decoding a video signal, and an apparatus for performing the method.

[0011] The purpose of this invention is to provide a method for deriving motion information of each prediction block in a plurality of prediction blocks when encoding / decoding a video signal, and an apparatus for performing the method.

[0012] The purpose of this invention is to provide a method for deriving merging candidates using an inter-frame motion information list when encoding / decoding video signals, and an apparatus for performing the method.

[0013] The technical problems to be solved by the present invention are not limited to those mentioned above, and those skilled in the art to which the present invention pertains will clearly understand other technical problems not mentioned through the following description.

[0014] Technical solution

[0015] The video signal encoding / decoding method of the present invention includes the following steps: determining whether to divide a coding block into a first prediction unit and a second prediction unit; when it is determined that the coding block should be divided, determining the division type of the coding block; deriving first motion information of the first prediction unit and second motion information of the second prediction unit of the coding block; and obtaining prediction samples in the coding block based on the first motion information and the second motion information. In this case, the first motion information of the first prediction unit can be obtained according to a first merging candidate, wherein the first merging candidate is specified by first index information among a plurality of merging candidates included in the merging candidate list, and the second motion information of the second prediction unit can be obtained according to a second merging candidate, wherein the second merging candidate is specified by second index information among a plurality of merging candidates included in the merging candidate list.

[0016] In the video signal encoding / decoding method of the present invention, when the value of the second index information is equal to or greater than the value of the first index information, the second merging candidate may have a value obtained by adding 1 to the value of the second index information as an index.

[0017] In the video signal encoding / decoding method of the present invention, when the value of the second index information is less than the value of the first index information, the value of the second index information can be used as the index of the second merging candidate.

[0018] In the video signal encoding / decoding method of the present invention, when the predicted sample is included in the boundary region between the first prediction unit and the second prediction unit, the predicted sample can be derived by weighted sum operation of the first predicted sample derived based on the first motion information and the second predicted sample derived based on the second motion information.

[0019] In the video signal encoding / decoding method of the present invention, a first weighting value applied to the first prediction sample can be determined based on the x-axis coordinates and y-axis coordinates of the prediction sample.

[0020] In the video signal encoding / decoding method of the present invention, a second weighting value applied to the second prediction sample can be derived by subtracting the first weighting value from a constant.

[0021] In the video signal encoding / decoding method of the present invention, the size of the boundary region can be determined based on at least one of the size of the encoding block or the shape of the encoding block.

[0022] The features briefly outlined above are merely exemplary embodiments of the invention as described in the detailed description to follow, and do not limit the scope of the invention.

[0023] Invention Effects

[0024] According to the present invention, inter-frame prediction efficiency can be improved by providing a method that divides a coded block into multiple prediction blocks and derives motion information of each prediction block in the multiple prediction blocks.

[0025] According to the present invention, the efficiency of inter-frame prediction can be improved by providing a method for deriving merge candidates using a list of inter-frame motion information.

[0026] The effects that can be obtained in this invention are not limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art through the following description. Attached Figure Description

[0027] Figure 1 This is a block diagram of a video encoder according to an embodiment of the present invention.

[0028] Figure 2 This is a block diagram of a video decoder according to an embodiment of the present invention.

[0029] Figure 3 This is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0030] Figure 4 This is a diagram illustrating the various partitioning types of coded blocks.

[0031] Figure 5 This is a diagram illustrating an example of how coding tree units are divided.

[0032] Figure 6 This is a flowchart of the inter-frame prediction method according to an embodiment of the present invention.

[0033] Figure 7 It is a diagram showing the nonlinear motion of an object.

[0034] Figure 8 This is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0035] Figure 9 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0036] Figure 10 This is a diagram showing an example of the affine vectors of a sub-block under a 4-parameter motion model.

[0037] Figure 11 This is a diagram showing adjacent blocks that can be used to derive merge candidates.

[0038] Figure 12 This is a diagram illustrating how the affine seed vector of the current block is derived based on the affine seed vectors of neighboring affine blocks.

[0039] Figure 13 This is a diagram illustrating an example of setting the motion vector of a sub-block to an affine seed vector that affines the adjacent block.

[0040] Figures 14 to 16 This is a diagram showing the location of the reference sample.

[0041] Figure 17 This is a diagram illustrating an example of applying a modified affine merging vector derivation method.

[0042] Figure 18 This is a diagram illustrating an example of deriving an affine seed vector for an affine merge candidate based on multiple motion vectors of multiple sub-blocks belonging to adjacent blocks.

[0043] Figure 19 This is a diagram illustrating an example of deriving the affine seed vector of an affine merge candidate based on the motion vectors of multiple sub-blocks located to the left of the current block.

[0044] Figure 20 This is a diagram illustrating an example of deriving the affine seed vector of an affine merge candidate based on motion information of a non-adjacent or adjacent block located to the left of the current block.

[0045] Figure 21 This is a diagram showing the location of the blocks used to derive the affine seed vectors for affine merging candidates.

[0046] Figure 22 This is a diagram illustrating an example of deriving a combined merge candidate by combining multiple motion vectors of multiple adjacent blocks.

[0047] Figure 23 This is a diagram showing which adjacent blocks are unavailable.

[0048] Figure 24 This is a flowchart of the process of exporting motion information of the current block in merge mode.

[0049] Figure 25 This is a diagram illustrating an example of updating the list of inter-frame motion information.

[0050] Figure 26 This is a diagram illustrating an embodiment of updating the inter-frame merging candidate list.

[0051] Figure 27 This is a diagram illustrating an example of how the index of a previously stored inter-frame merge candidate is updated.

[0052] Figure 28 This is a diagram showing the location of a representative sub-block.

[0053] Figure 29 An example of generating a list of inter-frame motion information for different inter-frame prediction modes is shown.

[0054] Figure 30 This is a diagram illustrating an example of adding inter-frame merge candidates included in the long-term motion information list to the merge candidate list.

[0055] Figure 31 This is a diagram illustrating an example of performing redundancy checks only on some of the merge candidates.

[0056] Figure 32 This is a diagram illustrating an example of omitting redundancy checks for a specific merge candidate.

[0057] Figure 33 This is a diagram illustrating an example of using diagonals to divide a coded block into multiple prediction units.

[0058] Figure 34 This is a diagram illustrating an example of dividing a coded block into two prediction units.

[0059] Figure 35 A diagram showing an example of dividing a coded block into multiple prediction blocks of different sizes is provided.

[0060] Figure 36 This is a diagram showing adjacent blocks used to derive triangle merging candidates.

[0061] Figure 37 This is a diagram used to illustrate an example of determining the availability of neighboring blocks for each triangular prediction unit.

[0062] Figure 38 and Figure 39 This is a diagram illustrating an example of deriving a prediction sample based on a weighted sum of a first and a second prediction sample.

[0063] Figure 40 This is a flowchart illustrating an intra-frame prediction method according to an embodiment of the present invention.

[0064] Figure 41 This is a diagram illustrating the intra-frame prediction mode.

[0065] Figure 42 and Figure 43 This is a diagram illustrating an example of a one-dimensional arrangement of reference samples in a row.

[0066] Figure 44 This is a diagram showing the angle formed between the prediction pattern within the angular frame and a straight line parallel to the x-axis.

[0067] Figure 45 This is a diagram showing an example of obtaining a predicted sample when the current block is not a square.

[0068] Figure 46 This is a diagram illustrating the wide-angle intra-frame prediction mode.

[0069] Figure 47 This is a flowchart illustrating the process of determining block strength.

[0070] Figure 48 This is a graph showing predefined filter candidates. Detailed Implementation

[0071] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0072] Video encoding and decoding are performed on a block-by-block basis. For example, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, or reconstruction can be performed on encoded blocks, transform blocks, or prediction blocks.

[0073] Hereinafter, the block to be encoded / decoded will be referred to as the "current block". For example, depending on the current encoding / decoding process step, the current block can represent an encoded block, a transform block, or a prediction block.

[0074] Additionally, as used herein, the term "unit" refers to a basic unit used to perform a particular encoding / decoding process, and "block" can be understood as representing a sample array of a predetermined size. Unless otherwise stated, "block" and "unit" are used interchangeably. For example, in the embodiments described later, encoding block and encoding unit can be understood to have the same meaning.

[0075] Figure 1 This is a block diagram of a video encoder according to an embodiment of the present invention.

[0076] Reference Figure 1 The video encoding device 100 may include an image segmentation unit 110, a prediction unit 120 and 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy coding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0077] Figure 1 The components shown are illustrated individually to represent the distinct functionalities of the video encoding device, and do not imply that each component is composed of separate hardware or a single software component. That is, for ease of explanation, each structural element is listed and included as a representative structural element, and at least two structural elements can be combined into one structural element or a structural element can be divided into multiple structural elements to perform functions. Such embodiments integrating structural elements and embodiments separating structural elements are also within the scope of this invention, provided they do not depart from the spirit of the invention.

[0078] Furthermore, some structural elements are not essential for performing the essential functions of this invention, but are optional structural elements used only to improve performance. This invention can be implemented by including only the components necessary to achieve the essence of the invention (excluding structural elements used to improve performance), and structures including only the essential structural elements (excluding structural elements used to improve performance) are also within the scope of this invention.

[0079] The image partitioning unit 110 can divide the input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transformation unit (TU), or a coding unit (CU). The image partitioning unit 110 divides an image into a combination of multiple coding units, prediction units, and transformation units, and can select the combination of coding units, prediction units, and transformation units to encode the image based on a predetermined criterion (e.g., a cost function).

[0080] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quadtree structure can be used. A video or the largest coding unit can be used as the root to divide the image into other coding units, which can have as many child nodes as the number of coding units into which they are divided. Coding units that are no longer divided according to certain constraints become leaf nodes. That is, when it is assumed that a coding unit can only be divided into squares, a maximum of four other coding units can be divided into one coding unit.

[0081] In the embodiments of the present invention, the encoding unit may refer to a unit that performs encoding or a unit that performs decoding.

[0082] A prediction unit within a coding unit can be divided into at least one square or rectangle of the same size, or a prediction unit within a coding unit can be divided into units with a different shape and / or size than another prediction unit.

[0083] Intra-prediction can be performed when the prediction unit for intra-prediction based on the coding unit is not the smallest coding unit, without having to divide it into multiple prediction units N×N.

[0084] Prediction units 120 and 125 may include an inter-frame prediction unit 120 that performs inter-frame prediction and an intra-frame prediction unit 125 that performs intra-frame prediction. It can be determined whether inter-frame prediction or intra-frame prediction is used for the prediction unit, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) is determined based on each prediction method. In this case, the processing unit performing the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode may be determined by the prediction unit, and the prediction may be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Furthermore, prediction mode information, motion vector information, etc., used for prediction, along with the residual value, can be encoded in the entropy coding unit 165 and transmitted to the decoder. When using a specific coding mode, the original block can also be directly encoded and transmitted to the decoder without generating a prediction block through the prediction units 120 and 125.

[0085] The inter-frame prediction unit 120 can predict prediction units based on information from at least one of the previous or next images of the current image. In some cases, it can also predict prediction units based on information from a portion of the encoded region within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0086] The reference image interpolation unit receives reference image information from memory 155 and can generate pixel information of integer pixels or fractional pixels from the reference image. For luminance pixels, in order to generate pixel information of fractional pixels in units of 1 / 4 pixels, an 8th-order DCT-based interpolation filter with different filter coefficients can be used. For chrominance signals, in order to generate pixel information of fractional pixels in units of 1 / 8 pixels, a 4th-order DCT-based interpolation filter with different filter coefficients can be used.

[0087] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as the Full Search-based Block Matching Algorithm (FBMA), the Three-Step Search (TSS), and the New Three-Step Search Algorithm (NTS). Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 pixel or 1 / 4 pixel. In the motion prediction unit, different motion prediction methods can be used to predict the current prediction unit. These methods include skipping, merging, Advanced Motion Vector Prediction (AMVP), and Intra Block Copying.

[0088] The intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block (the reference pixel information is pixel information within the current image). If the adjacent block of the current prediction unit is a block that has already undergone inter-frame prediction and the reference pixel is a pixel that has undergone inter-frame prediction, the reference pixels included in the block that has undergone inter-frame prediction can be used as the reference pixel information for the surrounding blocks that have undergone intra-frame prediction. That is, if a reference pixel is unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0089] In intra-frame prediction, the prediction mode can have an angular prediction mode that uses reference pixel information based on the prediction direction and a non-angular mode that does not use direction information when performing prediction. The mode used to predict luminance information and the mode used to predict chrominance information can be different. To predict chrominance information, either the intra-frame prediction mode information used to predict luminance information or the predicted luminance signal information can be used.

[0090] When performing intra-frame prediction, if the size of the prediction unit is the same as the size of the transform unit, intra-frame prediction can be performed based on the pixels to the left, upper left, and upper above the prediction unit. However, when performing intra-frame prediction, if the size of the prediction unit is different from the size of the transform unit, intra-frame prediction can be performed based on the reference pixels of the transform unit. Alternatively, intra-frame prediction using an N×N partition can be applied only to the smallest coding unit.

[0091] After applying an Adaptive IntraSmoothing (AIS) filter to a reference pixel based on the prediction mode, the intra-prediction method can generate a prediction block. The type of adaptive intra-smoothing filter applied to the reference pixel may differ. To perform the intra-prediction method, the intra-prediction mode of the current prediction unit can be predicted based on the intra-prediction modes of prediction units surrounding it. When using mode information predicted from surrounding prediction units to predict the prediction mode of the current prediction unit, if the intra-prediction modes of the current prediction unit and those of surrounding prediction units are the same, predetermined flag information can be used to convey information indicating that the prediction modes of the current prediction unit and those of surrounding prediction units are the same. If the prediction modes of the current prediction unit and those of surrounding prediction units are different, entropy coding can be performed to encode the prediction mode information of the current block.

[0092] Additionally, a residual block including residual information can be generated, the residual information being the difference between the prediction unit performing the prediction based on the prediction unit generated in the prediction units 120, 125 and the original block of the prediction unit. The generated residual block can be input to the transformation unit 130.

[0093] In the transform unit 130, transform methods such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST) can be used to transform the residual block, which includes residual information between the original block and the prediction unit generated by the prediction units 120 and 125. The DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on the intra-frame prediction mode information of the prediction unit used to generate the residual block. Transformation of the residual block can also be skipped. A flag indicating whether to skip the transformation of the residual block can be encoded. Transformation skipping is allowed for residual blocks with a size below a threshold, or for luma or chroma components (4:4:4 format or below).

[0094] The quantization unit 135 can quantize the values ​​that have been transformed into the frequency domain in the transform unit 130. The quantization coefficients can be changed according to the importance of the block or image. The values ​​calculated in the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0095] The rearrangement unit 160 can rearrange the coefficient values ​​of the quantized residual values.

[0096] The rearrangement unit 160 can convert 2D block shape coefficients into 1D vector form using a coefficient scanning method. For example, the rearrangement unit 160 can use a zig-zag scan method to scan the DC coefficients and even the coefficients in the high-frequency domain, and convert them into 1D vector form. Depending on the size of the transform unit and the intra-frame prediction mode, instead of zig-zag scanning, vertical scanning along the column direction and horizontal scanning along the row direction can also be used to scan the 2D block shape coefficients. That is, the choice between zig-zag scanning, vertical scanning, and horizontal scanning can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0097] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. For example, entropy coding can use various coding methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0098] The entropy coding unit 165 can encode various information such as residual coefficient information and block type information, prediction mode information, partitioning unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding units originating from the rearrangement unit 160 and the prediction units 120 and 125.

[0099] The entropy coding unit 165 can entropy code the coefficient values ​​of the coding units input from the rearrangement unit 160.

[0100] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on multiple values ​​quantized by the quantization unit 135, and perform inverse transform on the values ​​transformed by the transform unit 130. A reconstructed block can be generated by merging the residual values ​​generated in the inverse quantization unit 140 and the inverse transform unit 145 with the prediction units predicted by the motion prediction unit, motion compensation unit, and intra-frame prediction unit included in the prediction units 120 and 125.

[0101] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0102] Deblocking filters remove block distortion generated in the reconstructed image due to the boundaries between blocks. To determine whether to perform deblocking, the number of pixels in the columns or rows included in the block can be used as a basis. When applying a deblocking filter to a block, a strong or weak filter can be applied depending on the desired deblocking intensity. Furthermore, when using a deblocking filter, horizontal and vertical filtering can be processed simultaneously.

[0103] The offset correction unit can correct the offset between the deblocked image and the original image on a pixel-by-pixel basis. Offset correction can be performed on a specified image in the following ways: after dividing the pixels included in the image into a predetermined number of regions, determine the regions to be offset and apply the offset to the corresponding regions, or apply the offset by taking into account the edge information of each pixel.

[0104] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered reconstructed image and the original image. After dividing the pixels in the image into predetermined groups, a filter can be determined for each group, and filtering can be performed differently for each group. Information regarding whether to apply adaptive loop filtering can be transmitted via a luminance signal for each coding unit (CU). The shape and filter coefficients of the applied adaptive loop filter can vary depending on the block. Alternatively, the same type (fixed type) of ALF can be applied regardless of the characteristics of the block to which it is applied.

[0105] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the prediction units 120 and 125 when performing inter-frame prediction.

[0106] Figure 2 This is a block diagram of a video decoder according to an embodiment of the present invention.

[0107] Reference Figure 2 The video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0108] When inputting a video bitstream from a video encoder, the input bitstream can be decoded by following the reverse steps of the video encoder.

[0109] The entropy decoding unit 210 can perform entropy decoding in the reverse order of the entropy encoding steps performed in the entropy encoding unit of the video encoder. For example, corresponding to the methods performed in the video encoder, various methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied.

[0110] The entropy decoding unit 210 can decode information related to intra-frame prediction and inter-frame prediction performed by the encoder.

[0111] The rearrangement unit 215 can perform rearrangement based on a method of rearranging the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. Multiple coefficients represented in 1D vector form can be reconstructed into 2D block-shaped coefficients for rearrangement. The rearrangement unit 215 can perform rearrangement by receiving information related to the coefficient scan performed by the encoding unit and performing a reverse scan based on the scan order performed by the corresponding encoding unit.

[0112] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged blocks.

[0113] The inverse transform unit 225 can perform inverse discrete cosine transform and inverse discrete sine transform on the quantization result performed by the video encoder. These inverse discrete cosine transforms and inverse discrete sine transforms are inverse transforms of the transforms performed in the transform unit, i.e., inverse transforms of discrete cosine transform and discrete sine transform. The DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform unit 225 may not perform the inverse transform. The inverse transform can be performed based on the transmission unit determined in the video encoder. The inverse transform unit 225 of the video decoder can selectively perform a transform method (e.g., DCT or DST) based on multiple pieces of information such as the prediction method, the size of the current block, and the prediction direction.

[0114] Prediction units 230 and 235 can generate prediction blocks based on information related to prediction block generation provided by entropy decoding unit 210 and previously decoded block or image information provided by memory 245.

[0115] As described above, when intra-prediction is performed in the same manner as in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra-prediction is performed on the prediction unit based on the pixels to its left, the pixels to its upper left, and the pixels above it. If the size of the prediction unit is different from the size of the transform unit, intra-prediction can be performed using reference pixels based on the transform unit. Alternatively, intra-prediction using an N×N partition can be applied only to the smallest coding unit.

[0116] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit receives various information input from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method. It classifies prediction units according to the current coding unit and determines whether the prediction unit is performing inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 can use information provided by the video encoder required to perform inter-frame prediction for the current prediction unit, and performs inter-frame prediction for the current prediction unit based on information included in at least one of the previous or next images of the current image to which the current prediction unit belongs. Alternatively, inter-frame prediction can also be performed based on information from a portion of the reconstructed region in the current image to which the current prediction unit belongs.

[0117] To perform inter-frame prediction, the motion prediction method of the prediction units included in the corresponding coding unit can be determined based on the coding unit, which of the following modes is used: Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), or Intra-Block Copy Mode.

[0118] The intra-prediction unit 235 can generate prediction blocks based on pixel information within the current image. When the prediction unit is one that has already performed intra-prediction, intra-prediction can be performed based on the intra-prediction mode information of the prediction unit provided by the video encoder. The intra-prediction unit 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra-smoothing filter is the part that performs filtering on the reference pixels of the current block, and whether to apply the filter can be determined based on the prediction mode of the current prediction unit. Adaptive intra-smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided by the video encoder and the adaptive intra-smoothing filter information. If the prediction mode of the current block is a mode that does not perform adaptive intra-smoothing filtering, then the adaptive intra-smoothing filter may not be applied.

[0119] If the prediction mode of the current prediction unit is a prediction unit that performs intra-frame prediction based on pixel values ​​interpolated from reference pixels, then the reference pixel interpolation unit can generate reference pixels with integer or fractional pixel units by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, then interpolation of reference pixels is not required. If the prediction mode of the current block is DC mode, then the DC filter can generate prediction blocks by filtering.

[0120] The reconstructed blocks or images can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0121] The video decoder can receive information from the video encoder regarding whether to apply a deblocking filter to a corresponding block or image, and information regarding whether to apply a strong or weak filter when applying the deblocking filter. The video decoder's deblocking filter can receive information from the video encoder regarding the deblocking filter, and the video decoder can perform deblocking filtering on the corresponding block.

[0122] The offset correction unit can perform offset correction on the reconstructed image based on the type and amount of offset correction applied to the image during encoding.

[0123] The ALF can be applied to the coding unit based on information provided by the encoder, such as whether the ALF is applied and ALF coefficient information. This ALF information can be provided by including it in a specific parameter set.

[0124] The memory 245 stores the reconstructed image or block, such that the image or block can be used as a reference image or reference block, and the reconstructed image can be provided to the output unit.

[0125] Figure 3 This is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0126] The largest coding block can be defined as the coding tree block. An image can be divided into multiple coding tree units (CTUs). The coding tree unit is the largest coding unit and can also be called the largest coding unit (LCU). Figure 3 An example of dividing an image into multiple coding tree units is shown.

[0127] The size of a coding tree unit can be defined at the image level or the sequence level. Therefore, information indicating the size of the coding tree unit can be sent via signals using either an image parameter set or a sequence parameter set.

[0128] For example, the size of the coding tree unit for the entire image within the sequence can be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image can be set to 128×128, and the size of the coding tree unit in the second image can be set to 256×256.

[0129] Coded blocks can be generated by dividing the coding tree into units. A coded block represents the basic unit used for encoding / decoding processing. For example, prediction or transformation can be performed on different coded blocks, or prediction coding modes can be determined on different coded blocks. The prediction coding mode represents the method for generating the predicted image. For example, prediction coding modes can include intra-prediction, inter-prediction, current picture referencing (CPR, or intra-block copy (IBC)), or combined prediction. For a coded block, at least one of the prediction coding modes—intra-prediction, inter-prediction, current picture referencing, or combined prediction—can be used to generate the prediction block associated with that coded block.

[0130] Information representing the predictive coding mode of the current block can be transmitted via a bitstream signal. For example, this information could be a 1-bit flag indicating whether the predictive coding mode is intra-frame or inter-frame. Current image reference or combined prediction can be used only if the predictive coding mode of the current block is determined to be inter-frame.

[0131] The current image reference is used to set the current image as the reference image and obtain the prediction block of the current block from the encoded / decoded regions within the current image. Here, the current image means the image that includes the current block. Information indicating whether the current image reference is applied to the current block can be sent via a bitstream signal. For example, this information could be a 1-bit flag. When the flag is true, the prediction coding mode of the current block can be determined as the current image reference; when the flag is false, the prediction mode of the current block can be determined as inter-frame prediction.

[0132] Alternatively, the predictive coding mode for the current block can be determined based on a reference image index. For example, when the reference image index points to the current image, the predictive coding mode for the current block can be determined as current image reference. When the reference image index points to another image instead of the current image, the predictive coding mode for the current block can be determined as inter-frame prediction. That is, current image reference is a prediction method that uses information from encoded / decoded regions in the current image, and inter-frame prediction is a prediction method that uses information from other encoded / decoded images.

[0133] Combinatorial prediction represents a coding mode composed of two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, when applying combinatorial prediction, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or the current image reference, and a second prediction block can be generated based on another. When generating the first and second prediction blocks, the final prediction block can be generated by averaging or weighted summing the first and second prediction blocks. Information indicating whether combinatorial prediction is applied can be transmitted via a bitstream signal. This information can be a 1-bit flag.

[0134] Figure 4 This is a diagram illustrating the various partitioning types of coded blocks.

[0135] A coded block can be divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. Furthermore, the divided coded blocks can be further divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0136] Quadtree partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block can be divided into four square partitions (see reference). Figure 4 ("SPLIT_QT(partition_quadtree)") in part (a).

[0137] Binary tree partitioning refers to a partitioning technique that divides the current block into two blocks. The process of partitioning the current block along a vertical direction (i.e., using a vertical line that crosses the current block) is called vertical binary tree partitioning, and the process of partitioning the current block along a horizontal direction (i.e., using a horizontal line that crosses the current block) is called horizontal binary tree partitioning. As a result of binary tree partitioning, the current block can be divided into two non-square partitions. Figure 4 In part (b), "SPLIT_BT_VER(Partition_Binary_Vertical)" represents the result of a vertical binary tree partition, and Figure 4 In part (c), “SPLIT_BT_HOR(Partition_Binary_Level)” represents the result of a horizontal binary tree partition.

[0138] Ternary tree partitioning refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along a vertical direction (i.e., using two vertical lines crossing the current block) is called vertical ternary tree partitioning, and the process of dividing the current block into three blocks along a horizontal direction (i.e., using two horizontal lines crossing the current block) is called horizontal ternary tree partitioning. As a result of ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. Figure 4 In part (d), "SPLIT_TT_VER(Partition_Ternary_Vertical)" represents the result of a ternary tree partition in the vertical direction, and Figure 4 In part (e), “SPLIT_TT_HOR(Partition_Ternary_Horizontal)” represents the result of horizontal ternary tree partitioning.

[0139] The number of partitions in a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or image level. Therefore, the maximum partitioning depth of a coding tree unit can vary depending on different sequences or images.

[0140] Alternatively, the maximum partition depth can be determined individually for each of the multiple partitioning techniques. For example, the maximum partition depth allowed for quadtree partitioning can be different from the maximum partition depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0141] The encoder can transmit information via signals from the bitstream indicating at least one of the partition type or partition depth of the current block. The decoder can determine the partition type and partition depth of the coded tree unit based on the information parsed from the bitstream.

[0142] Figure 5 This is a diagram illustrating an example of how coding tree units are divided.

[0143] The process of dividing coding blocks using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning is called multitree partitioning.

[0144] A coded block generated by applying a multi-way tree partitioning to a coded block can be called a downstream coded block. When the partitioning depth of a coded block is k, the partitioning depth of multiple downstream coded blocks is set to k+1.

[0145] On the other hand, for a coding block with a partitioning depth of k+1, the coding block with a partitioning depth of k can be called the upstream coding block.

[0146] The partition type of the current coding block can be determined based on at least one of the partition types of the upstream coding block or the partition types of adjacent coding blocks. Adjacent coding blocks are adjacent to the current coding block and can include at least one of the current coding block's upper adjacent block, left adjacent block, or upper-left adjacent block. The partition type can include at least one of whether to partition into a quadtree, whether to partition into a binary tree, the binary tree partition direction, whether to partition into a ternary tree, or the ternary tree partition direction.

[0147] To determine the type of code block partitioning, information indicating whether the code block has been partitioned can be sent via a bitstream using a signal. This information is a 1-bit flag "split_cu_flag," where a true flag indicates that the code block was partitioned using a multi-way tree partitioning technique.

[0148] When "split_cu_flag" is true, information indicating whether the coded block has been partitioned by a quadtree can be sent via a bitstream signal. This information is a 1-bit flag "split_qt_flag". When this flag is true, the coded block can be divided into 4 blocks.

[0149] For example, in Figure 5 The example shown illustrates the generation of four coded blocks with a partition depth of 1 when the coding tree unit is partitioned by a quadtree. Furthermore, the example illustrates applying quadtree partitioning again to the first and fourth coded blocks, which are the result of the quadtree partitioning. Ultimately, this generates four coded blocks with a partition depth of 2.

[0150] Alternatively, a code block with a partition depth of 3 can be generated by applying a quadtree partition to the code block with a partition depth of 2 again.

[0151] When a quadtree partition is not applied to the coded block, it can be determined whether to perform a binary tree partition or a ternary tree partition by considering at least one of the following: the size of the coded block, whether the coded block is located at an image boundary, the maximum partition depth, or the partition type of adjacent blocks. When it is determined whether to perform a binary tree partition or a ternary tree partition, information indicating the partition direction can be transmitted via a bitstream signal. This information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partition direction (vertical or horizontal) can be determined based on this flag. Alternatively, information indicating whether a binary tree partition or a ternary tree partition is applied to the coded block can be transmitted via a bitstream signal. This information can be a 1-bit flag "mtt_split_cu_binary_flag". The binary tree partition or ternary tree partition can be determined based on this flag.

[0152] For example, in Figure 5The example shown illustrates the application of a vertical binary tree partitioning to a coded block with a partitioning depth of 1, the application of a vertical ternary tree partitioning to the left coded block in the resulting coded block, and the application of a vertical binary tree partitioning to the right coded block.

[0153] Inter-frame prediction refers to using information from the previous image to predict the predictive coding mode of the current block. For example, a block in the previous image that is at the same position as the current block (hereinafter referred to as a collocated block) can be set as the prediction block for the current block. Hereinafter, the prediction block generated based on the block at the same position as the current block will be called a collocated prediction block.

[0154] On the other hand, if an object present in the previous image has moved to a different location in the current image, the object's motion can be used to effectively predict the current block. For example, if the direction and size of the object's movement can be known by comparing the previous and current images, the object's motion information can be considered to generate a predicted block (or predicted image) for the current block. Hereinafter, the predicted block generated using motion information can be referred to as a motion prediction block.

[0155] Residual blocks can be generated by subtracting prediction blocks from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using motion prediction blocks instead of co-position prediction blocks, thereby improving the compression performance of the residual block.

[0156] As mentioned above, the process of generating prediction blocks using motion information can be called motion-compensated prediction. In most inter-frame predictions, prediction blocks can be generated based on motion-compensated prediction.

[0157] Motion information may include at least one of motion vectors, reference image indices, prediction directions, or bidirectional weighted indexes. Motion vectors represent the direction and size of an object's movement. Reference image indices specify the reference image for the current block from among multiple reference images included in the reference image list. Prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 and L1 prediction). Motion information in either the L0 or L1 direction can be used based on the prediction direction of the current block. Bidirectional weighted indexes specify the weights applied to the L0 prediction block and the weights applied to the L1 prediction block.

[0158] Figure 6 This is a flowchart of the inter-frame prediction method according to an embodiment of the present invention.

[0159] refer to Figure 6The inter-frame prediction method includes the following steps: determining the inter-frame prediction mode of the current block (S601); obtaining motion information of the current block according to the determined inter-frame prediction mode (S602); and performing motion compensation prediction on the current block based on the obtained motion information (S603).

[0160] Inter-frame prediction modes represent various techniques used to determine the motion information of the current block, and may include inter-frame prediction modes using translational motion information and inter-frame prediction modes using affine motion information. For example, inter-frame prediction modes using translational motion information may include merging mode and advanced motion vector prediction mode, while inter-frame prediction modes using affine motion information may include affine merging mode and affine motion vector prediction mode. Depending on the inter-frame prediction mode, the motion information of the current block can be determined based on neighboring blocks adjacent to the current block or information parsed from the bitstream.

[0161] The following section details the inter-frame prediction method using affine motion information.

[0162] Figure 7 It is a diagram illustrating the nonlinear motion of the object.

[0163] The motion of objects within a video may be non-linear. For example, such as... Figure 7 The example shown may involve non-linear motion of the object, such as camera zoom-in, zoom-out, rotation, or affine transformation. When non-linear motion occurs, it is impossible to effectively represent the object's motion using translational motion vectors. Therefore, in parts where non-linear motion occurs, affine motion can be used instead of translational motion, thereby improving coding efficiency.

[0164] Figure 8 This is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0165] Whether to apply an affine motion-based inter-frame prediction technique to the current block can be determined based on information parsed from the bitstream. Specifically, whether to apply an affine motion-based inter-frame prediction technique to the current block can be determined based on at least one of a flag indicating whether to apply an affine merging mode or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0166] When an inter-frame prediction technique based on affine motion is applied to the current block, the affine motion model of the current block can be determined (S801). The affine motion model can be determined to be at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent the affine motion, and the 4-parameter affine motion model uses 4 parameters to represent the affine motion.

[0167] Equation 1 represents the case of affine motion using six parameters. Affine motion represents translational motion within a predetermined region defined by an affine seed vector.

[0168] Equation 1

[0169] v x =ax-by+e

[0170] v y =cx+dy+f

[0171] While using six parameters to represent affine motion allows for the representation of complex motions, the increased number of bits required to encode each parameter reduces encoding efficiency. Therefore, affine motion can also be represented using four parameters. Equation 2 illustrates the case of representing affine motion using four parameters.

[0172] Equation 2

[0173] v x =ax-by+e

[0174] v y =bx+ay+f

[0175] Information used to determine the affine motion model for the current block can be encoded and transmitted via a bitstream signal. For example, this information could be a 1-bit flag, "affine_type_flag". A value of 0 indicates the application of a 4-parameter affine motion model, and a value of 1 indicates the application of a 6-parameter affine motion model. The flag can be encoded at the slice, tile, or block level (e.g., coded block or coded tree unit). When the flag is transmitted at the slice level, the affine motion model determined at that slice level can be applied to all blocks belonging to that slice.

[0176] Alternatively, the affine motion model of the current block can be determined based on the affine inter-frame prediction mode of the current block. For example, when applying the affine merging mode, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when applying the affine motion vector prediction mode, the information used to determine the affine motion model of the current block can be encoded and transmitted as a signal via a bitstream. For example, when applying the affine motion vector prediction mode to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0177] Next, the affine seed vector of the current block can be exported (S802). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be exported. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block can be exported. The motion vectors at the control points can be called affine seed vectors. Control points can include at least one of the upper left, upper right, or lower left corners of the current block.

[0178] Figure 9 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0179] In a 4-parameter affine motion model, two related affine seed vectors can be derived from the top left, top right, or bottom left corners. For example, ... Figure 9 As shown in the example in section (a), when the 4-parameter affine motion model is selected, the affine vectors can be derived by using the affine seed vector sv0 associated with the top-left corner of the current block (e.g., the top-left sample (x0, y0)) and the affine seed vector sv1 associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)). Alternatively, the affine seed vector associated with the bottom-left corner can be used instead of the affine seed vector associated with the top-left corner, or vice versa.

[0180] In a 6-parameter affine motion model, affine seed vectors related to the top-left, top-right, and bottom-left corners can be derived. For example, as... Figure 9 As shown in section (b), when the 6-parameter affine motion model is selected, the affine vectors can be derived by using the affine seed vector sv0 associated with the top-left corner of the current block (e.g., the top-left sample (x0, y0)), the affine seed vector sv1 associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)), and the affine seed vector sv2 associated with the top-left corner of the current block (e.g., the top-left sample (x2, y2)).

[0181] In the embodiments described later, under the 4-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments using the first and second affine seed vectors described later, at least one of the first and second affine seed vectors can be replaced by the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0182] Furthermore, in the 6-parameter affine motion model, the affine seed vectors of the upper left control point, upper right control point, and lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiments using the first, second, and third affine seed vectors described later, at least one of the first, second, and third affine seed vectors can be replaced by the affine seed vector of the lower right control point (the fourth affine seed vector).

[0183] An affine seed vector can be used to derive affine vectors for different sub-blocks (S803). Here, the affine vector represents the translational motion vector derived based on the affine seed vector. The affine vector of a sub-block can be called the affine sub-block motion vector or the sub-block motion vector.

[0184] Figure 10 This is a diagram showing an example of the affine vectors of a sub-block under a 4-parameter motion model.

[0185] The affine vector of a sub-block can be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine sub-block vector.

[0186] Equation 3

[0187]

[0188]

[0189] In Equation 3, (x, y) represents the position of the sub-block. The position of the sub-block refers to the position of the reference sample included within it. The reference sample can be the sample located at the top left corner of the sub-block, or a sample whose x-axis or y-axis coordinate is at its center. (x0, y0) represents the position of the first control point, and (sv 0x sv 0y Let (x1, y1) represent the first affine seed vector. Additionally, (x1, y1) represents the position of the second control point, and (sv... 1x sv 1y ) represents the second affine seed vector.

[0190] When the first control point and the second control point correspond to the top left and top right corners of the current block, respectively, x1-x0 can be set to the same value as the width of the current block.

[0191] Then, motion compensation prediction can be performed on each sub-block using the affine vectors of each sub-block (S804). After performing motion compensation prediction, prediction blocks associated with each sub-block can be generated. The prediction blocks of the sub-blocks can be set as the prediction blocks of the current block.

[0192] The affine seed vector of the current block can be derived based on the affine seed vectors of its neighboring blocks. When the inter-frame prediction mode for the current block is affine merging mode, the affine seed vectors of the merge candidates included in the merge candidate list can be determined as the affine seed vector of the current block. Furthermore, when the inter-frame prediction mode for the current block is affine merging mode, motion information containing at least one of the following—the reference image index, the specific direction prediction flag, or the bidirectional weighting value for the current block—can be set to be the same as that of the merge candidates.

[0193] Merging candidates can be derived based on neighboring blocks of the current block. The neighboring blocks may include at least one of spatially adjacent blocks that are spatially adjacent to the current block and temporally adjacent blocks included in images different from the current image.

[0194] Figure 11 This is a diagram showing adjacent blocks that can be used to derive merge candidates.

[0195] The adjacent blocks of the current block can include at least one of the following: the adjacent block to the left of the current block (A), the adjacent block above the current block (B), the adjacent block to the upper right corner of the current block (C), the adjacent block to the lower left corner of the current block (D), or the adjacent block to the upper left corner of the current block. If the coordinates of the upper left sample of the current block are (x0, y0), then the left adjacent block A includes the sample at the position (x0-1, y0+H-1), and the upper adjacent block B includes the sample at the position (x0+W-1, y0-1). Here, W and H represent the width and height of the current block, respectively. The upper right adjacent block C includes the sample at the position (x0+W, y0-1), and the lower left adjacent block D includes the sample at the position (x0-1, y0+H). The upper left adjacent block E includes the sample at the position (x0-1, y0-1).

[0196] When neighboring blocks are encoded in affine inter-frame prediction mode, the affine seed vectors of merge candidates can be derived based on the affine seed vectors of the corresponding neighboring blocks. Hereinafter, neighboring blocks encoded in affine inter-frame prediction mode are referred to as affine neighboring blocks, and the merge candidates derived from affine neighboring blocks are referred to as affine merge candidates.

[0197] Affine merging candidates associated with the current block can be generated by searching neighboring blocks in a predefined scan order. The scan order can be predefined in the encoder and decoder. For example, neighboring blocks can be searched in the order A, B, C, D, E. Alternatively, merging candidates can be derived sequentially from the searched affine neighboring blocks. Alternatively, the scan order can be adaptively determined based on at least one of the current block's size, shape, or affine motion model. That is, blocks with different sizes, shapes, or affine motion models will have different scan orders.

[0198] Alternatively, multiple blocks above the current block are searched sequentially, and an affine merge candidate is derived from the first found affine neighboring block. Then, multiple blocks to the left of the current block are searched sequentially, and an affine merge candidate is derived from the first found affine neighboring block. The multiple neighboring blocks above the current block include at least one of neighboring block E, neighboring block B, or neighboring block C, and the multiple blocks to the left of the current block include at least one of block A or block D. In this case, neighboring block E can also be classified as a block to the left of the current block.

[0199] Although not illustrated, affine merging candidates can also be derived from temporally adjacent blocks of the current block. Temporally adjacent blocks can include blocks located at the same position as the current block in the co-located image or blocks adjacent to it. Specifically, if the temporally adjacent blocks of the current block are encoded using an affine inter-frame prediction mode, affine merging candidates can be derived based on the affine seed vector of the temporal affine merging candidates.

[0200] A merge candidate list including affine merge candidates can be generated, and the affine seed vector of one of the merge candidates included in the merge candidate list can be determined as the affine seed vector of the current block. To this end, index information identifying any one of the multiple merge candidates can be encoded and transmitted via a bitstream.

[0201] As another example, multiple neighboring blocks can be searched in scan order, and the affine seed vector of the current block can be derived from the affine seed vector of the first found affine neighboring block.

[0202] As mentioned above, in affine merge mode, the affine seed vector of the current block can be derived using the affine seed vector of the adjacent blocks.

[0203] When the inter-frame prediction mode for the current block is affine motion vector prediction mode, the affine seed vectors of the motion vector prediction candidates included in the motion vector prediction candidate list can be determined as the affine seed vector prediction value for the current block. The affine seed vector for the current block can be derived by adding the affine seed vector difference to the predicted affine seed vector value.

[0204] Affine seed vector prediction candidates can be derived based on the neighboring blocks of the current block. Specifically, multiple neighboring blocks above the current block are searched in a predetermined scan order, and a first affine seed vector prediction candidate can be derived from the first found affine neighboring block. Additionally, multiple neighboring blocks to the left of the current block are searched in a predetermined scan order, and a second affine seed vector prediction candidate can be derived from the first found affine neighboring block.

[0205] Information used to determine the affine seed vector difference can be encoded and transmitted via a bitstream. This information may include magnitude information representing the size of the affine seed vector difference and a sign signal representing the sign of the affine seed vector difference. The affine seed vector difference associated with each control point can be set to be the same. Alternatively, the affine seed vector difference can be set differently for each control point.

[0206] As described above, the affine seed vectors of affine merge candidates or affine seed vector prediction candidates can be derived from the affine seed vectors of affine neighboring blocks, and the affine seed vectors of the current block can be derived using the derived affine merge candidates or affine seed vector prediction candidates. Alternatively, after searching multiple affine neighboring blocks in a predetermined scan order, the affine seed vector of the current block can be derived from the affine seed vectors of the first found affine neighboring blocks.

[0207] The following details a method for deriving the affine seed vector, affine merge candidate, or affine seed vector prediction candidate of the current block from the affine seed vectors of neighboring blocks. In embodiments described later, deriving the affine seed vector of the current block can be understood as deriving the affine seed vector of the affine merge candidate, or it can also be understood as deriving the affine seed vector of the affine seed vector prediction candidate.

[0208] Figure 12 This is a diagram illustrating how the affine seed vector of the current block is derived based on the affine seed vectors of neighboring affine blocks.

[0209] If the affine neighboring block stores a first affine seed vector nv0 associated with the top-left control point and a second affine seed vector nv1 associated with the top-right control point, then a third affine seed vector nv2 associated with the bottom-left control point of the affine neighboring block can be derived based on the first and second affine seed vectors. Equation 4 shows an example of deriving the third affine seed vector.

[0210] Equation 4

[0211]

[0212]

[0213] In equation 4, (nv0x , nv 0y ) represents the first affine seed vector nv0, (nv 1x , nv 1y ) represents the second affine seed vector nv1, and (nv 2x , nv 2y () represents the third affine seed vector nv2. Additionally, (x n0 x n0 (x) represents the position of the first control point. n1 x n1 ) represents the location of the second control point, and (x n2 x n2 ) indicates the location of the third control point.

[0214] Then, the affine seed vector of the current block can be derived using the first affine seed vector, the second affine seed vector, and the third affine seed vector. Equation 5 shows an example of deriving the first affine seed vector v0 of the current block, and Equation 6 shows an example of deriving the second affine seed vector v1 of the current block.

[0215] Equation 5

[0216]

[0217]

[0218] Equation 6

[0219]

[0220]

[0221] In equations 5 and 6, (v 0x v 0y ) represents the first affine seed vector sv0 of the current block, and (v 1x v 1y Let (x0, y0) represent the second affine seed vector sv1 of the current block. Additionally, (x0, y0) represents the position of the first control point, and (x1, y1) represents the position of the second control point. For example, the first control point might represent the top-left corner of the current block, and the second control point might represent the top-right corner.

[0222] The example illustrates using three affine seed vectors associated with affine neighboring blocks to derive multiple affine seed vectors for the current block. As another example, it is also possible to derive the affine seed vector for the current block using only two of the multiple affine seed vectors from the affine neighboring blocks.

[0223] Alternatively, multiple affine seed vectors for the current block can be derived using the fourth affine seed vector associated with the bottom right corner, instead of using the first affine seed vector associated with the top left corner, the second affine seed vector associated with the top right corner, or the third affine seed vector associated with the bottom left corner of the affine neighboring block.

[0224] In particular, when the upper boundary of the current block touches the upper boundary of the coding tree unit and the affine seed vector of the upper control point (e.g., upper left or upper right corner) of the affine neighbor block (hereinafter referred to as the upper affine neighbor block) is to be used, these need to be pre-stored in memory, which may cause an increase in the number of line buffers. Therefore, when the upper boundary of the current block touches the upper boundary of the coding tree unit, it can be configured to use the affine seed vector of the lower control point (e.g., lower left or lower right corner) for the upper affine neighbor block instead of using the affine seed vector of the upper control point. For example, multiple affine seed vectors of the current block can be derived using the third affine seed vector associated with the lower left corner and the fourth affine seed vector associated with the lower right corner of the upper affine neighbor block. In this case, the affine seed vector associated with the lower corner can be derived by copying the affine seed vector associated with the upper corner, or it can be derived from multiple affine seed vectors associated with the upper corner. For example, the first, second, or third affine seed vector can be transformed / replaced with a fourth affine seed vector associated with the lower right corner.

[0225] Equations 7 and 8 represent examples of deriving the first and second affine seed vectors of the current block using the third affine seed vector associated with the lower left control point of the adjacent affine vector and the fourth affine seed vector associated with the lower right control point.

[0226] Equation 7

[0227]

[0228]

[0229] Equation 8

[0230]

[0231]

[0232] In equations 7 and 8, (x n2 y n2 ) represents the coordinates of the lower left control point of the affine neighboring block, and (x n3 y n3(x0, y0) represents the coordinates of the bottom-right control point of the affine neighboring block, and (x1, y1) represents the coordinates of the top-left control point of the current block. (nv) 2x , nv 2y ) represents the affine seed vector (i.e., the third affine seed vector) that affines the lower left control point of the adjacent block, and (nv 3x , nv 3y ) represents the affine seed vector (i.e., the fourth affine seed vector) that maps the lower right control point of the adjacent block. (v 0x v 0y ) represents the affine seed vector of the top-left control point of the current block (i.e., the first affine seed vector), and (v 1x v 1y ) represents the affine seed vector (i.e., the second affine seed vector) of the upper right control point of the current block.

[0233] The division operations included in Equations 7 and 8 can also be replaced with shift operations. Shift operations can be based on the width between the lower left and lower right control points (i.e., (x...)). n3 -x n2 The exported value is used for execution.

[0234] In the example above, multiple affine seed vectors of the current block can be derived from multiple affine seed vectors of the encoded / decoded affine neighboring blocks. This requires storing these multiple affine seed vectors in memory. However, storing multiple affine seed vectors of the affine neighboring blocks in memory, in addition to storing multiple translational motion vectors (i.e., multiple affine vectors) of the multiple sub-blocks included in the affine neighboring blocks, increases memory usage. To eliminate this problem, the affine seed vectors of the current block can be derived using the motion vectors of the sub-blocks adjacent to the control points of the affine neighboring blocks, thus replacing the affine seed vectors of the affine neighboring blocks. That is, the motion vectors of the sub-blocks adjacent to the control points of the affine neighboring blocks can be set as the affine seed vectors of the affine neighboring blocks. Here, a sub-block is a block with a predefined size / shape in the encoder and decoder, and can be a block with a basic size / shape for storing motion vectors. For example, a sub-block can be a 4×4 square block. Alternatively, the motion vector at a specified sample position can be set as the affine seed vector of the affine neighboring blocks.

[0235] Figure 13 This is a diagram illustrating an example of setting the motion vector of a sub-block to an affine seed vector that affines the adjacent block.

[0236] The motion vector of a sub-block adjacent to a control point can be set as the affine seed vector of the corresponding control point. For example, in Figure 13In the example shown, the motion vector (nv) of the child block adjacent to the bottom left corner of the affine neighboring block (bottom left child block) can be... 4x , nv 4y ) set as the affine seed vector (nv) of the lower left control point 2x , nv 2y And it can also transfer the motion vector (nv) of the sub-block adjacent to the bottom right corner (bottom right sub-block). 5x , nv 5y Set the affine seed vector (nv) to the control point at the bottom right corner. 3x , nv 3y The lower left sub-block refers to the block that includes the lower left control point (x) of the adjacent affine block. n2 y n2 Adjacent samples (e.g., (x) n2 y n2 -1) The sub-block of the sample at position), and the lower right sub-block refers to the sub-block that includes the lower right control point (x) of the adjacent affine block. n3 y n3 Adjacent samples (e.g., (x) n3 -1, y n3 -1)) block of sample). When deriving the affine seed vector of the current block based on Equations 7 and 8, the third affine seed vector of the affine neighboring block can be replaced by the motion vector of the lower left sub-block, and the fourth affine seed vector can be replaced by the motion vector of the lower right sub-block.

[0237] In the embodiments described later, the sub-blocks used as affine seed vectors for affine neighboring blocks will be referred to as affine sub-blocks.

[0238] According to one embodiment of the present invention, affine sub-blocks can be determined based on samples at specific locations. For example, sub-blocks including samples at specific locations can also be set as affine sub-blocks. Hereinafter, the samples at specific locations are referred to as affine reference samples. In addition, the reference sample used to determine the lower left control point of the affine sub-block is referred to as the lower left reference sample, and the reference sample used to determine the lower right control point of the affine sub-block is referred to as the lower right reference sample.

[0239] The lower-left and lower-right reference samples can be selected from multiple samples included in the affine neighboring blocks. For example, at least one of the upper-left, lower-left, upper-right, or lower-left samples of the lower-left sub-block can be set as the lower-left reference sample, and at least one of the upper-left, lower-left, upper-right, or lower-left samples of the lower-right sub-block can be set as the lower-right reference sample. Therefore, the motion vectors of the lower-left sub-block including the lower-left reference sample and the lower-right sub-block including the lower-right reference sample can be set as the affine seed vector associated with the lower-left control point and the affine seed vector associated with the lower-right control point, respectively.

[0240] As another example, at least one of the lower-left or lower-right reference samples can be set to a sample located outside the affine neighboring block. For this, the reference... Figures 14 to 16 Please provide a detailed explanation.

[0241] Figures 14 to 16 This is a diagram showing the location of the reference sample.

[0242] like Figure 14 In the example shown in part (a), for the lower left control point, the upper left sample of the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it is possible to include the reference sample (x). n4 y n4 The lower left sub-block of ) is set as an affine sub-block associated with the lower left control point.

[0243] For the bottom right control point, the sample to the right of the top right sample in the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it can be combined with the reference sample (x). n5 y n5 The rightmost adjacent sub-block of the bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0244] Alternatively, such as Figure 14 In the example shown in part (b), for the lower left control point, the sample to the left of the upper left sample in the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it can be combined with the reference sample (x). n4 y n4 The left-adjacent sub-block of the lower left sub-block is set as an affine sub-block associated with the lower left control point.

[0245] For the bottom right control point, the top right sample of the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it is possible to include the reference sample (x). n5 y n5 The bottom right sub-block of ) is set as an affine sub-block associated with the bottom right control point.

[0246] Alternatively, such as Figure 15 In the example shown in part (a), for the lower left control point, the lower left sample of the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it is possible to include the reference sample (x). n4 y n4 The lower left sub-block of ) is set as an affine sub-block associated with the lower left control point.

[0247] For the bottom right control point, the sample to the right of the bottom right sample in the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it can be combined with the reference sample (x). n5 y n5 The rightmost adjacent sub-block of the bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0248] Alternatively, such as Figure 15 In the example shown in part (b), for the lower left control point, the sample to the left of the lower left sample in the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it can be combined with the reference sample (x). n4 y n4 The left-adjacent sub-block of the lower left sub-block is set as an affine sub-block associated with the lower left control point.

[0249] For the bottom right control point, the bottom right sample of the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it is possible to include the reference sample (x). n5 y n5 The bottom right sub-block of ) is set as an affine sub-block associated with the bottom right control point.

[0250] Alternatively, such as Figure 16 In the example shown in part (a), for the lower left control point, the sample between the upper left and lower left samples located in the lower left sub-block (e.g., the middle left sample) can be set as the reference sample (x). n4 y n4 Therefore, it is possible to include the reference sample (x). n4 y n4 The lower left sub-block of ) is set as an affine sub-block associated with the lower left control point.

[0251] For the bottom right control point, the sample to the right of the sample located between the top right and bottom right samples in the bottom right sub-block (e.g., the middle sample on the right) can be set as the reference sample (x). n5 y n5 Therefore, it can be combined with the reference sample (x). n5 y n5 The rightmost adjacent sub-block of the bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0252] Alternatively, such as Figure 16In the example shown in part (b), for the lower left control point, the sample to the left of the sample located between the upper left and lower left samples in the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it can be combined with the reference sample (x). n4 y n4 The left-adjacent sub-block of the lower left sub-block is set as an affine sub-block associated with the lower left control point.

[0253] For the bottom right control point, the sample between the top right sample and the bottom right sample located in the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it is possible to include the reference sample (x). n5 y n5 The bottom right sub-block of ) is set as an affine sub-block associated with the bottom right control point.

[0254] When deriving multiple affine seed vectors for the current block based on Equations 7 and 8, the third affine seed vector of the affine neighboring block can be replaced by the motion vector of the affine sub-block associated with the lower left control point, and the fourth affine seed vector can be replaced by the motion vector of the affine sub-block associated with the lower right control point. Furthermore, the position of the lower left control point can be replaced by the position of the lower left reference sample, and the position of the lower right control point can be replaced by the position of the lower right reference sample.

[0255] and Figures 14 to 16 The content differs, and sub-blocks including samples adjacent to the reference sample can also be set as affine sub-blocks. Specifically, samples located outside of affine adjacent sub-blocks can be set as reference samples, and sub-blocks included in affine adjacent blocks can be set as affine sub-blocks. For example, in Figure 14 In the example shown in part (a), the sample to the right of the upper-right sample located in the lower-right sub-block can be set as the reference sample (x). n5 y n5 ), and the bottom-right sub-block can be set as an affine sub-block associated with the bottom-right corner. Alternatively, in Figure 14 In the example shown in part (b), the sample to the left of the top-left sample in the bottom-left sub-block can be set as the reference sample (x). n4 y n4 Furthermore, the bottom left sub-block can be set as an affine sub-block associated with the bottom left corner.

[0256] Can be applied equivalently Figure 15 and Figure 16 The embodiments described herein. That is, in Figure 15 (a) part or Figure 16In the example shown in part (a), the bottom right sample of the bottom right sub-block or the sample to the right of the middle sample on the right can be set as the reference sample (x). n5 y n5 ), and the bottom-right sub-block can be set as an affine sub-block associated with the bottom-right corner. Alternatively, in Figure 15 (b) part or Figure 16 In the example shown in part (b), the bottom left sample of the bottom left sub-block or the sample to the left of the middle left sample can be set as the reference sample (x). n4 y n4 Furthermore, the bottom left sub-block can be set as an affine sub-block associated with the bottom left corner.

[0257] In the example above, the affine seed vector of the affine neighboring block can be derived using the motion vector of the affine sub-block. Therefore, for an encoded / decoded block, the motion vector can be stored on a sub-block basis.

[0258] As another example, after storing a minimum number of affine seed vectors in the affine neighboring blocks, the motion vectors of the affine sub-blocks can be derived using the stored multiple affine seed vectors.

[0259] Equations 9 and 10 represent examples of deriving the motion vector of an affine sub-block using the affine seed vector of an affine neighboring block.

[0260] Equation 9

[0261]

[0262]

[0263] Equation 10

[0264]

[0265]

[0266] In equations 9 and 10, (nv 4x , nv 4y ) represents the motion vector of the affine sub-block associated with the lower left control point, and (nv 5x , nv 5y (nv) represents the motion vector of the affine sub-block associated with the lower right control point. Since the motion vector of the affine sub-block is set to the same as the affine seed vector of the control point, (nv) 4x , nv 4y It can be derived from the affine seed vector (nv) associated with the lower left control point. 2x , nv 2y ) instead, or (nv 5x , nv5y It can be derived from the affine seed vector (nv) associated with the lower right control point. 3x , nv 3y )replace.

[0267] (x n4 y n4 () indicates the position of the reference sample in the bottom-left sub-block. Alternatively, the center position of the bottom-left sub-block or the position of the bottom-left control point can be used instead. (x n5 y n5 This indicates the position of the reference sample in the bottom right sub-block. Alternatively, the center position of the bottom right sub-block or the position of the bottom right control point can be used instead.

[0268] Equations 9 and 10 can be applied when the current block does not contact the boundary of the coding tree unit. When the current block contacts the upper boundary of the coding tree unit, instead of using Equations 9 and 10, the translational motion vector of the affine sub-block determined based on the lower left reference sample can be set as the third affine seed vector, and the translational motion vector of the affine sub-block determined based on the lower right reference sample can be set as the fourth affine seed vector.

[0269] In equations 7 and 8, (x n3 -x n2 The 'x' represents the width between the bottom left and bottom right control points. As mentioned above, x n3 The position x of the reference sample in the lower right corner can be used as a reference. n5 Instead, and x n2 The position x of the lower left reference sample can be used as a reference. n4 Instead. Below, (x) n3 -x n2 ) or the value obtained by replacing the position of the aforementioned equation with the position of the reference sample (e.g., (x) n5 -x n4 Defined as variable W seed The variable mentioned is called the sub-seed vector width.

[0270] Depending on the location of the reference sample, the width of the sub-seed vector may not be a power of 2 (e.g., 2^32). nIn cases where the bottom left sample of the bottom left sub-block is set as the bottom left reference sample and the bottom right sample of the bottom right sub-block is set as the bottom right reference sample, the width of the sub-seed vector is not a multiple of 2. As mentioned above, when the width of the sub-seed vector is not a power of 2, it can be transformed into a power of 2. This transformation can include adding / subtracting an offset to the sub-seed vector width, or using the position of a sample adjacent to the reference sample instead of the reference sample's position. For example, the transformed sub-seed vector width can be derived by adding 1 to the width between the bottom left and bottom right reference samples. Alternatively, the width between the adjacent reference sample to the right of the bottom right reference sample and the bottom left reference sample can be set as the transformed sub-seed vector width. Then, the affine seed vector of the current block can be derived by substituting the transformed sub-seed vector width into Equations 7 and 8.

[0271] The division operations included in Equations 7 and 8 can also be replaced with shift operations. Shift operations can be performed based on values ​​derived from the transformed sub-seed vector width (i.e., values ​​represented as powers of 2).

[0272] When the reference sample used to determine the affine sub-block does not belong to the affine neighboring block, the affine seed vector of the affine neighboring block can be derived based on the samples adjacent to the reference sample among the multiple samples included in the affine neighboring block. Specifically, the translational motion vector of the sub-block that includes samples adjacent to the reference sample in the affine neighboring block (hereinafter referred to as adjacent reference samples) can be set as the affine seed vector of the affine neighboring block. As described above, the method of deriving the affine seed vector using adjacent reference samples can be defined as a modified affine merge vector derivation method.

[0273] Figure 17 This is a diagram illustrating an example of applying a modified affine merging vector derivation method.

[0274] When affine the lower right reference sample (x) of adjacent block E n5 y n5 When a sample is not part of an affine neighbor block, it can be determined based on the sample (x) that is to the left of the lower-right reference sample within the samples included in the affine neighbor block. n5 -1, y n5 This is used to derive the affine seed vector. Specifically, it can include neighboring reference samples (x... n5 -1, y n5 The translational motion vector of the sub-block is set as the affine seed vector of the lower right control point.

[0275] exist Figure 17In the example shown, the sample adjacent to the right side of the upper right sample of the lower right sub-block is shown as the lower right reference sample. When the sample adjacent to the right side of the lower right sample of the lower right sub-block or the sample adjacent to the right side of the middle right sample of the lower right sub-block is set as the lower right reference sample, the affine seed vector can be derived based on the sample adjacent to the left side of the adjacent reference sample.

[0276] Additionally, when the lower left reference sample does not belong to the affine neighboring block, the affine seed vector can also be derived based on the sample adjacent to the right side of the lower left reference sample, as described in the embodiment.

[0277] By setting the position of the reference sample and the sub-blocks used to derive the affine seed vector in different ways, the width of the sub-seed vector can be set to a power of 2.

[0278] Alternatively, if the motion vectors of the lower left and lower right sub-blocks are available, multiple affine seed vectors can be derived based on the width of adjacent blocks.

[0279] Figure 18 This is a diagram illustrating an example of deriving an affine seed vector for an affine merge candidate based on multiple motion vectors of multiple sub-blocks belonging to adjacent blocks.

[0280] When the upper boundary of the current block is in contact with the boundary of the coding tree unit, the affine seed vector of the affine merging candidate can be derived based on the motion vectors of the lower left and lower right sub-blocks of the upper adjacent block above the current block. Assume that the position of the upper left sample of the upper adjacent block is (xNb, yNb), the width and height of the upper adjacent block are NbW and NbH respectively, the lower left sub-block can include the sample at the position (xNb, yNb+yNbH-1), and the lower right sub-block can include the sample at the position (xNb+NbW-1, yNb+NbH-1).

[0281] Multiple affine seed vectors for affine merge candidates can be derived based on the width of adjacent blocks and the coordinate difference between adjacent blocks and the current block. For example, multiple affine seed vectors for affine merge candidates can be derived based on the following equations 11 to 13.

[0282] Equation 11

[0283] V 0x =V LBx <<7+((V) RBx -V LBx )<<(7-log2(Nb W)*(xCb-xNb)

[0284] v 0y =V LBy <<7+((V) RBy -V LBy)<<(7-lo g 2(NbW))*(xCb-xNb)

[0285] Equation 12

[0286] v 1x =V LBx <<7+((V) RBx -V LBx )<<(7-log2(Nn W)*(x Cn+xCb W-xNb)

[0287] v 1y =V LBy <<7+((V) RBy -V LBy )<<(7-log2(Nb W))*(xCb+xCbW-xNb)

[0288] Equation 13

[0289] v 2x -V LBx <<7+((V) RBx -C LBx )<<(7-log2(NbW))*(xCb+xCbW-xNb)+((V LBy -V RBy ) << (7-log2(NbW))*xCbH

[0290] v 2y =V LBy <<7+((V) RBy -V LBy )<<(7-log2(NbW))*(xCb+xCbW-xNb)+((V LBx -V RBx ) << (7-log2(NbW))*xCbH

[0291] In equations 11 to 13, (v 0x v 0y ) represents the first affine seed vector, (v 1x v 1y ) represents the second affine seed vector, and (v 2x v 2y) represents the third affine seed vector. Vlbx represents the horizontal component motion vector of the lower left sub-block, and Vlby represents the vertical component motion vector of the lower left sub-block. Vrbx represents the horizontal component motion vector of the lower right sub-block, and Vrby represents the vertical component motion vector of the lower right sub-block. NbW represents the width of the adjacent blocks, and xCbW and xCbH represent the width and height of the current block, respectively. xCb represents the x-coordinate of the upper left sample of the current block, and xNb represents the x-coordinate of the upper left sample of the adjacent block.

[0292] Affine merging candidates can also be generated by combining multiple affine seed vectors derived from the first adjacent block and the motion vectors of the second adjacent block. For example, in Figure 18 In the example shown, instead of deriving the third affine seed vector of the affine merge candidate based on the motion vectors of the bottom left and bottom right sub-blocks, the third affine seed vector of the affine merge candidate can also be derived based on the motion vector of the sub-block located to the left of the current block.

[0293] Figure 19 This is a diagram illustrating an example of deriving the affine seed vector of an affine merge candidate based on the motion vectors of multiple sub-blocks located to the left of the current block.

[0294] Based on the motion vectors of the lower-left and lower-right sub-blocks of the upper adjacent block above the current block, the affine seed vectors of the affine merge candidates can be derived. Specifically, the first and second affine seed vectors of the affine merge candidates can be derived based on the motion vectors of the lower-left and lower-right sub-blocks. For example, the first and second affine seed vectors of the affine merge candidates can be derived based on Equations 11 and 12.

[0295] A third affine seed vector for affine merging candidates can be derived based on the motion vectors of adjacent blocks to the left of the current block. For example, the motion vectors of adjacent blocks to the left of the current block or adjacent blocks to the lower left corner of the current block can be set as the third affine seed vector. Adjacent blocks to the left of the current block can include samples at positions (xCb-1, yCb+CbH-1), and adjacent blocks to the lower left corner of the current block can include samples at positions (xCb-1, yCb+CbH). Equations 14 and 15 illustrate an example of setting the motion vectors of adjacent blocks to the left of the current block as the third affine seed vector.

[0296] Equation 14

[0297] v 2x =V Lefttx

[0298] v 2y =V Lefty

[0299] Equation 15

[0300] v 2x =V LBx

[0301] v 2y =V LBy

[0302] In equation 14, V Leftx V represents the horizontal motion vector of the block to the left of the current block, and V Lefty This represents the vertical motion vector of the adjacent block to the left of the current block. In Equation 15, V LBx V represents the horizontal motion vector of the adjacent block to the bottom left corner of the current block, and V LBy This represents the vertical motion vector of the adjacent block that is next to the bottom left corner of the current block.

[0303] The third affine seed vector can only be derived using the motion vector of the block adjacent to the bottom left corner of the current block if the motion vector of the block adjacent to the left of the current block is unavailable.

[0304] The method for deriving the third affine seed vector of the affine merge candidate can be set differently depending on whether the motion vector of the adjacent block to the left of the current block is available. For example, if the motion vector of the adjacent block to the left of the current block (e.g., the sub-block adjacent to the left of the current block or the sub-block adjacent to the lower left corner of the current block) is available, Equation 14 or Equation 15 can be used to derive the third affine seed vector. On the other hand, if the motion vector of the adjacent block to the left of the current block is not available, Equation 13 can be used to derive the third affine seed vector.

[0305] The third affine seed vector can also be derived based on the motion vectors of non-adjacent blocks that are not adjacent to the current block in the reference sample. To derive the third affine seed vector, the position of the upper adjacent block used when deriving the affine merging candidate can be used to determine whether to use the motion vectors of adjacent blocks that are adjacent to the current block in the reference sample or the motion vectors of non-adjacent blocks that are not adjacent to the current block in the reference sample.

[0306] Figure 20 This is a diagram illustrating an example of deriving the affine seed vector of an affine merge candidate based on motion information of a non-adjacent or adjacent block located to the left of the current block.

[0307] The position of the block used to derive the third affine seed vector can be determined by comparing the position of the top-left sample of the upper neighboring block used to derive the affine merge candidate with the position of the top-left sample of the current block. For example, as... Figure 20In the example shown in part (a), where the x-coordinate (xNb) of the top-left sample of the upper neighboring block is less than the x-coordinate (xCb) of the top-left sample of the current block, a third affine seed vector can be derived based on the motion vector of a non-adjacent block not adjacent to the left side of the current block. Specifically, the third affine seed vector can be derived based on the motion vector of either the left-side non-adjacent block containing a sample (xNb, yCb+CbH-1) with the same x-coordinate as the top-left sample of the upper neighboring block, or the lower-left non-adjacent block containing a sample at position (xNb, yCb+CbH). Equations 16 and 17 illustrate examples of deriving the third affine seed vector based on the motion vectors of non-adjacent blocks.

[0308] Equation 16

[0309] v 2x =V Leftt2x

[0310] v 2y =V Left2y

[0311] Equation 17

[0312] v 2x =V LB2x

[0313] v 2y =V LB2y

[0314] In equation 16, V Left2x V represents the horizontal motion vector of the non-adjacent block on the left, and V Left2y This represents the vertical motion vector of the non-adjacent block on the left. In Equation 17, V LB2x V represents the horizontal motion vector of the lower left non-adjacent block, and V LB2y This represents the vertical motion vector of the lower left non-adjacent block.

[0315] The third affine seed vector can only be derived using the motion vector of the bottom left non-adjacent block if the motion vector of the left non-adjacent block is unavailable.

[0316] like Figure 20In the example shown in section (b), if the x-coordinate (xNb) of the top-left sample of the upper adjacent block is the same as the x-coordinate (xCb) of the top-left sample of the current block, or if the x-coordinate (xNb) of the top-left sample of the upper adjacent block is greater than the x-coordinate (xCb) of the top-left sample of the current block, the third affine seed vector can be derived based on the motion vector of the adjacent block adjacent to the left or bottom-left corner of the current block. Specifically, the third affine seed vector can be derived based on the motion vector of the left adjacent block including the sample (xCb, yCb+CbH-1) adjacent to the left of the current block, or the left adjacent block including the sample at the position (xCb, yCb+CbH) adjacent to the bottom-left corner of the current block.

[0317] It can also be with Figures 18 to 20 The diagram illustrates different ways to set the position of reference samples for left-adjacent blocks or left-non-adjacent blocks. For example, a block that includes samples (xCb-1, yCb+subH-1) adjacent to the current block can be set as a left-adjacent block, and a block that includes samples (xNb, yCb+subH-1) not adjacent to the current block can be set as a left-non-adjacent block. Here, subH represents the minimum height of the block storing motion information, which can be an integer such as 2, 4, or 8.

[0318] Figure 21 This is a diagram showing the location of the blocks used to derive the affine seed vectors for affine merging candidates.

[0319] For example, such as Figure 21 In the example shown in part (a), where the x-coordinate (xNb) of the top-left sample of the upper neighboring block is less than the x-coordinate (xCb) of the top-left sample of the current block, a third affine seed vector can be derived based on the motion vector of a non-adjacent block that is not adjacent to the left side of the current block. Specifically, the third affine seed vector can be derived based on the motion vector of a non-adjacent block to the left of a sample (xNb, yCb+subH-1) that has the same x-coordinate as the top-left sample of the upper neighboring block and is separated from the bottom-left sample of the upper neighboring block by a predetermined distance (e.g., subH). Equation 18 represents an example of deriving the third affine seed vector based on the motion vector of a non-adjacent block.

[0320] Equation 18

[0321] v 2x =V LeftT2x

[0322] v 2y =V LeftT2y

[0323] In equation 18, V LeftT2x V represents the horizontal motion vector of the non-adjacent block to the left of the sample, which is separated from the lower left sample of the upper adjacent block by a predetermined distance.LeftT2y This represents the vertical motion vector of the non-adjacent block to the left of the sample, which is separated from the lower left sample of the upper adjacent block by a predetermined distance.

[0324] Alternatively, such as Figure 21 In the example shown in part (b), if the x-coordinate (xNb) of the top-left sample of the upper adjacent block is the same as the x-coordinate (xCb) of the top-left sample of the current block, or if the x-coordinate (xNb) of the top-left sample of the upper adjacent block is greater than the x-coordinate (xCb) of the top-left sample of the current block, a third affine seed vector can be derived based on the motion vector of the left adjacent block that is adjacent to the left side of the current block and includes samples (xCb-1, yCb+subH-1) separated from the top-left sample of the current block by a predetermined distance (e.g., subH-1) in the vertical direction. Equation 19 shows an example of deriving the third affine seed vector based on the motion vectors of adjacent blocks.

[0325] Equation 19

[0326] v 2x =V LeftTx

[0327] v 2y =V LeftTy

[0328] In equation 19, V LeftTx V represents the horizontal motion vector of the left adjacent block to the left of the current block, including samples separated by a predetermined vertical distance from the top-left sample of the current block. LeftTy This represents the vertical motion vector of the left adjacent block that is adjacent to the left side of the current block and includes samples that are separated by a predetermined vertical distance from the top left sample of the current block.

[0329] Alternatively, the reference sample can be set relative to... Figures 18 to 21 The positions shown are different. For example, a sample (xCb-1, yCb) that is adjacent to the left side of the current block and has the same y-coordinate as the top-left sample of the current block, or a sample (xNb, yCb) that is not adjacent to the left side of the current block and has the same y-coordinate as the top-left sample of the current block, can be set as a reference sample.

[0330] As described in the example, a first and second affine seed vector of an affine merge candidate can be derived from the adjacent block above the current block, and a third affine seed vector of an affine merge candidate can be derived from the adjacent block to the left of the current block. Similarly, a first and third affine seed vector of an affine merge candidate can be derived from the adjacent block to the left of the current block, and a second affine seed vector of an affine merge candidate can be derived from the adjacent block above the current block.

[0331] As illustrated in Equations 11 to 13, the motion vector V of the lower left sub-block of the upper adjacent block can be used as a basis. LB With the motion vector V of the lower right sub-block RB The difference between the motion vectors is used to derive the affine seed vector for the affine merge candidate. In this case, instead of the motion vector of the lower left or lower right sub-block, the motion vectors of other blocks can be used to derive the affine seed vector for the affine merge candidate. For example, the motion vector V of the lower left sub-block of the upper adjacent block. LB and the motion vector V of the lower right sub-block RB If the values ​​are the same, the motion vector of the block, including the sample located to the right or below the lower right sub-block, can be used instead of V. RB For example, the motion vector V replacing the bottom right sub-block. RB The motion vectors of affine merging candidates can be derived using the motion vectors of a block including samples at positions (xNb+NbW, yNb+NbH-1).

[0332] Alternatively, the motion vector V of the lower left sub-block of the upper adjacent block. LB and the motion vector V of the lower right sub-block RB If the values ​​are the same, the motion vector of the block, which includes samples located to the left or below the lower left sub-block, can be used instead of V. LB For example, the motion vector V that replaces the lower left sub-block. LB The motion vectors of affine merging candidates can be derived using the motion vectors of a block including samples at positions (xNb, yNb+NbH-1+offset). The offset can be an integer greater than 1.

[0333] Alternatively, V can be replaced by the motion vector of the block that includes samples located to the left or below the lower left sub-block. RB Alternatively, V can be replaced by the motion vector of the block that includes samples located to the right or below the lower right sub-block. LB .

[0334] Merging candidates can also be derived by combining multiple motion vectors from multiple adjacent blocks. A merging candidate generated by combining multiple motion vectors from multiple adjacent blocks is called a combined merging candidate. When deriving a combined merging candidate, the affine seed vector of the control point can be derived from one of the multiple adjacent blocks adjacent to the control point.

[0335] Figure 22 This is a diagram illustrating an example of deriving a combined merge candidate by combining multiple motion vectors of multiple adjacent blocks.

[0336] Adjacent blocks can include the upper adjacent block above the current block and the left adjacent block to the left of the current block. The upper adjacent block can be at least one of the following blocks: block B0, which includes the sample at position (xCb+CbW, yCb-1); block B1, which includes the sample at position (xCb+CbW-1, yCb-1); block B2, which includes the sample at position (xCb-1, yCb-1); or block B3, which includes the sample at position (xCb, yCb-1). Here, (xCb, yCb) represents the position of the upper left sample of the current block, and CbW represents the width of the current block. The left adjacent block can be at least one of the following blocks: block A0, which includes the sample at position (xCb-1, yCb+CbH); block A1, which includes the sample at position (xCb-1, yCb+CbH-1); or block A2, which includes the sample at position (xCb-1, yCb). Where CbH represents the height of the current block.

[0337] In addition, adjacent blocks can also include time-adjacent blocks of the current block.

[0338] Candidate blocks for merging can be derived by combining the motion information of two or more adjacent blocks. Candidate blocks for merging can also be derived by combining the motion information of two or three adjacent blocks. Specifically, candidates for merging can also be derived by combining two or more of the motion information of adjacent blocks adjacent to the first control point, adjacent blocks adjacent to the second control point, or adjacent blocks adjacent to the third control point.

[0339] For example, candidate blocks for combination and merging can be derived by combining the motion vectors of two or more adjacent blocks in the following groups: adjacent blocks A2, B2 or B3 adjacent to the upper left control point CP0, adjacent blocks B1 or B0 adjacent to the upper right control point CP1, or temporally adjacent blocks T related to the lower left control point CP2.

[0340] In this scenario, when scanning multiple adjacent blocks adjacent to a control point in a predefined scanning order, the motion vector of the first available adjacent block found can be set as the affine seed vector of the control point. For example, the first affine seed vector of the combination merge candidate associated with CP0 can be set as the motion vector of the first available adjacent block found when searching multiple adjacent blocks in the order of B2, B3, and A2. The second affine seed vector of the combination merge candidate associated with CP1 can be set as the motion vector of the first available adjacent block found when searching multiple adjacent blocks in the order of B1 and B0. The third affine seed vector of the combination merge candidate associated with CP2 can be set as the motion vector of the first available adjacent block found when searching multiple adjacent blocks in the order of A1 and A0.

[0341] When deriving a combined merge candidate, neighboring blocks encoded with an affine motion model or neighboring blocks derived from an affine merge candidate can be set to unavailable.

[0342] Figure 23 This is a diagram showing which adjacent blocks are unavailable.

[0343] When deriving an affine merge candidate from the adjacent block B1 located above the current block, it can be determined that the adjacent block B1 cannot be used to derive a combined merge candidate. Therefore, when deriving the affine seed vector of the combined merge candidate associated with CP1, the motion vector of B1 may not be used.

[0344] Alternatively, when encoding neighboring blocks using an affine motion model, combination and merge candidates can be derived based on the affine vectors of the sub-blocks that include the reference sample. For example, if a neighboring block including a sample at position B1 is encoded using an affine motion model, combination and merge candidates can be derived using the affine vectors of the sub-blocks that include the sample at position B1 of the neighboring block.

[0345] Alternatively, the scan order can be determined by considering whether neighboring blocks are encoded using an affine motion model or whether neighboring blocks are derived through affine merge candidates. For example, the scan order of neighboring blocks encoded using an affine motion model or neighboring blocks derived through affine merge candidates can be set last. For instance, when encoding the neighboring block at position B1 using an affine motion model, a second affine seed vector of the combined merge candidate associated with CP2 can be derived by searching multiple neighboring blocks in the order of B0 and B1.

[0346] When reference image indices differ between multiple adjacent blocks, motion vectors can be scaled based on the reference image with the smallest or largest index. This scaling can also be performed based on the difference in Picture Order Count (POC) between the reference image and the current image.

[0347] Alternatively, the combination of multiple adjacent blocks can be performed using only adjacent blocks with the same reference image index. For example, if at least one reference image index of the multiple adjacent blocks is different, the combination can be set as unusable as a combination merging candidate. Furthermore, the availability of a combination can be determined regardless of whether the motion information or motion vectors of the adjacent blocks are the same. For example, when the motion vectors (e.g., CP0 affine seed vector and CP1 affine seed vector) of multiple adjacent blocks are the same, a combination merging candidate can also be derived by combining multiple adjacent blocks with the same reference image index. Alternatively, the scanning order of the multiple adjacent blocks can be determined by considering the reference image indices of the multiple adjacent blocks. Alternatively, the combination can be performed using only multiple adjacent blocks with the same prediction direction among the multiple adjacent blocks.

[0348] Merge candidates can be generated based on a pre-defined combination order. For example, the combination order can be set as follows.

[0349] 1. {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector}

[0350] 2. {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector}

[0351] 3. {CP0 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0352] 4. {CP1 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0353] 5. {CP0 affine seed vector, CP1 affine seed vector}

[0354] 6. {CP0 affine seed vector, CP2 affine seed vector}

[0355] Although six examples of combinations are shown, fewer or more combinations can also be used.

[0356] The combined candidate generated by combining 3 affine seed vectors can be called a 6-parameter set combined candidate, and the combined candidate generated by combining 2 affine seed vectors can be called a 4-parameter set combined candidate.

[0357] The combination order can be predefined in the encoder and decoder. Alternatively, the combination order can be determined based on at least one of the following: the size, shape, partition type, affine motion model, position of the current block within the coding tree unit, or output order of the reference image. For example, when applying a 4-parameter affine motion model to the current block, multiple combination examples used for a 4-parameter set and candidate can be set to have a higher priority than multiple combination examples used for a 6-parameter set and candidate.

[0358] Merge candidates can be generated based on the combination order, but it can be determined that only merge candidates generated by combining multiple adjacent blocks with the same reference image are usable. Alternatively, when at least one reference image in multiple adjacent blocks is different, merge candidates can be derived by scaling the motion vector based on the reference image with the largest or smallest index. This scaling can be performed based on the difference in output order between the current image and the reference images. Alternatively, when two reference images in multiple adjacent blocks are the same but another reference image is different, merge candidates can be derived by scaling the motion vector of the other adjacent block based on the reference images that are applied to the two adjacent blocks.

[0359] The number of combined merge candidates to be added to the merge candidate list can be determined based on at least one of the number of merge candidates already included in the merge candidate list or the maximum number of merge candidates. Alternatively, depending on the affine motion model of the current block, only 6-parameter set merge candidates can be added to the merge candidate list, or only 4-parameter set merge candidates can be added to the combined merge candidate list.

[0360] For example, when the number of combined merge candidates to be added to the merge candidate list is 1 and a 6-parameter motion model is applied to the current block, a set of 6-parameter merge candidates based on the set combination order can be added to the merge candidate list. Specifically, when the CP0 affine seed vector, CP1 affine seed vector, and CP2 affine seed vector are all available, the combined merge candidate {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector} can be added to the merge candidate list. Alternatively, when the CP2 affine seed vector is unavailable and the CP3 affine seed vector is available, the combined merge candidate {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector} can be added to the merge candidate list.

[0361] Alternatively, when the number of combined merge candidates to be added to the merge candidate list is 1 and a 6-parameter motion model is applied to the current block, a set of 4-parameter merge candidates based on the set combination order can be added to the merge candidate list.

[0362] As another example, the number of combined merge candidates to be added to the merge candidate list can be set differently depending on the affine motion model of the current block. For example, when applying a 6-parameter motion model to the current block, one combined merge candidate can be added to the merge candidate list. On the other hand, when applying a 4-parameter motion model to the current block, two combined merge candidates can be added to the merge candidate list.

[0363] Depending on the number of available affine merge candidates, a combination of merge candidates can be selected to be added to the merge candidate list. For example, when there are two or more available affine merge candidates, only 4-parameter set merge candidates can be added to the merge candidate list. On the other hand, when there are one or fewer available affine merge candidates, only 6-parameter set merge candidates can be added to the merge candidate list. Alternatively, when there are one or fewer available affine merge candidates, N 6-parameter set merge candidates and M 4-parameter set merge candidates can be added to the merge candidate list. Here, N and M are integers, and M can be derived based on a value derived from the maximum number of merge candidates minus N. Alternatively, when there are one or fewer available affine merge candidates, only 4-parameter set merge candidates can be added to the merge candidate list.

[0364] Alternatively, the combination order can be determined based on the availability of multiple affine seed vectors. For example, the availability of multiple affine seed vectors can be considered to add combination merge candidates to the merge candidate list in the following order.

[0365] 1. Cases where CP0, CP1, and CP2 affine seed vectors are available.

[0366] {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector}

[0367] 2. Cases where CP0, CP1, and CP3 affine seed vectors are available.

[0368] {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector}

[0369] 3. When are CP0, CP2, and CP3 affine seed vectors available?

[0370] {CP0 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0371] 4. When are CP0, CP2, and CP3 affine seed vectors available?

[0372] {CP1 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0373] 5. Cases where CP0 and CP1 affine seed vectors are available.

[0374] {CP0 affine seed vector, CP1 affine seed vector}

[0375] 6. Cases where CP0 and CP2 affine seed vectors are available.

[0376] {CP0 affine seed vector, CP2 affine seed vector}

[0377] When there is one combined merge candidate that can be added to the merge candidate list, the combined merge candidate whose condition among conditions 1 to 6 is met first can be added to the merge candidate list. If conditions 1 to 6 are not met, the combined merge candidate may not be added to the merge candidate list.

[0378] As another example, the maximum number of combined merge candidates that can be added to the merge candidate list can also be determined based on the number of available affine merge candidates.

[0379] When the number of merge candidates in the merge candidate list is less than the maximum number, zero merge candidates with a motion vector of 0 can be added to the merge candidate list. Therefore, in affine merge mode, merge candidates can be derived in the following order.

[0380] 1. Export affine merge candidates

[0381] 2. Combining and merging candidates

[0382] 3. Zero merge candidates

[0383] Next, we will explain in detail the inter-frame prediction method that uses translational motion information.

[0384] Motion information for the current block can be derived from the motion information of other blocks. These other blocks can be blocks that were encoded / decoded using inter-frame prediction before the current block. Setting the motion information of the current block to be the same as that of other blocks is defined as a merging mode. Additionally, setting the motion vectors of other blocks to the predicted values ​​of the motion vectors of the current block is defined as a motion vector prediction mode.

[0385] Figure 24 This is a flowchart of the process of exporting motion information of the current block in merge mode.

[0386] Merging candidates for the current block can be exported (S2401). Merging candidates for the current block can be exported from blocks that were encoded / decoded using inter-frame prediction before the current block.

[0387] Candidate blocks used to derive merge candidates can include adjacent blocks, i.e., samples adjacent to the current block. For example, if the coordinates of the top-left sample of the current block are (0, 0), then at least one of the following blocks can be used as candidate blocks: a block including a reference sample at position (-1, H-1), a block including a reference sample at position (W-1, -1), a block including a reference sample at position (W, -1), a block including a reference sample at position (-1, H).

[0388] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block can be set as unsuitable for merging. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, candidate blocks that include the reference sample can be set as unsuitable for merging.

[0389] Merge candidates can also be derived from temporally adjacent blocks included in images different from the current block. For example, merge candidates can be derived from co-located blocks included in a co-located image. Any one of multiple reference images included in a list of reference images can be set as a co-located image. Index information for identifying co-located images among multiple reference images can be sent via a bitstream signal. Alternatively, a reference image with a predefined index among multiple reference images can be identified as a co-located image.

[0390] The motion information of the merged candidate can be set to be the same as that of the candidate block. For example, at least one of the motion vector, reference image index, prediction direction, or bidirectional weighted index of the candidate block can be set as the motion information of the merged candidate.

[0391] A list of merge candidates, including merge candidates, can be generated (S2402).

[0392] The indices of multiple merge candidates in the merge candidate list can be assigned in a predetermined order. For example, the indices can be assigned in the order of merge candidates derived from the left adjacent block, merge candidates derived from the top adjacent block, merge candidates derived from the top right adjacent block, merge candidates derived from the bottom left adjacent block, merge candidates derived from the top left adjacent block, and merge candidates derived from the time adjacent block.

[0393] When there are multiple merge candidates, at least one of them can be selected (S2403). Specifically, information specifying any one of the multiple merge candidates can be sent via a bitstream signal. For example, information 'merge_idx' representing the index of any one of the multiple merge candidates included in the merge candidate list can be sent via a bitstream signal.

[0394] When the number of merge candidates in the merge candidate list is less than a threshold, merge candidates included in the inter-frame motion information list can be added to the merge candidate list. The threshold can be a value calculated from the maximum number of merge candidates the merge candidate list can include, or the maximum number of merge candidates minus an offset. The offset can be an integer such as 1 or 2. The inter-frame motion information list can include merge candidates derived based on blocks encoded / decoded prior to the current block.

[0395] The inter-frame motion information list includes merging candidates derived from blocks encoded / decoded based on inter-frame prediction in the current image. For example, the motion information of the merging candidates included in the inter-frame motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on inter-frame prediction. The motion information may include at least one of motion vectors, reference image indices, prediction directions, or bidirectional weighted indexes.

[0396] For ease of explanation, the merging candidates included in the inter-frame motion information list are referred to as inter-frame merging candidates.

[0397] The maximum number of merge candidates that can be included in the inter-frame motion information list can be predefined in the encoder and decoder. For example, the maximum number of merge candidates that can be included in the inter-frame motion information list can be 1, 2, 3, 4, 5, 6, 7, 8 or greater (e.g., 16).

[0398] Alternatively, information representing the maximum number of merge candidates in the inter-frame motion information list can be transmitted via a bitstream signal. This information can be transmitted at the sequence level, image level, or strip level signaling.

[0399] Alternatively, the maximum number of merged candidates for the inter-frame motion information list can be determined based on the image size, the strip size, or the size of the coding tree unit.

[0400] The inter-frame motion information list can be initialized at the level of images, stripes, tiles, blocks, coding tree units, or rows (or columns) of coding tree units. For example, when initializing a strip, the inter-frame motion information list is also initialized, and the inter-frame motion information list may not include any merge candidates.

[0401] Alternatively, information indicating whether to initialize the inter-frame motion information list can be sent via a bitstream signal. This information can be sent at the strip, tile, brick, or block level. A pre-configured inter-frame motion information list can be used before the information indicates initialization.

[0402] Alternatively, information related to inter-frame merge candidates can be signaled via an image parameter set or a strip header. Even if the strip is initialized, the inter-frame motion information list can include initial inter-frame merge candidates. Therefore, inter-frame merge candidates can be used for the first block encoded / decoded within a strip.

[0403] The blocks are encoded / decoded according to the encoding / decoding order, and multiple blocks encoded / decoded based on inter-frame prediction can be set as inter-frame merging candidates in sequence according to the encoding / decoding order.

[0404] Figure 25 This is a diagram illustrating an example of updating the list of inter-frame motion information.

[0405] When performing inter-frame prediction on the current block (S2501), inter-frame merging candidates can be derived based on the current block (S2502). The motion information of the inter-frame merging candidates can be set to be the same as the motion information of the current block.

[0406] When the inter-frame motion information list is empty (S2503), inter-frame merging candidates derived from the current block can be added to the inter-frame motion information list (S2504).

[0407] When the inter-frame merge candidate is already included in the inter-frame motion information list (S2503), a redundancy check can be performed on the motion information of the current block (or the inter-frame merge candidate derived from the current block) (S2505). The redundancy check is used to determine whether the motion information of the inter-frame merge candidates stored in the inter-frame motion information list is the same as the motion information of the current block. Redundancy checks can be performed on all inter-frame merge candidates stored in the inter-frame motion information list. Alternatively, redundancy checks can be performed on inter-frame merge candidates whose index is above or below a threshold among the inter-frame merge candidates stored in the inter-frame motion information list.

[0408] If inter-frame merge candidates with the same motion information as the current block are not included, inter-frame merge candidates derived from the current block can be added to the inter-frame motion information list (S2508). Whether inter-frame merge candidates are the same can be determined based on whether the motion information (e.g., motion vectors and / or reference image indexes, etc.) of the inter-frame merge candidates are the same.

[0409] At this point, when the maximum number of inter-frame merge candidates has been stored in the inter-frame motion information list (S2506), the earliest inter-frame merge candidate is deleted (S2507), and inter-frame merge candidates derived based on the current block can be added to the inter-frame motion information list (S2508).

[0410] Multiple inter-frame merge candidates can be identified based on their indices. When an inter-frame merge candidate derived from the current block is added to the inter-frame motion information list, the lowest index (e.g., 0) is assigned to the candidate, and the indices of already stored candidates can be incremented by 1. In this case, when the maximum number of inter-frame merge candidates is stored in the inter-frame motion information list, the candidate with the highest index is removed.

[0411] Alternatively, when adding inter-frame merge candidates derived from the current block to the inter-frame motion information list, the inter-frame merge candidate can be assigned the largest index. For example, if the number of inter-frame merge candidates already stored in the inter-frame motion information list is less than the maximum value, the inter-frame merge candidate can be assigned an index with the same value as the number of stored inter-frame merge candidates. Alternatively, if the number of inter-frame merge candidates already stored in the inter-frame motion information list is equal to the maximum value, the inter-frame merge candidate can be assigned an index that is 1 less than the maximum value. Furthermore, the inter-frame merge candidate with the smallest index is removed, and the indices of the remaining stored inter-frame merge candidates are each reduced by 1.

[0412] Figure 26 This is a diagram illustrating an embodiment of updating the inter-frame merging candidate list.

[0413] Assume that inter-frame merge candidates derived from the current block are added to the inter-frame merge candidate list, and that the largest index is assigned to the inter-frame merge candidate. Additionally, assume that the inter-frame merge candidate list already stores the maximum number of inter-frame merge candidates.

[0414] When adding the inter-frame merge candidate HmvpCand[n+1] exported from the current block to the inter-frame merge candidate list HmvpCandList, the inter-frame merge candidate HmvpCand[0] with the smallest index is removed from the stored inter-frame merge candidates, and the indices of the remaining inter-frame merge candidates are decreased by 1 respectively. Alternatively, the index of the inter-frame merge candidate HmvpCand[n+1] exported from the current block can be set to the maximum value (e.g., ...). Figure 26 In the example shown, n).

[0415] If an inter-frame merge candidate that is the same as the inter-frame merge candidate derived from the current block is already stored (S2505), the inter-frame merge candidate derived from the current block may not be added to the inter-frame motion information list (S2509).

[0416] Alternatively, as inter-frame merge candidates derived from the current block are added to the inter-frame motion information list, previously stored inter-frame merge candidates that are identical to those candidates can also be removed. In this case, the indexes of the previously stored inter-frame merge candidates will be updated.

[0417] Figure 27 This is a diagram illustrating an example of how the index of a previously stored inter-frame merge candidate is updated.

[0418] When the index of a stored inter-frame merge candidate that is the same as the inter-frame merge candidate mvCand derived based on the current block is hIdx, deleting the stored inter-frame merge candidate can reduce the index of each inter-frame merge candidate with an index greater than hIdx by 1. For example, in Figure 27 The example shown illustrates that HmvpCand[2], which is the same as mvCand, is removed from the inter-frame motion information list HvmpCandList, and the indices of HmvpCand[3] to HmvpCand[n] are each reduced by 1.

[0419] Additionally, inter-frame merge candidate mvCands derived from the current block can be added to the end of the inter-frame motion information list.

[0420] Alternatively, the index of a stored inter-frame merge candidate that is assigned to the same inter-frame merge candidate derived based on the current block can be updated. For example, the index of a stored inter-frame merge candidate can be changed to the minimum or maximum value.

[0421] Motion information for blocks included in a predetermined region can be set to not be added to the inter-frame motion information list. For example, inter-frame merge candidates derived from motion information of blocks included in the merge processing region cannot be added to the inter-frame motion information list. Since the encoding / decoding order of the blocks included in the merge processing region is not defined, it is inappropriate to use motion information from any of these blocks for inter-frame prediction of other blocks. Therefore, inter-frame merge candidates derived from blocks included in the merge processing region can be excluded from the inter-frame motion information list.

[0422] When performing motion compensation prediction based on sub-blocks, inter-frame merge candidates can be derived based on the motion information of representative sub-blocks among the multiple sub-blocks included in the current block. For example, when using sub-block merge candidates for the current block, inter-frame merge candidates can be derived based on the motion information of representative sub-blocks within the sub-block.

[0423] The motion vectors of sub-blocks can be derived in the following order. First, any of the merge candidates included in the merge candidate list of the current block can be selected, and the initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. Additionally, by adding the initial shift vector to the positions (xSb, ySb) of the reference samples (e.g., the top-left sample or the middle sample) of each sub-block of the coded block, a shifted sub-block with reference sample positions (xColSb, yColSb) can be derived. Equation 20 below represents the equation for deriving the shifted sub-block.

[0424] Equation 20

[0425] (xColSb, yColSb) = (xSb+shVector[0]>>4, ySb+shVector[1]>>4)

[0426] Next, the motion vector of the co-position block corresponding to the center position of the sub-block including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0427] A representative sub-block can mean a sub-block that includes the top-left sample or the center sample of the current block.

[0428] Figure 28 This is a diagram showing the location of a representative sub-block.

[0429] Figure 28(a) shows an example where the child block located to the upper left of the current block is set as the representative child block, and Figure 28 (b) shows an example where the sub-block located at the center of the current block is set as the representative sub-block. When performing motion compensation prediction on a sub-block basis, inter-frame merge candidates for the current block can be derived based on the motion vectors of the sub-blocks that include the top-left sample of the current block or the sub-blocks that include the center sample of the current block.

[0430] The inter-frame prediction mode of the current block can also be used to determine whether to use the current block as an inter-frame merging candidate. For example, blocks encoded / decoded based on an affine motion model can be set as non-inter-frame merging candidates. Therefore, even if the current block is encoded / decoded using inter-frame prediction, the inter-frame prediction motion information list will not be updated based on the current block if the current block's inter-frame prediction mode is affine prediction mode.

[0431] Alternatively, inter-frame merge candidates can be derived from at least one sub-block vector within the sub-blocks included in the block being encoded / decoded based on an affine motion model. For example, an inter-frame merge candidate can be derived using a sub-block located to the upper left, center, or upper right of the current block. Alternatively, the average of the sub-block vectors of multiple sub-blocks can be used as the motion vector for the inter-frame merge candidate.

[0432] Alternatively, inter-frame merge candidates can be derived based on the average of the affine seed vectors of the blocks encoded / decoded using an affine motion model. For example, the average of at least one of the first, second, or third affine seed vectors of the current block can be set as the motion vector of the inter-frame merge candidate.

[0433] Alternatively, the inter-frame motion information list can be configured for different inter-frame prediction modes. For example, at least one of the following can be defined: an inter-frame motion information list for blocks encoded / decoded via intra-block copying, an inter-frame motion information list for blocks encoded / decoded based on a translational motion model, or an inter-frame motion information list for blocks encoded / decoded based on an affine motion model. Depending on the inter-frame prediction mode of the current block, any one of the multiple inter-frame motion information lists can be selected.

[0434] Figure 29 An example of generating a list of inter-frame motion information for different inter-frame prediction modes is shown.

[0435] When encoding / decoding a block based on a non-affine motion model, inter-frame merge candidate `mvCand` derived from the block can be added to the inter-frame non-affine motion information list `HmvpCandList`. Conversely, when encoding / decoding a block based on an affine motion model, inter-frame merge candidate `mvAfCand` derived from the block can be added to the inter-frame affine motion information list `HmvpAfCandList`.

[0436] The affine seed vector of a block can be stored in an inter-frame merge candidate derived from the block encoded / decoded based on the affine motion model. Therefore, the inter-frame merge candidate can be used as a merge candidate for deriving the affine seed vector of the current block.

[0437] In addition to the described list of inter-frame motion information, another list of inter-frame motion information can be defined. Besides the described list of inter-frame motion information (hereinafter referred to as the first inter-frame motion information list), a long-term motion information list (hereinafter referred to as the second inter-frame motion information list) can also be defined. The long-term motion information list includes long-term merging candidates.

[0438] When both the first and second inter-frame motion information lists are empty, inter-frame merge candidates can be added to the second inter-frame motion information list first. Only after the maximum number of available inter-frame merge candidates in the second inter-frame motion information list has been reached can inter-frame merge candidates be added to the first inter-frame motion information list.

[0439] Alternatively, an inter-frame merge candidate can be added to both the second inter-frame motion information list and the first inter-frame motion information list.

[0440] In this case, the already configured second inter-frame motion information list may no longer be updated. Alternatively, the second inter-frame motion information list may be updated when the decoded region is above a predetermined ratio of the stripes. Alternatively, the second inter-frame motion information list may be updated every N coding tree unit rows.

[0441] On the other hand, the first inter-frame motion information list can be updated whenever a block is generated that is encoded / decoded using inter-frame prediction. However, inter-frame merge candidates added to the second inter-frame motion information list can also be set not to be used to update the first inter-frame motion information list.

[0442] Information for selecting either a first inter-frame motion information list or a second inter-frame motion information list can be transmitted via a bitstream signal. When the number of merge candidates included in the merge candidate list is less than a threshold, merge candidates included in the inter-frame motion information list indicated by the information can be added to the merge candidate list.

[0443] Alternatively, the list of inter-frame motion information can be selected based on the size and shape of the current block, the inter-frame prediction mode, whether bidirectional prediction is enabled or disabled, whether motion vectors are refined or not, or whether triangulation is enabled or not.

[0444] Alternatively, if the number of merge candidates in the merge candidate list is still less than the maximum number of merges even after adding the inter-frame merge candidates included in the first inter-frame motion information list, then the inter-frame merge candidates included in the second inter-frame motion information list can be added to the merge candidate list.

[0445] Figure 30 This is a diagram illustrating an example of adding inter-frame merge candidates included in the long-term motion information list to the merge candidate list.

[0446] If the number of merge candidates in the merge candidate list is less than the maximum number, inter-frame merge candidates included in the first inter-frame motion information list HmvpCandList can be added to the merge candidate list. If, even after adding inter-frame merge candidates included in the first inter-frame motion information list to the merge candidate list, the number of merge candidates in the merge candidate list is still less than the maximum number, then inter-frame merge candidates included in the long-term motion information list HmvpLTCandList can be added to the merge candidate list.

[0447] Table 1 illustrates the process of adding inter-frame merging candidates included in the long-term motion information list to the merge candidate list.

[0448] Table 1

[0449]

[0450]

[0451] Inter-frame merge candidates can be configured to include additional information besides motion information. For example, the size, shape, or partitioning information of storage blocks can be added to the inter-frame merge candidates. When constructing the merge candidate list for the current block, only inter-frame merge candidates with the same or similar size, shape, or partitioning information as the current block are used in the inter-frame merge candidate list, or inter-frame merge candidates with the same or similar size, shape, or partitioning information as the current block are preferentially added to the merge candidate list.

[0452] Alternatively, inter-frame motion information lists can be generated for different block sizes, shapes, or partitioning information. Multiple inter-frame motion information lists corresponding to the shape, size, or partitioning information of the current block can be used to generate a merge candidate list for the current block.

[0453] If the number of merge candidates in the current block's merge candidate list is less than a threshold, inter-frame merge candidates included in the inter-frame motion information list can be added to the merge candidate list. The addition process is performed in ascending or descending order of the index. For example, the inter-frame merge candidate with the largest index can be added to the merge candidate list.

[0454] When adding inter-frame merge candidates included in the inter-frame motion information list to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidates and multiple merge candidates already stored in the merge candidate list.

[0455] For example, Table 2 shows the process of adding inter-frame merge candidates to the merge candidate list.

[0456] Table 2

[0457]

[0458] Redundancy checks can also be performed only on some of the inter-frame merging candidates included in the inter-frame motion information list. For example, redundancy checks can be performed only on inter-frame merging candidates with indices above or below a threshold. Alternatively, redundancy checks can be performed only on the N merging candidates with the largest indices or the N merging candidates with the smallest indices.

[0459] Alternatively, redundancy checks can be performed only on some of the merge candidates already stored in the merge candidate list. For example, redundancy checks can be performed only on merge candidates indexed above or below a threshold, or on merge candidates derived from a block at a specific location. This specific location can include at least one of the current block's left neighbor, top neighbor, top-right neighbor, or bottom-left neighbor.

[0460] Figure 31 This is a diagram illustrating an example of performing redundancy checks only on some of the merge candidates.

[0461] When adding an inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the two merge candidates with the largest indices, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1]. Here, NumMerge can represent the number of available spatial merge candidates and temporal merge candidates.

[0462] Unlike the example shown in the figure, when adding an inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can also be performed between the inter-frame merge candidate and the two merge candidates with the smallest index. For example, it can be verified whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, a redundancy check can be performed only on merge candidates derived from a specific location. For example, a redundancy check can be performed only on at least one of the merge candidates derived from the adjacent block to the left of the current block or the merge candidate derived from the adjacent block above the current block. When there is no merge candidate derived from a specific location in the merge candidate list, the inter-frame merge candidate can be added to the merge candidate list without performing a redundancy check.

[0463] If a merge candidate that is identical to the first inter-frame merge candidate is found, the redundancy check of the merge candidate that is identical to the first inter-frame merge candidate can be omitted when performing a redundancy check on the second inter-frame merge candidate.

[0464] Figure 32 This is a diagram illustrating an example of omitting redundancy checks for a specific merge candidate.

[0465] When adding the inter-frame merge candidate HmvpCand[i] at index i to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the merge candidates already stored in the merge candidate list. In this case, if a merge candidate mergeCandList[j] identical to the inter-frame merge candidate HmvpCand[i] is found, the inter-frame merge candidate HmvpCand[i] will not be added to the merge candidate list, and a redundancy check can be performed between the inter-frame merge candidate HmvpCand[i-1] at index i-1 and the merge candidate. In this case, the redundancy check between the inter-frame merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0466] For example, in Figure 32 In the example shown, HmvpCand[i] is determined to be the same as mergeCandList[2]. Therefore, HmvpCand[i] is not added to the merge candidate list, and a redundancy check can be performed on HmvpCand[i-1]. In this case, the redundancy check between HvmpCand[i-1] and mergeCandList[2] can be omitted.

[0467] When the number of merge candidates in the current block's merge candidate list is less than a threshold, in addition to inter-frame merge candidates, at least one of paired merge candidates or zero merge candidates may be included. Paired merge candidates refer to merge candidates whose motion vectors are averaged from two or more merge candidates, and zero merge candidates refer to merge candidates whose motion vectors are 0.

[0468] The current block's merge candidate list can be added in the following order.

[0469] Spatial merge candidate - Temporal merge candidate - Inter-frame merge candidate - (Inter-frame affine merge candidate) - Pairwise merge candidate - Zero merge candidate

[0470] Spatial merge candidates refer to merge candidates derived from at least one of adjacent or non-adjacent blocks, while temporal merge candidates refer to merge candidates derived from the previous reference image. The inter-frame affine merge candidate column represents inter-frame merge candidates derived from blocks encoded / decoded using an affine motion model.

[0471] In advanced motion vector prediction mode, an inter-frame motion information list can be used. For example, when the number of motion vector prediction candidates included in the current block's motion vector prediction candidate list is less than a threshold, the inter-frame merging candidates included in the inter-frame motion information list are set as motion vector prediction candidates related to the current block. Specifically, the motion vectors of the inter-frame merging candidates are set as motion vector prediction candidates.

[0472] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list for the current block is selected, the selected candidate is set as the motion vector prediction value for the current block. After decoding the motion vector residual value for the current block, the motion vector for the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0473] The candidate list for motion vector prediction of the current block can be constructed in the following order.

[0474] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Inter-frame decoding region merging candidate - (Inter-frame decoding region affine merging candidate) - Zero motion vector prediction candidate

[0475] Spatial motion vector prediction candidates refer to motion vector prediction candidates derived from at least one of neighboring or non-neighboring blocks, while temporal motion vector prediction candidates refer to motion vector prediction candidates derived from the previous reference image. The inter-frame affine merging candidate column represents inter-frame motion vector prediction candidates derived from blocks encoded / decoded using an affine motion model. Zero motion vector prediction candidates represent candidates with a motion vector value of 0.

[0476] The coded block can be divided into multiple prediction units, and prediction can be performed on each of the divided prediction units. Here, a prediction unit represents the basic unit used for prediction.

[0477] At least one of vertical, horizontal, diagonal, or slanted lines can be used to divide a coded block. Information for determining at least one of the number, angle, or position of lines used to divide the coded block can be transmitted via a bitstream signal. For example, information indicating any of the candidate partition types for the coded block can be transmitted via a bitstream signal, or information specifying any of a plurality of line candidates for dividing the coded block can be transmitted via a bitstream signal. Alternatively, information for determining the number or type of line candidates for dividing the coded block can be transmitted via a bitstream signal. For example, using a 1-bit flag, it can be determined whether a slanted line with an angle greater than the diagonal and / or an angle less than the diagonal can be used as a line candidate.

[0478] Alternatively, at least one of the following can be adaptively determined based on at least one of the intra-frame prediction mode, inter-frame prediction mode, available merge candidate positions, or adjacent block partitioning types of the coding block: the number, angle, or position of the lines that partition the coding block.

[0479] If a coding block is divided into multiple prediction units, then intra-frame prediction or inter-frame prediction can be performed on each prediction unit.

[0480] Figure 33 This is a diagram illustrating an example of using diagonals to divide a coded block into multiple prediction units.

[0481] like Figure 33 The examples shown in parts (a) and (b) demonstrate that the coded block can be divided into two triangular prediction units using diagonals.

[0482] exist Figure 33 Parts (a) and (b) show the division of a coded block into two prediction units using a diagonal line connecting the two vertices of the coded block. However, a diagonal line whose at least one end does not cross a vertex of the coded block can also be used to divide the coded block into two prediction units.

[0483] Figure 34 This is a diagram illustrating an example of dividing a coded block into two prediction units.

[0484] like Figure 34 The examples shown in parts (a) and (b) can be used to divide the coded block into two prediction units using diagonal lines whose ends touch the upper and lower boundaries of the coded block, respectively.

[0485] Alternatively, such as Figure 34The examples shown in parts (c) and (d) can divide the coded block into two prediction units using diagonal lines whose ends contact the left and right boundaries of the coded block, respectively.

[0486] Alternatively, the coded block can be divided into two prediction blocks of different sizes. For example, the coded block can be divided into two prediction units of different sizes by setting the diagonal line dividing the coded block to contact the two boundary surfaces that form a vertex.

[0487] Figure 35 A diagram showing an example of dividing a coded block into multiple prediction blocks of different sizes is provided.

[0488] As in Figure 35 (a) and Figure 35 In the example shown in (b), the coding block can be divided into two prediction units of different sizes by setting the diagonal connecting the top-left or bottom-right corner of the coding block to pass through the left, right, top, or bottom boundary of the coding block, instead of passing through the top-left or bottom-right corner of the coding block.

[0489] Alternatively, such as in Figure 35 (c) and Figure 35 In the example shown in (d), the coding block can be divided into two prediction units of different sizes by setting the diagonal connecting the top right or bottom left corner of the coding block to pass through the left, right, top, or bottom boundary of the coding block, instead of passing through the top left or bottom right corner of the coding block.

[0490] Each prediction unit generated by dividing the coding block is called the "Nth prediction unit". For example, such as Figures 33 to 35 In the example shown, PU1 can be defined as a first prediction unit, and PU2 can be defined as a second prediction unit. The first prediction unit can refer to a prediction unit that includes samples located in the lower left or upper left of the coded block, and the second prediction unit can refer to a prediction unit that includes samples located in the upper right or lower right of the coded block.

[0491] Conversely, a prediction unit that includes a sample located in the upper right or lower right of the coded block can be defined as a first prediction unit, and a prediction unit that includes a sample located in the lower left or upper left of the coded block can be defined as a second prediction unit.

[0492] The embodiments described later primarily illustrate examples of partitioning using diagonals. Specifically, the process of dividing a coding block into two prediction units using diagonals is called diagonal partitioning or triangular partitioning, and the prediction units generated based on diagonal partitioning are called triangular prediction units. However, it is also possible, in the embodiments described later, to use examples of partitioning using oblique lines at angles different from vertical lines, horizontal lines, or diagonals.

[0493] Whether to apply diagonal partitioning to a coding block can be determined based on at least one of the following: stripe type, maximum number of merging candidates that may be included in the merge candidate list, size of the coding block, shape of the coding block, predicted coding mode of the coding block, or partitioning type of the parent node.

[0494] For example, whether to apply diagonal partitioning to the coded block can be determined based on whether the current stripe is of type B. Diagonal partitioning is only allowed when the current stripe is of type B.

[0495] Alternatively, the decision to apply diagonal partitioning to the coded block can be based on whether the maximum number of merge candidates included in the merge candidate list is two or more. Diagonal partitioning is only permitted when the maximum number of merge candidates included in the merge candidate list is two or more.

[0496] Alternatively, when at least one of the width or height of the hardware is greater than 64, there is a drawback that 64×64-sized data processing units are redundantly accessed. Therefore, when at least one of the width or height of the coded block is greater than a threshold, it may not be permissible to divide the coded block into multiple prediction blocks. For example, when at least one of the height or width of the coded block is greater than 64, diagonal partitioning may not be used.

[0497] Alternatively, whether to apply diagonal partitioning to the coding block can be determined based on at least one of whether the number of samples included in the coding block is below a first threshold or above a second threshold. For example, if the number of samples included in the coding block is below the first threshold or above the second threshold, the coding block is set not to be partitioned diagonally. Alternatively, whether to apply diagonal partitioning to the coding block can be determined based on whether the width-to-height ratio of the coding block is less than the first threshold or greater than the second threshold. The width-to-height ratio whRatio of the coding block can be determined as the ratio of the width CbW to the height CbH of the coding block, as shown in Equation 21 below.

[0498] Equation 21

[0499] whRatio=CbW / CbH

[0500] The second threshold can be the reciprocal of the first threshold. For example, when the first threshold is k, the second threshold can be 1 / k.

[0501] Diagonal division can only be applied to a coding block if the width-to-height ratio of the coding block is between the first and second thresholds.

[0502] Alternatively, triangular partitioning can only be used if the width-to-height ratio of the coded block is less than a first threshold or greater than a second threshold. For example, when the first threshold is 16, diagonal partitioning may not be allowed for coded blocks of size 64×4 or 4×64.

[0503] Alternatively, the permission to split diagonally can be determined based on the partitioning type of the parent node. For example, when the coded block serving as the parent node is partitioned based on a quadtree, diagonal partitioning can be applied to the coded blocks serving as leaf nodes. On the other hand, when the coded block serving as the parent node is partitioned based on a binary or ternary tree, the coded blocks serving as leaf nodes can be set to not allow diagonal partitioning.

[0504] Alternatively, the permissibility of diagonal splitting can be determined based on the predictive coding mode of the coding block. For example, when a coding block is coded with intra-frame prediction, diagonal splitting is permitted only if the coding block is coded with inter-frame prediction or with a predefined inter-frame prediction mode. The predefined inter-frame prediction mode can represent at least one of a merging mode, an advanced motion vector prediction mode, an affine merging mode, or an affine motion vector prediction mode.

[0505] Alternatively, the size of the parallel processing region can be used to determine whether diagonal partitioning is allowed. For example, when the size of the coded block is larger than the size of the parallel processing region, diagonal partitioning may not be used.

[0506] You can also consider two or more of the listed conditions to determine whether to apply diagonal partitioning to the coded block.

[0507] Furthermore, information indicating whether diagonal partitioning is permitted can be transmitted via a bitstream signal. This information can be transmitted at the sequence level, image level, stripe level, or block level. Diagonal partitioning can only be applied to a coded block if the information indicates that diagonal partitioning is permitted and at least one of the listed conditions is met.

[0508] When determining whether to apply diagonal division to a coded block, information indicating the number of lines dividing the coded block or the position of the lines can be sent via a bit stream signal.

[0509] For example, when a coded block is divided diagonally, information indicating the direction of the diagonal dividing the coded block can be sent via a bitstream signal. For instance, a flag indicating the direction of the diagonal, `triangle_partition_type_flag`, can be sent via a bitstream signal. This flag indicates whether the coded block is divided by connecting the top-left and bottom-right diagonals, or by connecting the top-right and bottom-left diagonals. Dividing the coded block by connecting the top-left and bottom-right diagonals can be called a left triangle partitioning type, and dividing the coded block by connecting the top-right and bottom-left diagonals can be called a right triangle partitioning type. For example, a flag value of 0 indicates a left triangle partitioning type, and a flag value of 1 indicates a right triangle partitioning type.

[0510] Information indicating the partition type of a coded block can be sent at the coded block level using signals. Therefore, the partition type can be determined for different coded blocks that are partitioned diagonally.

[0511] As another example, for sequence, image, strip, tile, or coding tree units, information indicating the partition type can be sent using signals. In this case, the partition type of the coding block with diagonal partitioning can be set to be the same within the sequence, image, strip, tile, or coding tree unit.

[0512] Alternatively, for the first coding unit within the coding tree unit that applies diagonal partitioning, the information used to determine the partitioning type is encoded and transmitted by signal, and the second and subsequent coding units that apply diagonal partitioning are set to use the same partitioning type as the first coding unit.

[0513] As another example, the partition type of a coded block can be determined based on the partition types of adjacent blocks. Adjacent blocks can include at least one of the following: the adjacent block to the top-left corner, the adjacent block to the top-right corner, the adjacent block to the bottom-left corner, the adjacent block above, or the adjacent block to the left. For example, the partition type of the current block can be set to the same type as that of its adjacent blocks. Alternatively, the partition type of the current block can be determined based on whether the top-left adjacent block applies a left triangle partition type, or whether the top-right or bottom-left adjacent block applies a right triangle partition type.

[0514] To perform motion prediction compensation on the first and second triangle prediction units, motion information for each unit can be derived. In this case, the motion information of the first and second triangle prediction units can be derived from the merging candidates included in the merging candidate list. To distinguish between a general merging candidate list and the merging candidate list used to derive motion information of triangle prediction units, the merging candidate list used to derive motion information of triangle prediction units is called a triangle merging candidate list, and the merging candidates included in the triangle merging candidate list are called triangle merging candidates. However, applying the aforementioned merging candidate derivation method and merging candidate list construction method to triangle merging candidates and triangle merging candidate list construction methods is also included within the spirit of this invention.

[0515] Information for determining the maximum number of triangle merging candidates that can be included in the triangle merging candidate list can be transmitted via a bitstream signal. This information can represent the difference between the maximum number of merge candidates that the merge candidate list can include and the maximum number of triangle merging candidates that the triangle merging candidate list can include.

[0516] Triangle merging candidates can be derived from spatially adjacent blocks and temporally adjacent blocks of the coded block.

[0517] Figure 36 This is a diagram showing adjacent blocks used to derive triangle merging candidates.

[0518] Triangle merging candidates can be derived using at least one of the following: an upper adjacent block, a left adjacent block, or a co-position block included in an image different from the coded block. An upper adjacent block can be at least one of the following: a block containing samples (xCb+CbW-1, yCb-1) above the coded block, a block containing samples (xCb+CbW, yCb-1) above the coded block, or a block containing samples (xCb-1, yCb-1) above the coded block. A left adjacent block can be at least one of the following: a block containing samples (xCb-1, yCb+CbH-1) to the left of the coded block, or a block containing samples (xCb-1, yCb+CbH) to the left of the coded block. A co-occurring block can be defined as either a block that includes samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coded block within the co-occurring image, or a block that includes samples (xCb / 2, yCb / 2) located at the center of the coded block.

[0519] Neighboring blocks can be searched in a predefined order, and triangle merge candidates can be constructed into a triangle merge candidate list in a predefined order. For example, triangle merge candidates can be searched in the order of B1, A1, B0, A0, C0, B2, and C1 to construct the triangle merge candidate list.

[0520] The motion information of the triangle prediction units can be derived based on the triangle merging candidate list. That is, triangle prediction units can share a single triangle merging candidate list.

[0521] To derive the motion information of the triangle merging unit, information specifying at least one of the triangle merging candidates included in the triangle merging candidate list can be transmitted via a bitstream signal. For example, the index information merge_triangle_idx specifying at least one of the triangle merging candidates can be transmitted via a bitstream signal.

[0522] The index information can specify a combination of merge candidates for the first triangular prediction unit and merge candidates for the second triangular prediction unit. For example, Table 3 below shows an example of a combination of merge candidates based on the index information merge_triangle_idx.

[0523] Table 3

[0524]

[0525]

[0526] A value of 1 in the index information `merge_triangle_idx` indicates that the motion information of the first triangular prediction unit is derived from the merge candidate at index 1, and the motion information of the second triangular prediction unit is derived from the merge candidate at index 0. The merge candidate triangles used to derive the motion information of the first triangular prediction unit and the merge candidate triangles used to derive the motion information of the second triangular prediction unit can be determined through the index information `merge_triangle_idx`.

[0527] The partitioning type of the coding block using diagonal partitioning can also be determined based on the index information. That is, the index information can specify a combination of the merging candidates for the first triangular prediction unit, the merging candidates for the second triangular prediction unit, and the partitioning direction of the coding block. When determining the partitioning type of the coding block based on the index information, the information indicating the diagonal direction of the partitioned coding block, `triangle_partition_type_flag`, does not need to be encoded. Table 4 shows the partitioning types of the coding block based on the index information `merge_triangle_idx`.

[0528] Table 4

[0529] merge_triangle_idx 0 1 2 3 4 5 6 7 8 TriangleDir 0 1 1 0 0 1 1 1 0 merge_triangle_idx 9 10 11 12 13 14 15 16 17 TriangleDir 0 0 0 1 0 0 0 0 1 merge_triangle_idx 18 19 20 21 22 23 24 25 26 TriangleDir 1 1 1 0 0 1 1 1 1 merge_triangle_idx 27 28 29 30 31 32 33 34 35 TriangleDir 1 1 1 0 0 1 0 1 0 merge_triangle_idx 36 37 38 39 TriangleDir 0 1 0 0

[0530] A value of 0 for the variable TriangleDir indicates that the coding block applies the left triangle partitioning type, while a value of 1 for TriangleDir indicates that the coding block applies the right triangle partitioning type. By combining Tables 3 and 4, it is possible to configure the combination of the merging candidates for the first triangle prediction unit, the merging candidates for the second triangle prediction unit, and the partitioning direction of the coding block, based on the index information merge_triangle_idx.

[0531] As another example, index information can be sent using signals only for either the first or second triangle prediction unit, and the index of the triangle merging candidate for the other triangle prediction unit can be determined based on this index information. For example, the triangle merging candidate for the first triangle prediction unit can be determined based on the index information merge_triangle_idx, which represents the index of either triangle merging candidate. Additionally, the triangle merging candidate for the second triangle prediction unit can be specified based on the merge_triangle_idx. For example, the triangle merging candidate for the second triangle prediction unit can be derived by adding or subtracting an offset from the index information merge_triangle_idx. The offset can be an integer such as 1 or 2. For example, the triangle merging candidate for the second triangle prediction unit can be determined as the triangle merging candidate with an index of merge_traingle_idx plus 1. When merge_triangle_idx indicates the triangle merging candidate with the largest index value among the triangle merging candidates, the motion information of the second triangle prediction unit can be derived from the triangle merging candidate with index 0 or the triangle merging candidate with an index obtained by subtracting 1 from merge_triangle_idx.

[0532] Alternatively, motion information for the second triangle prediction unit can be derived from a triangle merging candidate having the same reference image as the triangle merging candidate of the first triangle prediction unit specified according to the index information. The triangle merging candidate having the same reference image as the triangle merging candidate of the first triangle prediction unit can represent at least one of an L0 reference image or an L1 reference image having the same triangle merging candidate as the first triangle prediction unit. When multiple triangle merging candidates exist that have the same reference image as the triangle merging candidate of the first triangle prediction unit, any one of them can be selected based on at least one of whether the merging candidate includes bidirectional motion information or the difference between the index of the merging candidate and the index information.

[0533] As another example, index information can be transmitted separately for the first triangle prediction unit and the second triangle prediction unit using signals. For instance, a first index information 1st_merge_idx for determining triangle merging candidates for the first triangle prediction unit and a second index information 2nd_merge_idx for determining triangle merging candidates for the second triangle prediction unit can be transmitted via a bitstream using signals. Motion information of the first triangle prediction unit can be derived from the triangle merging candidates determined based on the first index information 1st_merge_idx, and motion information of the second triangle prediction unit can be derived from the triangle merging candidates determined based on the second index information 2nd_merge_idx.

[0534] The first index information 1st_merge_idx can represent any index among the triangle merging candidates included in the triangle merging candidate list. The triangle merging candidate of the first triangle prediction unit can be determined as the triangle merging candidate pointed to by the first index information 1st_merge_idx.

[0535] The triangle merging candidate indicated by the first index information 1st_merge_idx can be set as a triangle merging candidate that cannot be used as a second triangle prediction unit. Therefore, the second index information 2nd_merge_idx of the second triangle prediction unit can indicate the index of any of the remaining triangle merging candidates other than the triangle merging candidate indicated by the first index information. When the second index information

[0536] When the value of 2nd_merge_idx is less than the value of the first index information 1st_merge_idx, the triangle merging candidate of the second triangle prediction unit can be determined as a triangle merging candidate with the index information represented by the second index information 2nd_merge_idx. On the other hand, when the value of the second index information 2nd_merge_idx is the same as or greater than the value of the first index information 1st_merge_idx, the triangle merging candidate of the second triangle prediction unit can be determined as a triangle merging candidate with the value of the second index information 2nd_merge_idx plus 1 as the index.

[0537] Alternatively, the decision to signal the second index information can be determined based on the number of triangle merging candidates included in the triangle merging candidate list. For example, if the maximum number of triangle merging candidates that the triangle merging candidate list can include is no more than 2, signaling the second index information can be omitted. When signaling the second index information is omitted, the second triangle merging candidate can be derived by adding or subtracting an offset from the first index information. For example, when the maximum number of triangle merging candidates that the triangle merging candidate list can include is 2 and the first index information is index 0, the second triangle merging candidate can be derived by adding 1 to the first index information. Alternatively, when the maximum number of triangle merging candidates that the triangle merging candidate list can include is 2 and the first index information is 1, the second triangle merging candidate can be derived by subtracting 1 from the first index information.

[0538] Alternatively, when omitting the signal transmission of the second index information, the second index information can be set to a default value. The default value can be 0. By comparing the first and second index information, second triangle merging candidates can be derived. For example, when the second index information is less than the first index information, the merging candidate for index 0 is set as the second triangle merging candidate; and when the second index information is the same as or greater than the first index information, the merging candidate for index 1 is set as the second triangle merging candidate.

[0539] When a triangle merging candidate has unidirectional motion information, that unidirectional motion information is set as the motion information of the triangle prediction unit. Conversely, when a triangle merging candidate has bidirectional motion information, only either L0 motion information or L1 motion information is set as the motion information of the triangle prediction unit. The choice between L0 and L1 motion information can be determined based on the index of the triangle merging candidate or the motion information of another triangle prediction unit.

[0540] For example, when the index of a triangle merging candidate is even, the L0 motion information of the triangle prediction unit is set to 0, and the L1 motion information of the triangle merging candidate is set to the L1 motion information of the triangle prediction unit. Conversely, when the index of a triangle merging candidate is odd, the L1 motion information of the triangle prediction unit is set to 0, and the L0 motion information of the triangle merging candidate is set to 0. Alternatively, when the index of a triangle merging candidate is even, the L0 motion information of the triangle merging candidate can be set to the L0 motion information of the triangle prediction unit, and when the index of a triangle merging candidate is odd, the L1 motion information of the triangle merging candidate can be set to the L1 motion information of the triangle prediction unit. Alternatively, for the first triangle prediction unit, when the number of triangle merging candidates is even, the L0 motion information of the triangle merging candidate can be set to the L0 motion information of the first triangle prediction unit; on the other hand, for the second triangle prediction unit, when the number of triangle merging candidates is odd, the L1 motion information of the triangle merging candidate can be set to the L1 motion information of the second triangle prediction unit.

[0541] Alternatively, when the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit can be set to 0, and the L1 motion information of the triangle merging candidate can be set as the L1 information of the second triangle prediction unit. On the other hand, when the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit can be set to 0, and the L0 motion information of the triangle merging candidate can be set as the L0 motion signal of the second triangle prediction unit.

[0542] The triangle merging candidate list used to derive motion information of the first triangle prediction unit and the triangle merging candidate list used to derive motion information of the second triangle prediction unit can also be set to be different.

[0543] For example, when specifying triangle merging candidates for deriving motion information of the first triangle prediction unit within the triangle merging candidate list based on index information associated with the first triangle prediction unit, the motion information of the second triangle prediction unit can be derived using a triangle merging list that includes the remaining triangle merging candidates other than those indicated by the index information. Specifically, the motion information of the second triangle prediction unit can be derived from any of the remaining triangle merging candidates.

[0544] Therefore, the maximum number of triangle merging candidates included in the triangle merging candidate list of the first triangle prediction unit and the maximum number of triangle merging candidates included in the triangle merging candidate list of the second triangle prediction unit will be different. For example, when the triangle merging candidate list of the first triangle prediction unit includes M merging candidates, the triangle merging candidate list of the second triangle prediction unit may include M-1 merging candidates other than those indicated by the index information of the first triangle prediction unit.

[0545] As another example, the merging candidates for each triangular prediction unit can be derived based on the neighboring blocks adjacent to the coded block, and the availability of neighboring blocks can be determined by taking into account the shape or position of the triangular prediction unit.

[0546] Figure 37 This is a diagram used to illustrate an example of determining the availability of neighboring blocks for each triangular prediction unit.

[0547] Neighboring blocks that are not adjacent to the first triangle prediction unit can be set as neighboring blocks that are unavailable to the first triangle prediction unit, and neighboring blocks that are not adjacent to the second triangle prediction unit can be set as neighboring blocks that are unavailable to the second triangle prediction unit.

[0548] For example, as in Figure 37 In the example shown in (a), when the left triangle partitioning type is applied to the coding block, it can be determined that blocks A1, A0, and A2, which are adjacent to the first triangle prediction unit in the adjacent blocks of the coding block, can be used for the first triangle prediction unit, while blocks B0 and B1 cannot be used for the first triangle prediction unit. Therefore, the triangle merging candidate list associated with the first triangle prediction unit includes triangle merging candidates derived from blocks A1, A0, and A2, but excludes triangle merging candidates derived from blocks B0 and B1.

[0549] As in Figure 37 In the example shown in (b), when the left triangle partitioning type is applied to the coding block, it can be determined that blocks B0 and B1, which are adjacent to the second triangle prediction unit, can be used for the second triangle prediction unit, while blocks A1, A0, and A2 cannot be used for the second triangle prediction unit. Therefore, the triangle merging candidate list associated with the second triangle prediction unit includes triangle merging candidates derived from blocks B0 and B1, but excludes triangle merging candidates derived from blocks A1, A0, and A2.

[0550] Therefore, the number or range of triangle merging candidates that a triangle prediction unit can use can be determined based on at least one of the location of the triangle prediction unit or the partitioning type of the coding block.

[0551] As another example, the merging mode can be applied to only either the first triangle prediction unit or the second triangle prediction unit. Alternatively, another piece of motion information in the first and second triangle prediction units can be set to be the same as the motion information of the triangle prediction unit to which the merging mode is applied, or the motion information of the triangle prediction unit to which the merging mode is applied can be refined to derive it.

[0552] For example, the motion vector and reference image index of the first triangle prediction unit can be derived based on the triangle merging candidate, and the motion vector of the first triangle prediction unit can be refined to derive the motion vector of the second triangle prediction unit. For example, the motion vector of the second triangle prediction unit can be derived by adding or subtracting the refined motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangle prediction unit. The reference image index of the second triangle prediction unit can be set to be the same as that of the first triangle prediction unit.

[0553] Information for determining the fine motion vector representing the difference between the motion vector of the first triangle prediction unit and the motion vector of the second triangle prediction unit can be transmitted via a bitstream signal. This information may include at least one of information representing the magnitude of the fine motion vector or information representing the sign of the fine motion vector.

[0554] Alternatively, the symbol of the fine motion vector can be derived based on at least one of the position, index, or partition type applied to the coding block of the triangular prediction unit.

[0555] As another example, the motion vector and reference image index of either the first triangle prediction unit or the second triangle prediction unit can be transmitted via signaling. The other motion vector in the first triangle prediction unit or the second triangle prediction unit can be derived by refining the motion vector transmitted via signaling.

[0556] For example, based on information transmitted as a signal from the bitstream, the motion vector and reference image index of the first triangle prediction unit can be determined. Furthermore, the motion vector of the second triangle prediction unit can be derived by refining the motion vector of the first triangle prediction unit. For example, the motion vector of the second triangle prediction unit can be derived by adding or subtracting the refined motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangle prediction unit. The reference image index of the second triangle prediction unit can be set to be the same as that of the first triangle prediction unit.

[0557] Motion prediction compensation can be performed on the coded block based on the motion information of the first and second triangle prediction units, respectively. In this case, image quality degradation may occur at the boundary between the first and second triangle prediction units. For example, the presence of edges at the boundaries of the first and second triangle prediction units can lead to a deterioration in image quality continuity. To reduce image quality degradation at the boundary, prediction samples can be derived through smoothing filtering or weighted prediction.

[0558] Predictive samples for the applied diagonal division of the coding block can be derived from a weighted sum of a first predictive sample obtained based on the motion information of the first triangular predictive unit and a second predictive sample obtained based on the motion information of the second triangular predictive unit. Alternatively, predictive samples of the first triangular predictive unit can be derived from a first prediction block determined based on the motion information of the first triangular predictive unit, and predictive samples of the second triangular predictive unit can be derived from a second prediction block determined based on the motion information of the second triangular predictive unit. Furthermore, predictive samples located in the boundary region of the first and second triangular predictive units can be derived from a weighted sum of the first predictive samples included in the first prediction block and the second predictive samples included in the second prediction block. For example, Equation 22 below shows an example of deriving predictive samples of the first and second triangular predictive units.

[0559] Equation 22

[0560] P(x, y)=w1*P1(x, y)+(1-w1)*P2(x, y)

[0561] In Equation 22, P1 represents the first predicted sample, and P2 represents the second predicted sample. w1 represents the weight applied to the first predicted sample, and (1-w1) represents the weight applied to the second predicted sample. As shown in the example in Equation 22, the weight applied to the second predicted sample can be derived by subtracting the weight applied to the first predicted sample from a constant.

[0562] When applying a left-triangle partitioning type to a coding block, the boundary region can include predicted samples with the same x-axis and y-axis coordinates. Conversely, when applying a right-triangle partitioning type to a coding block, the boundary region can include predicted samples with the same x-axis and y-axis coordinates. y The sum of the axis coordinates is the sum of the predicted samples that is above the first threshold and below the second threshold.

[0563] The size of the boundary region can be determined based on at least one of the following: the size of the coding block, the shape of the coding block, the motion information of the triangular prediction unit, the motion vector difference of the triangular prediction unit, the output order of the reference image, or the difference between the first and second prediction samples in the diagonal boundary.

[0564] Figure 38 and Figure 39 This is a diagram illustrating an example of deriving a prediction sample based on a weighted sum of a first and a second prediction sample. Figure 38 This illustrates the case where the left triangle partitioning type is applied to the coded block, and Figure 39 This illustrates the case where the right triangle partitioning type is applied to the coded block. Additionally, Figure 38 (a) and Figure 39 (a) is a diagram showing the predicted state related to the luminance component, and Figure 38 (b) and Figure 39 (b) is a diagram showing the predicted state associated with the chromaticity component.

[0565] In the diagram shown, the numbers written in the prediction samples near the boundary between the first and second prediction units represent the weighting values ​​applied to the first prediction sample. For example, when the number written in the prediction sample is N, the prediction sample can be derived by applying a weighting value of N / 8 to the first prediction sample and a weighting value of (1-(N / 8)) to the second prediction sample.

[0566] In non-boundary regions, either the first or second predicted sample can be identified as a predicted sample. (Refer to...) Figure 38 For example, in regions where the absolute value of the difference between the x-axis and y-axis coordinates is greater than a threshold and belong to the first triangular prediction unit, a first prediction sample derived from the motion information of the first triangular prediction unit can be determined as a prediction sample. On the other hand, in regions where the difference between the x-axis and y-axis coordinates is greater than a threshold and belong to the second triangular prediction unit, a second prediction sample derived from the motion information of the second triangular prediction unit can be determined as a prediction sample.

[0567] Reference Figure 39 For example, in regions where the sum of the x-axis and y-axis coordinates is less than a first threshold, a first predicted sample derived from the motion information of the first triangular prediction unit can be determined as a predicted sample. On the other hand, in regions where the sum of the x-axis and y-axis coordinates is greater than a second threshold, a second predicted sample derived from the motion information of the second triangular prediction unit can be determined as a predicted sample.

[0568] The threshold for identifying non-boundary regions can be determined based on at least one of the size of the coded block, the shape of the coded block, or the color components. For example, when the threshold associated with the luminance component is set to N, the threshold associated with the chrominance component can be set to N / 2.

[0569] The predicted samples included in the boundary region can be derived based on a weighted sum of the first and second predicted samples. In this case, the weighting value applied to the first and second predicted samples can be determined based on at least one of the position of the predicted sample, the size of the coding block, and the shape or color component of the coding block.

[0570] For example, as in Figure 38 In the example shown in (a), prediction samples with the same x-axis and y-axis coordinates can be derived by applying the same weighting to the first and second prediction samples. A prediction sample with an absolute difference of 1 between the x-axis and y-axis coordinates can be derived by setting the weighting ratio applied to the first and second prediction samples to (3:1) or (1:3). Alternatively, a prediction sample with an absolute difference of 2 between the x-axis and y-axis coordinates can be derived by setting the weighting ratio applied to the first and second prediction samples to (7:1) or (1:7).

[0571] Alternatively, such as Figure 38 In the example shown in (b), prediction samples with the same x-axis and y-axis coordinates can be derived by applying the same weighting to the first and second prediction samples, and prediction samples with an absolute value of 1 for the difference between the x-axis and y-axis coordinates can be derived by setting the weighting ratio applied to the first and second prediction samples to (7:1) or (1:7).

[0572] For example, as in Figure 39 In the example shown in (a), a prediction sample whose sum of x-axis and y-axis coordinates is 1 less than the width or height of the coding block can be derived by applying the same weighting to the first and second prediction samples. A prediction sample whose sum of x-axis and y-axis coordinates is the same as or 2 less than the width or height of the coding block can be derived by setting the weighting ratio applied to the first and second prediction samples to (3:1) or (1:3). A prediction sample whose sum of x-axis and y-axis coordinates is 1 greater than or 3 less than the width or height of the coding block can be derived by setting the weighting ratio applied to the first and second prediction samples to (7:1) or (1:7).

[0573] Alternatively, such as in Figure 39In the example shown in (b), a prediction sample whose sum of x-axis and y-axis coordinates is 1 less than the width or height of the coded block can be derived by applying the same weighting values ​​to the first and second prediction samples. Alternatively, a weighting ratio of (7:1) or (1:7) can be set to derive a prediction sample whose sum of x-axis and y-axis coordinates is the same as or 2 less than the width or height of the coded block.

[0574] As another example, the location of the predicted sample or the shape of the coding block can be considered to determine the weighting value. Equations 23 through 25 show examples of deriving the weighting value when applying a left-triangle partitioning type to the coding block. Equation 23 shows an example of deriving the weighting value applied to the first predicted sample when the coding block is a square.

[0575] Equation 23

[0576] w1 = (x - y + 4) / 8

[0577] In Equation 23, x and y represent the positions of the predicted samples. When the coded block is not square, the weights applied to the first predicted sample can be derived as shown in Equations 24 or 25. Equation 24 shows the case where the width of the coded block is greater than its height, and Equation 25 shows the case where the width of the coded block is less than its height.

[0578] Equation 24

[0579] w1=((x / whRatio)-y+4) / 8

[0580] Equation 25

[0581] w1 = (x - (y * whRatio) + 4) / 8

[0582] When the right triangle partitioning type is applied to the coded block, the weights applied to the first predicted sample can be determined as shown in Equations 26 to 28. Equation 26 shows an example of deriving the weights applied to the first predicted sample when the coded block is a square.

[0583] Equation 26

[0584] w1=(CbW-1-xy)+4) / 8

[0585] In Equation 26, CbW represents the width of the coded block. When the coded block is not square, the weighting applied to the first predicted sample can be derived as shown in Equations 27 or 28. Equation 27 shows the case where the width of the coded block is greater than its height, and Equation 28 shows the case where the width of the coded block is less than its height.

[0586] Equation 27

[0587] w1=(CbH-1-(x / whRano)-y)+4) / 8

[0588] Equation 28

[0589] w1=(CbW-1-x-(y*whRatio)+4) / 8

[0590] In Equation 27, CbH represents the height of the coded block.

[0591] As shown in the example, for the predicted samples within the boundary region, the samples included in the first triangular prediction unit can be derived by assigning a larger weighting value to the first predicted sample than to the second predicted sample, and the samples included in the second triangular prediction unit can be derived by assigning a larger weighting value to the second predicted sample than to the first predicted sample.

[0592] When applying diagonal partitioning to a coding block, the coding block can be set to a combined prediction mode that does not apply the combination of intra-prediction mode and merging mode.

[0593] Intra-frame prediction uses reconstructed samples that have already been encoded / decoded from the surrounding blocks to predict the current block. In this case, intra-frame prediction of the current block can use reconstructed samples before the application of the in-loop filter.

[0594] Intra-prediction techniques include matrix-based intra-prediction and general intra-prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra-prediction technique for the current block can be signaled via a bitstream. This information can be a 1-bit flag. Alternatively, the intra-prediction technique for the current block can be determined based on at least one of the current block's position, size, shape, or the intra-prediction techniques of neighboring blocks. For example, when the current block crosses an image boundary, it can be set not to apply matrix-based intra-prediction.

[0595] Matrix-based intra-frame prediction is a method that obtains the predicted block for the current block by performing matrix multiplication between matrices stored in the encoder and decoder and reconstructed samples surrounding the current block. Information specifying any one of the stored matrices can be sent via a bitstream signal. The decoder can then determine the matrix for intra-frame prediction of the current block based on this information and the size of the current block.

[0596] General intra-frame prediction is a method for obtaining the prediction block of the current block based on non-angular intra-frame prediction mode or angular intra-frame prediction mode. The process of performing intra-frame prediction based on general intra-frame prediction is described in more detail below with reference to the accompanying drawings.

[0597] Figure 40 This is a flowchart illustrating an intra-frame prediction method according to an embodiment of the present invention.

[0598] The reference sample line (S4001) for the current block can be determined. The reference sample line refers to the set of reference samples included in the Kth line offset from above and / or to the left of the current block. The reference samples can be derived from the reconstructed samples that have been encoded / decoded around the current block.

[0599] Index information of reference sample lines identifying the current block among multiple reference sample lines can be transmitted via a bitstream using signals. For example, the index information intra_luma_ref_idx for specifying the reference sample line of the current block can be transmitted via a bitstream using signals. This index information can be transmitted via signals on a block-by-block basis.

[0600] Multiple reference sample lines may include at least one of the first, second, third, or fourth lines above and / or to the left of the current block. A reference sample line consisting of the row adjacent to the top of the current block and the column adjacent to the left of the current block can be called an adjacent reference sample line, while other reference sample lines can be called non-adjacent reference sample lines.

[0601] Only some of the multiple reference sample lines can be selected as reference sample lines for the current block. For example, the remaining reference sample lines, excluding the third non-adjacent reference sample line, can be set as candidate reference sample lines. Table 5 shows the indices assigned to each candidate reference sample line.

[0602] Table 5

[0603] Index (intra_luma_ref_idx) Reference sample line 0 Adjacent reference sample lines 1 First non-adjacent reference sample line 2 Second non-adjacent reference sample line

[0604] More candidate reference sample lines than described can be set, or fewer can be set. Furthermore, the number or location of non-adjacent reference sample lines set as candidate reference sample lines is not limited to the described example. For example, a first non-adjacent reference sample line and a third non-adjacent reference sample line can be set as candidate reference sample lines, or a second non-adjacent reference sample line and a third non-adjacent reference sample line can be set as candidate reference sample lines. Alternatively, the first, second, and third non-adjacent reference sample lines can all be set as candidate reference sample lines.

[0605] The number or type of candidate reference sample lines can be determined based on at least one of the following: the size, shape, position, whether or not the current block is divided into sub-blocks, or the intra-frame prediction mode.

[0606] The reference sample line for the current block can also be determined based on at least one of the following: the position, size, shape, or predicted coding patterns of adjacent blocks. For example, when the current block is in contact with the boundary of an image, tile, strip, or coding tree unit, the first reference sample line can be determined as the reference sample line for the current block.

[0607] Alternatively, when the current block is not square, adjacent reference sample lines can be set as reference sample lines for the current block. Alternatively, when the width-to-height ratio of the current block is above or below a threshold, adjacent reference sample lines can be determined as reference sample lines for the current block.

[0608] The reference sample line can include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper and left reference samples can be derived from the reconstructed samples surrounding the current block. The reconstructed samples can be in a state prior to the application of the in-loop filter.

[0609] Next, the intra-prediction mode of the current block can be determined (S4002). For the intra-prediction mode of the current block, at least one of a non-angular intra-prediction mode or an angular intra-prediction mode can be determined as the intra-prediction mode of the current block. Non-angular intra-prediction modes include Planer and DC, and angular intra-prediction modes include 33 or 65 modes from the lower left diagonal to the upper right diagonal.

[0610] Figure 41 This is a diagram illustrating the intra-frame prediction mode.

[0611] Figure 41 (a) shows 35 intra-frame prediction modes, and Figure 41 (b) shows 67 intra-frame prediction modes.

[0612] It can also be defined as compared to Figure 41 The number of more or fewer intra-frame prediction modes shown.

[0613] The Most Probable Mode (MPM) can be set based on the intra-prediction modes of neighboring blocks adjacent to the current block. Neighboring blocks can include the left neighboring block to the left of the current block and the upper neighboring block above the current block.

[0614] The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs can be sent as a signal via a bitstream. Alternatively, the number of MPMs can be determined based on at least one of the predictive coding modes of neighboring blocks, the size of the current block, or its shape. For example, N MPMs can be used when an adjacent reference sample line is determined as the reference sample line of the current block, while M MPMs can be used when a non-adjacent reference sample line is determined as the reference sample line of the current block. M is an integer less than N; for example, N can be 6, and M can be 5, 4, or 3. Therefore, when the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra-prediction mode of the current block can be determined as any one of 6 candidate intra-prediction modes, while when the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra-prediction mode of the current block can be determined as any one of 5 candidate intra-prediction modes.

[0615] Alternatively, a fixed number (e.g., 6 or 5) of MPM candidates can be used, regardless of the index of the reference sample line of the current block.

[0616] An MPM list comprising multiple MPMs can be generated, and information indicating whether an MPM with the same intra-prediction mode as the current block is included in the MPM list can be transmitted via a bitstream signal. This information is a 1-bit flag, referred to as the MPM flag. When the MPM flag indicates that an MPM with the same mode as the current block is included in the MPM list, index information identifying one of the MPMs can be transmitted via a bitstream signal. For example, index information `mpm_idx` specifying any one of multiple MPMs can be transmitted via a bitstream signal. The MPM specified by the index information can be set as the intra-prediction mode for the current block. When the MPM flag indicates that an MPM with the same mode as the current block is not included in the MPM list, residual mode information indicating any of the residual intra-prediction modes other than the MPM can be transmitted via a bitstream signal. The residual mode information represents the index value corresponding to the intra-prediction mode of the current block when the index is reallocated to residual intra-prediction modes other than the MPM. The decoder can sort the MPMs in ascending order and determine the intra-prediction mode of the current block by comparing the residual mode information with the MPMs. For example, when the residual mode information is the same as or smaller than the MPM, the intra-prediction mode of the current block can be derived by adding 1 to the residual mode information.

[0617] When deriving the intra-prediction mode for the current block, some comparisons with residual mode information in the MPM can be omitted. For example, the MPM for non-angular intra-prediction modes can be excluded from the comparison. When a non-angular intra-prediction mode is set to MPM, since the residual mode information explicitly indicates an angular intra-prediction mode, the intra-prediction mode for the current block can be derived by comparing the remaining MPMs (excluding non-angular intra-prediction modes) with the residual mode information. Instead of excluding non-angular intra-prediction modes from the comparison, the result value after adding the number of non-angular intra-prediction modes to the residual mode information can be compared with the MPM.

[0618] Instead of setting the default mode to MPM, information indicating whether the intra-prediction mode of the current block is the default mode can be signaled via a bitstream. This information is a 1-bit flag, which may be called the default mode flag. The default mode flag can be signaled only if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As mentioned above, the default mode can include at least one of planar, DC, vertical, or horizontal modes. For example, when planar is set as the default mode, the default mode flag can indicate whether the intra-prediction mode of the current block is planar. When the default mode flag indicates that the intra-prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information can be set as the intra-prediction mode of the current block.

[0619] When using the default mode flag, the intra prediction mode that is the same as the default mode may not be set to MPM. For example, when the default mode flag indicates whether the intra prediction mode of the current block is planar, the intra prediction mode of the current block can be derived by using 5 MPMs other than the MPM that is equivalent to planar.

[0620] When multiple intra-prediction modes are set as the default mode, index information indicating any of the default modes can be further sent using a signal. The intra-prediction mode of the current block can be set to the default mode indicated by the index information.

[0621] When the index of the reference sample line in the current block is not 0, the default mode is set not to be used. For example, when a non-adjacent reference sample line is determined to be the reference sample line of the current block, it can be set not to use non-angular intra-frame prediction modes such as DC mode or planar mode. Therefore, when the index of the reference sample line is not 0, the default mode flag is not sent by signal, and the value of the default mode flag can be set to a predefined value (i.e., false).

[0622] If the intra-prediction mode of the current block is determined, prediction samples related to the current block can be obtained based on the determined intra-prediction mode (S4003).

[0623] When DC mode is selected, predicted samples related to the current block can be generated based on the average of reference samples. Specifically, the values ​​of all samples in the predicted block can be generated based on the average of reference samples. The average can be derived using at least one of the upper reference sample located above the current block and the left reference sample located to the left of the current block.

[0624] The number or range of reference samples used to derive the average may vary depending on the shape of the current block. For example, when the current block is a non-square block with a width greater than its height, the average can be calculated using only the top reference sample. On the other hand, when the current block is a non-square block with a width less than its height, the average can be calculated using only the left reference sample. That is, when the width and height of the current block are different, the average can be calculated using only the reference sample adjacent to the longer length. Alternatively, it can be determined whether to use only the top reference sample or only the left reference sample to calculate the average based on the width-to-height ratio of the current block.

[0625] When the planar mode is selected, prediction samples can be obtained using horizontal and vertical prediction samples. Specifically, the horizontal prediction sample is obtained based on left and right reference samples located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on upper and lower reference samples located on the same vertical line as the prediction sample. The right reference sample can be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample can be generated by copying the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample can be obtained based on a weighted sum of the left and right reference samples, and the vertical prediction sample can be obtained based on a weighted sum of the upper and lower reference samples. In this case, the weighting value assigned to each reference sample can be determined based on the position of the prediction sample. Prediction samples can also be obtained based on the average or weighted sum of the horizontal and vertical prediction samples. When performing a weighted sum operation, the weighting value assigned to the horizontal and vertical prediction samples can be determined based on the position of the prediction sample.

[0626] When an angle prediction mode is selected, parameters representing the prediction direction (or prediction angle) of the selected angle prediction mode can be determined. Table 6 below shows the intrapredAng parameter for each intrapredangling prediction mode.

[0627] Table 6

[0628]

[0629] Table 6 shows the intra-direction parameters for each intra-prediction mode with an index of any one of 2 to 34 when 35 intra-prediction modes are defined. When more than 33 angular intra-prediction modes are defined, Table 6 further breaks down the intra-direction parameters for setting each angular intra-prediction mode.

[0630] After aligning the top and left reference samples of the current block into a single row, predicted samples can be obtained based on the values ​​of the intra-frame orientation parameters. In this case, when the value of the intra-frame orientation parameter is negative, the left and top reference samples can be aligned into a single row.

[0631] Figure 42 and Figure 43 This is a diagram illustrating an example of a one-dimensional arrangement of reference samples in a row.

[0632] Figure 42 An example of a vertically oriented one-dimensional array of reference samples is shown, and Figure 43 An example of a horizontally oriented one-dimensional array of reference samples is shown. This will be described under the assumption of defining 35 intra-frame prediction modes. Figure 42 and 43 Examples of implementations.

[0633] When the intra-prediction mode index is any one of 11 to 18, a one-dimensional horizontal arrangement of the upper reference sample can be applied, rotating counterclockwise. When the intra-prediction mode index is any one of 19 to 25, a one-dimensional vertical arrangement of the left reference sample can be applied, rotating clockwise. The intra-prediction mode angle can be considered when arranging the reference samples in a row.

[0634] Reference sample determination parameters can be determined based on intra-frame orientation parameters. These parameters may include a reference sample index for specifying the reference sample and weighting parameters for determining the weights applied to the reference sample.

[0635] The reference sample index iIdx and the weighted parameter ifact can be obtained through the following equations 30 and 31, respectively.

[0636] Equation 30

[0637] iIdx=(y+1)*P ang / 32

[0638] Equation 31

[0639] i fact =[(y+1)*P ang ]&31

[0640] In equations 30 and 31, P angThis represents the intra-frame orientation parameter. The reference sample specified by the reference sample index iIdx is equivalent to an integer pixel (Integer pel).

[0641] To derive predicted samples, more than one reference sample can be specified. Specifically, the slope of the prediction pattern can be considered to specify the location of the reference sample used to derive the predicted samples. For example, the reference sample index iIdx can be used to specify the reference sample used to derive the predicted samples.

[0642] In this scenario, when the slope of the intra-prediction mode is not represented by a single reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, when the slope of the intra-prediction mode is the value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample can be obtained by interpolating the first and second reference samples. That is, when the angular line following the intra-prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample can be obtained by interpolating the reference samples that are adjacent to the left, right, or top and bottom of the position through which the angular line passes.

[0643] Equation 32 below represents an example of obtaining a predicted sample based on a reference sample.

[0644] Equation 32

[0645] P(x, y)=((32-i) fact ) / 32)*Ref_1D(x+iIdx+1)+(i fact / 32)*Ref_1D(x+iIdx+2)

[0646] In Equation 32, P represents the predicted sample, and Ref_1D represents any one of the reference samples in a one-dimensional arrangement. In this case, the position of the reference sample can be determined based on the position (x, y) of the predicted sample and the index iIdx of the reference sample.

[0647] When the slope of the intra-frame prediction mode can be represented by a reference sample, the weighting parameter i can be... fact Set it to 0. Therefore, equation 32 can be simplified to equation 33 as shown below.

[0648] Equation 33

[0649] P(x, y) = Ref_1D(x + iIdx + 1)

[0650] Intra-prediction can also be performed on the current block based on multiple intra-prediction modes. For example, intra-prediction modes can be derived for different prediction samples, and prediction samples can be derived based on the intra-prediction modes assigned to each prediction sample.

[0651] Alternatively, intra-prediction modes can be derived for different regions, and intra-prediction can be performed on each region based on the intra-prediction modes assigned to each region. Each region may include at least one sample. The size or shape of the region can be adaptively determined based on at least one of the size, shape, or intra-prediction mode of the current block. Alternatively, at least one of the size or shape of the region can be predefined in the encoder and decoder, independent of the size or shape of the current block.

[0652] Alternatively, intra-prediction can be performed based on multiple intra-prediction methods, and the final prediction sample can be derived based on the average or weighted sum of multiple prediction samples obtained through multiple intra-prediction operations. For example, a first prediction sample can be obtained by performing intra-prediction based on a first intra-prediction mode, and a second prediction sample can be obtained by performing intra-prediction based on a second intra-prediction mode. Then, the final prediction sample can be obtained based on the average or weighted sum of the first and second prediction samples. In this case, the weighting values ​​assigned to the first and second prediction samples can be determined by considering at least one of whether the first intra-prediction mode is a non-angle / angle prediction mode, whether the second intra-prediction mode is a non-angle / angle prediction mode, or the intra-prediction modes of adjacent blocks.

[0653] Multiple intra-frame prediction modes can be a combination of non-angle intra-frame prediction modes and angle prediction modes, a combination of angle prediction modes, or a combination of non-angle prediction modes.

[0654] Figure 44 This is a diagram showing the angle formed between the prediction pattern within the angular frame and a straight line parallel to the x-axis.

[0655] like Figure 44 In the example shown, the angle prediction pattern can exist between the lower left diagonal direction and the upper right diagonal direction. When described as the angle formed by the x-axis and the angle prediction pattern, the angle prediction pattern can exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0656] If the current block is not square, the prediction sample is derived by using a reference sample that is farther away from the prediction sample from the reference samples located on the corner that follows the intra-prediction angle, rather than a reference sample that is closer to the prediction sample, depending on the intra-prediction mode of the current block.

[0657] Figure 45 This is a diagram showing an example of obtaining a predicted sample when the current block is not square.

[0658] For example, as in Figure 45In the example shown in (a), it is assumed that the current block is a non-square shape with a width greater than its height, and the intra-frame prediction mode of the current block is an angular intra-frame prediction mode with an angle between 0 and 45 degrees. In this case, when deriving the prediction sample A near the right column of the current block, a left reference sample L, which is far from the prediction sample in the angular mode located at the angle, is used instead of the upper reference sample T, which is close to the prediction sample.

[0659] As another example, such as in Figure 45 In the example shown in (b), it is assumed that the current block is a non-square shape with a height greater than its width, and the intra-frame prediction mode of the current block is an angular intra-frame prediction mode with an angle between -90 degrees and -135 degrees. In the above case, when deriving the prediction sample A near the lower row of the current block, a situation occurs where an upper reference sample T, which is far from the prediction sample in the angular mode located at the angle, is used instead of a left reference sample L that is close to the prediction sample.

[0660] To address the above issue, when the current block is not square, the intra-prediction mode of the current block can be replaced with an intra-prediction mode in the opposite direction. Therefore, for non-square blocks, a mode with a higher frequency of prediction can be used. Figure 41 The angle prediction modes shown are for angles larger or smaller than the indicated angles. This type of intra-frame prediction mode can be defined as a wide-angle intra-frame prediction mode. A wide-angle intra-frame prediction mode refers to an intra-frame prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0661] Figure 46 This is a diagram illustrating the wide-angle intra-frame prediction mode.

[0662] exist Figure 46 In the example shown, the intra-prediction modes with indices -1 to -14 and the intra-prediction modes with indices 67 to 80 represent wide-angle intra-prediction modes.

[0663] Despite Figure 46 The diagram shows 14 wide-angle intra-prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra-prediction modes (67 to 80) with angles less than -135 degrees, but more or fewer wide-angle intra-prediction modes can be defined.

[0664] When using the wide-angle intra-frame prediction mode, the length of the upper reference sample is set to 2W+1, and the length of the left reference sample is set to 2H+1.

[0665] When using the wide-angle intra-frame prediction mode, a reference sample T can be used for prediction. Figure 45 The sample A shown in (a) can be used to predict the reference sample L. Figure 45 Sample A is shown in (b).

[0666] By adding the existing intra-prediction modes to N wide-angle intra-prediction modes, a total of 67+N intra-prediction modes can be used. For example, Table 7 shows the intra-prediction mode intra-direction parameters when 20 wide-angle intra-prediction modes are defined.

[0667] Table 7

[0668] PredModeIntra -10 -9 -8 -7 -6 -5 -4 -3 -2 intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -1 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 11 12 13 14 15 16 17 18 intraPredAngle 13 11 9 7 5 3 2 1 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -1 -2 -3 -5 -7 -9 -11 -13 -15 PredModeIntra 28 29 30 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -11 -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -1 0 1 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 11 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

[0669] When the current block is not square and the intra-prediction mode of the current block obtained in step S4002 falls within the transformation range, the intra-prediction mode of the current block can be transformed into a wide-angle intra-prediction mode. The transformation range can be determined based on at least one of the size, shape, or ratio of the current block. The ratio can represent the ratio between the width and height of the current block.

[0670] When the current block is a non-square with a width greater than its height, the transformation range can be set from the index of the intra-prediction mode in the upper right diagonal direction (e.g., 66) to (the index of the intra-prediction mode in the upper right diagonal direction - N). Here, N can be determined based on the ratio of the current block. When the intra-prediction mode of the current block falls within the transformation range, the intra-prediction mode can be transformed into a wide-angle intra-prediction mode. The transformation can be performed by subtracting a predefined value from the intra-prediction mode; the predefined value can be the total number of intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 67).

[0671] According to the embodiment, the intra-frame prediction modes between the 66th and 53rd frames can be transformed into wide-angle intra-frame prediction modes between the -1st and -14th frames, respectively.

[0672] When the current block is a non-square with a height greater than its width, the transformation range can be set from the intra-prediction mode index in the lower left diagonal direction (e.g., 2) to (the index of the intra-prediction mode in the lower left diagonal direction + M). Here, M can be determined based on the ratio of the current block. When the intra-prediction mode of the current block falls within the transformation range, the intra-prediction mode can be transformed into a wide-angle intra-prediction mode. The transformation can be performed by adding a predefined value to the intra-prediction mode; the predefined value can be the total number of angular intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 65).

[0673] According to the embodiment, the intra-frame prediction modes between the 2nd and 15th frames can be transformed into wide-angle intra-frame prediction modes between the 67th and 80th frames, respectively.

[0674] Hereinafter, the intra-frame prediction modes that fall within the transform range will be referred to as wide-angle intra-frame replacement prediction modes.

[0675] The transform range can be determined based on the ratio of the current block. For example, Tables 8 and 9 show the transform range when 35 intra-prediction modes and 67 intra-prediction modes, excluding the wide-angle intra-prediction mode, are defined, respectively.

[0676] Table 8

[0677] condition Replace Intra-Prediction Mode W / H = 2 Pattern 2, 3, 4 W / H>2 Patterns 2, 3, 4, 5, 6 W / H = 1 none H / W = 1 / 2 Patterns 32, 33, 34 H / W<1 / 2 Patterns 30, 31, 32, 33, 34

[0678] Table 9

[0679] condition Replace Intra-Prediction Mode W / H = 2 Patterns 2, 3, 4, 5, 6, 7 W / H>2 Patterns 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H = 1 none H / W = 1 / 2 Patterns 61, 62, 63, 64, 65, 66 H / W<1 / 2 Patterns 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0680] As shown in the examples in Tables 8 and 9, the number of wide-angle intra-frame replacement prediction modes falling within the transform range can vary depending on the ratio of the current block.

[0681] The transformation range can be set as shown in Table 10 by subdividing the current block by a certain ratio.

[0682] Table 10

[0683] condition Replace Intra-Prediction Mode W / H = 16 Patterns 12, 13, 14, 15 W / H = 8 Modes 12, 13 W / H = 4 Patterns 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 H / W = 2 Patterns 2, 3, 4, 5, 6, 7 H / W = 1 none W / H = 1 / 2 Patterns 61, 62, 63, 64, 65, 66 W / H = 1 / 4 Patterns 57, 58, 59, 60, 61, 62, 63, 64, 65, 66 W / H = 1 / 8 Pattern 55, 56 H / W = 1 / 16 Patterns 53, 54, 55, 56

[0684] When a non-adjacent reference sample line is determined as the reference sample line for the current block, or when a multi-line intra-prediction coding method is used to select any one of multiple reference sample lines, it can be configured not to use the wide-angle intra-prediction mode. That is, even if the current block is not square and the intra-prediction mode of the current block falls within the transform range, it is possible not to transform the intra-prediction mode of the current block to the wide-angle intra-prediction mode.

[0685] Alternatively, when the intra-prediction mode for the current block is determined to be wide-angle intra-prediction mode, non-adjacent reference sample lines can be set to be unusable as reference sample lines for the current block, or the multi-line intra-prediction coding method for selecting any one of the multiple reference sample lines can be disabled. When the multi-line intra-prediction coding method is not used, adjacent reference sample lines can be set as reference sample lines for the current block.

[0686] When the wide-angle intra-prediction mode is not used, refW and refH are set to the sum of nTbW and nTbH. Therefore, in addition to the top-left reference sample, non-adjacent reference samples at a distance i from the current block can include (nTbW + nTbH + offsetX[i]) upper reference samples and (nTbW + nTbH + offsetY[i]) left reference samples. That is, non-adjacent reference samples at a distance i from the current block can include (2nTbW + 2nTbH + offsetX[i] + offsetY[i] + 1) reference samples. For example, when the value of whRatio is greater than 1, the value of offsetX is set to be greater than the value of offsetY. For example, the value of offsetX is set to 1, and the value of offsetY is set to 0. On the other hand, when the value of whRatio is less than 1, the value of offsetY is set to be greater than the value of offsetX. For example, the value of offsetX is set to 0, and the value of offsetY is set to 1.

[0687] With the use of a wide-angle intra prediction mode in addition to the existing intra prediction modes, the resources required for encoding the wide-angle intra prediction mode increase, potentially reducing coding efficiency. Therefore, instead of directly encoding the wide-angle intra prediction mode, it is proposed to encode an alternative intra prediction mode associated with the wide-angle intra prediction mode, thereby improving coding efficiency.

[0688] For example, when encoding the current block using the 67th wide-angle intra-prediction mode, the 67th wide-angle replacement intra-prediction mode (number 2) can be encoded as the intra-prediction mode for the current block. Conversely, when encoding the current block using the -1st wide-angle intra-prediction mode, the -1st wide-angle replacement intra-prediction mode (number 66) can be encoded as the intra-prediction mode for the current block.

[0689] The decoder can decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode is included in the transform range. When the decoded intra-prediction mode is a wide-angle replacement intra-prediction mode, it can transform the intra-prediction mode into a wide-angle intra-prediction mode.

[0690] Alternatively, when encoding the current block in wide-angle intra-prediction mode, wide-angle intra-prediction mode can also be encoded directly.

[0691] Encoding of the intra-prediction mode can be implemented based on the MPM list. Specifically, when encoding adjacent blocks in the wide-angle intra-prediction mode, the MPM can be set based on the wide-angle replacement intra-prediction mode corresponding to the wide-angle intra-prediction mode.

[0692] The resulting residual image can be derived by subtracting the predicted image from the original image. In this case, transforming the residual image into the frequency domain does not significantly degrade the subjective image quality of the video, even if high-frequency components are removed. Therefore, converting the values ​​of high-frequency components to smaller values ​​or setting them to 0 can improve compression efficiency without causing significant visual distortion. Reflecting these characteristics, the current block can be transformed to decompose the residual image into 2D frequency components. This transformation can be performed using techniques such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST).

[0693] DCT uses cosine transform to decompose (or transform) the residual image into 2D frequency components, while DST uses sine transform to decompose (or transform) the residual image into 2D frequency components. As the transformed result of the residual image, the frequency components can be represented as the basic image. For example, when performing a DCT transform on a block of size N×N, N can be obtained. 2 There are several basic pattern components. The sizes of each basic pattern component in an N×N block can be obtained through transformation. Depending on the transformation technique used, the sizes of the basic pattern components can be called DCT coefficients or DST coefficients.

[0694] The Direct Transformation Technique (DCT) is primarily used to transform images with a high proportion of low-frequency non-zero components. The Direct Transformation Technique (DST) is primarily used for images with a high proportion of high-frequency components.

[0695] Transformation techniques other than DCT or DST can also be used to transform residual images.

[0696] The process of transforming the residual image into two-dimensional frequency components is referred to as a two-dimensional image transformation. Furthermore, the magnitudes of the fundamental pattern components obtained through the transformation are called transformation coefficients. For example, transformation coefficients can refer to DCT coefficients or DST coefficients. When the primary and secondary transformations, which will be described later, are applied simultaneously, the transformation coefficients can represent the magnitudes of the fundamental pattern components generated by the result of the secondary transformation.

[0697] Transform techniques can be determined on a block-by-block basis. A transform technique can be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, when the current block is coded in intra-predictive mode and the size of the current block is less than N×N, the transform technique DST can be used to perform the transform. On the other hand, when the aforementioned conditions cannot be met, the transform technique DCT can be used to perform the transform.

[0698] In the residual image, a portion of the image may not undergo 2D image transformation. This omission of 2D image transformation is called transform skipping. When transform skipping is applied, quantization can be applied to the residual values ​​for which no transformation was performed.

[0699] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the DCT- or DST-based transformation can be defined as the primary transformation, and the process of transforming the block again using the primary transformation is called the secondary transformation.

[0700] The main transform can be performed using any of several transform kernel candidates. For example, the main transform can be performed using any of DCT2, DCT8, or DCT7.

[0701] Different transform cores can be used for the horizontal and vertical directions. Information representing combinations of horizontal and vertical transform cores can also be transmitted as signals via bitstreams.

[0702] The execution units for the primary and secondary transformations will differ. For example, a primary transformation can be performed on an 8×8 block, and a secondary transformation can be performed on 4×4 sub-blocks within the transformed 8×8 block. In this case, the transformation coefficients of the remaining regions where the secondary transformation is not performed can also be set to 0.

[0703] Alternatively, a primary transformation can be performed on a 4×4 block, and a secondary transformation can be performed on an 8×8 region of the 4×4 block that includes the transformation.

[0704] Information indicating whether a second transformation should be performed can be sent via a bitstream signal.

[0705] The inverse of the second inverse transform (second inverse transform) can be performed in the decoder, and the inverse of the main transform (first inverse transform) can be performed on its result. The residual signal of the current block can be obtained as the result of the second inverse transform and the first inverse transform.

[0706] Quantization is used to reduce the energy of the block, and the quantization process involves dividing the transformation coefficients by a specific constant. This constant can be derived from quantization parameters, which can be defined as values ​​between 1 and 63.

[0707] If transform and quantization are performed in the encoder, the decoder can obtain the residual block through inverse quantization and inverse transform. The decoder can obtain the reconstructed block of the current block by adding the predicted block and the residual block.

[0708] If a reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss during quantization and encoding. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter will be referred to as the first reconstructed block, and the reconstructed block after applying the in-loop filter will be referred to as the second reconstructed block.

[0709] A second reconstructed block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block. In this case, SAO or ALF can be applied after the deblocking filter.

[0710] Deblocking filters are used to mitigate the image quality degradation (blocking artifact) that occurs at block boundaries when quantization is performed on a block-by-block basis. To apply a deblocking filter, the block strength (BS) between the first reconstructed block and its adjacent reconstructed blocks can be determined.

[0711] Figure 47 This is a flowchart illustrating the process of determining block strength.

[0712] like Figure 47 In the example shown, P represents the first reconstructed block, and Q represents the adjacent reconstructed block. The adjacent reconstructed block can be adjacent to the left or top of the current block.

[0713] exist Figure 47 The example shown illustrates how to determine block strength by considering the predictive coding patterns of P and Q, whether non-zero transform coefficients are included, whether the same reference image is used to perform inter-frame prediction, and whether the difference in motion vectors is greater than or equal to a threshold.

[0714] Based on the block strength, it can be determined whether a deblocking filter has been applied. For example, when the block strength is 0, filtering may not be performed.

[0715] SAO (Sound Analysis and Offset) is used to mitigate the ringing artifact that occurs when performing quantization in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. Methods for determining the offset include edge offset (EO) or band offset. EO indicates a method of determining the offset of the current sample based on the pattern of surrounding pixels. BO indicates a method of applying a common offset to a set of pixels with similar brightness values ​​within a region. Specifically, pixel brightness can be divided into 32 equal intervals, and pixels with similar brightness values ​​can be grouped together. For example, four adjacent bands out of the 32 bands can be grouped together, and the same offset can be applied to samples belonging to the four bands.

[0716] ALF is a method for generating a second reconstructed image by applying a predefined filter of size / shape to a first reconstructed image or a reconstructed image with a deblocking filter applied. Equation 34 below shows an example of applying ALF.

[0717] Equation 34

[0718]

[0719] You can select any of the predefined filter candidates at the image, coding tree unit, coding block, prediction block, or transform block level. Each filter candidate can have a different size or shape.

[0720] Figure 48 This is a graph showing predefined filter candidates.

[0721] As in Figure 48 In the example shown, at least one of the following rhombuses can be selected: 5×5, 7×7, and 9×9.

[0722] Only 5×5 rhombuses can be used for chromaticity components.

[0723] Examples described focusing on the decoding or encoding process are also included within the scope of this invention. Variations of multiple embodiments described in a predetermined order, in a different order than those described, are also included within the scope of this invention.

[0724] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention, and they can be performed simultaneously or in a different order as needed. Furthermore, in the above embodiments, the structural elements constituting the block diagrams (e.g., units, modules, etc.) can also be implemented as hardware devices or software, and multiple structural elements can be combined to implement a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc., individually or in combination. Examples of computer-readable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. The hardware device can be configured to operate as one or more software modules to perform the processing according to the invention, and vice versa.

[0725] [Industrial Applicability]

[0726] This invention can be applied to electronic devices that encode / decode video.

Claims

1. A method of video decoding, comprising the steps of: determining a first prediction unit and a second prediction unit in a coding block; deriving first motion information of the first prediction unit and second motion information of the second prediction unit; and obtaining a prediction sample in the coding block based on the first motion information and the second motion information, wherein the first motion information of the first prediction unit is obtained according to a first merge candidate, the first merge candidate being specified by first index information among a plurality of merge candidates included in a merge candidate list; the second motion information of the second prediction unit is obtained according to a second merge candidate, the second merge candidate being specified by second index information among the plurality of merge candidates included in the merge candidate list; wherein when the second index information is smaller than the first index information, the second index information is taken as an index of the second merge candidate; wherein when the prediction sample is included in a boundary region of the first prediction unit and the second prediction unit, the prediction sample is derived according to a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information.

2. The video decoding method of claim 1, wherein when the second index information is equal to or greater than the first index information, the second merge candidate has a value obtained by adding 1 to the second index information as an index.

3. The video decoding method of claim 1, wherein a first weighting value applied to the first prediction sample is determined based on an x-axis coordinate and a y-axis coordinate of the prediction sample.

4. The video decoding method of claim 3, wherein a second weighting value applied to the second prediction sample is derived by subtracting the first weighting value from a constant.

5. The video decoding method of claim 1, wherein a size of the boundary region is determined based on at least one of a size of the coding block or a shape of the coding block.

6. A video encoding method comprising the steps of: determining a first prediction unit and a second prediction unit in a coding block; deriving first motion information of the first prediction unit and second motion information of the second prediction unit; and obtaining a prediction sample in the coding block based on the first motion information and the second motion information, wherein the first motion information of the first prediction unit is obtained according to a first merge candidate among a plurality of merge candidates included in a merge candidate list; the second motion information of the second prediction unit is obtained according to a second merge candidate among the plurality of merge candidates included in the merge candidate list, first index information for specifying the first merge candidate and second index information for specifying the second merge candidate are encoded respectively; wherein when an index of the second merge candidate is smaller than an index of the first merge candidate, the second index information is encoded as the index value of the second merge candidate; wherein, when the prediction sample is included in a boundary region of the first prediction unit and the second prediction unit, the prediction sample is derived from a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information. 7.The video encoding method of claim 6, wherein, when the index of the second merge candidate is larger than the index of the first merge candidate, the second index information is encoded using a value obtained by subtracting 1 from the index of the second merge candidate. 8.The video encoding method of claim 6, wherein, a first weighting value applied to the first prediction sample is determined based on x-axis and y-axis coordinates of the prediction sample. 9.The video encoding method of claim 8, wherein, a second weighting value applied to the second prediction sample is derived by subtracting the first weighting value from a constant. 10.The video encoding method of claim 6, wherein, a size of the boundary region is determined based on at least one of a size of the coding block or a shape of the coding block. 11.A video decoding device comprising a processor and a memory for storing a computer program executable on the processor, the processor configured to: determine a first prediction unit and a second prediction unit in a coding block; derive first motion information of the first prediction unit and second motion information of the second prediction unit; and obtain a prediction sample in the coding block based on the first motion information and the second motion information, the first motion information of the first prediction unit is obtained from a first merge candidate, the first merge candidate being specified by first index information among a plurality of merge candidates included in a merge candidate list; The second motion information of the second prediction unit is obtained according to a second merge candidate, the second merge candidate being specified in a plurality of merge candidates included in the merge candidate list by second index information; wherein when the second index information is smaller than the first index information, the second index information is taken as an index of the second merge candidate; wherein the processor is configured to: when the prediction sample is included in a boundary region of the first prediction unit and the second prediction unit, the prediction sample is derived from a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information. 12.The video decoding device of claim 11, wherein, the processor is configured to: when the second index information is equal to or larger than the first index information, the second merge candidate has a value obtained by adding 1 to the second index information as an index. 13.The video decoding device of claim 11, wherein, the processor is configured to: a first weighting value applied to the first prediction sample is determined based on x-axis and y-axis coordinates of the prediction sample. 14.The video decoding device of claim 13, wherein, the processor is configured to: a second weighting value applied to the second prediction sample is derived by subtracting the first weighting value from a constant. 15.The video decoding device of claim 11, wherein, the processor is configured to: determine a size of the boundary region based on at least one of a size of the coding block or a shape of the coding block.

16. A video encoding device comprising: A processor and a memory for storing a computer program capable of running on the processor, the processor is configured to: determine a first prediction unit and a second prediction unit in a coding block; derive first motion information of the first prediction unit and second motion information of the second prediction unit; and obtain a prediction sample in the coding block based on the first motion information and the second motion information, wherein, the first motion information of the first prediction unit is obtained according to a first merge candidate among a plurality of merge candidates included in a merge candidate list; the second motion information of the second prediction unit is obtained according to a second merge candidate among the plurality of merge candidates included in the merge candidate list, first index information for specifying the first merge candidate and second index information for specifying the second merge candidate are encoded respectively; wherein, when the index of the second merge candidate is less than the index of the first merge candidate, the second index information is encoded as the index value of the second merge candidate; wherein the processor is configured to: when the prediction sample is included in a boundary region of the first prediction unit and the second prediction unit, derive the prediction sample according to a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information.

17. The video encoding device of claim 16, wherein, the processor is configured to: when the index of the second merge candidate is greater than the index of the first merge candidate, encode the second index information using a value obtained by subtracting 1 from the index of the second merge candidate.

18. The video encoding device of claim 16, wherein, the processor is configured to: determine a first weighting value applied to the first prediction sample based on an x-coordinate and a y-coordinate of the prediction sample.

19. The video encoding device of claim 18, wherein, the processor is configured to: derive a second weighting value applied to the second prediction sample by subtracting the first weighting value from a constant.

20. The video encoding device of claim 16, wherein, the processor is configured to: determine a size of the boundary region based on at least one of a size of the coding block or a shape of the coding block.

21. A storage medium storing an executable program, the executable program being executed by a processor to implement the video decoding method of any one of claims 1 to 5.

22. A storage medium storing an executable program, the executable program being executed by a processor to implement the video encoding method of any one of claims 6 to 10.

23. A method of transmitting a bitstream, the method comprising: execute the video encoding method of any one of claims 6 to 10 to generate a bitstream; and transmit the bitstream.

Images