Video signal encoding / decoding method and apparatus for the method

Abstract

The video decoding method of the present invention includes the following steps: determining whether to divide an encoded block into a first prediction unit and a second prediction unit; when it is determined that the encoded block is divided, determining the division type of the encoded block; deriving first motion information of the first prediction unit of the encoded block and second motion information of the second prediction unit; and obtaining prediction samples in the encoded block based on the first motion information and the second motion information.

Description

[0001] Division Explanation

[0002] This application is a divisional application of Chinese Patent Application No. 201980070594.4, titled "Video Signal Encoding / Decoding Method and Apparatus for the Method", which is the national stage entry of PCT International Patent Application PCT / KR2019 / 015096 with a filing date of November 7, 2019.

[0003] Cross - Reference to Related Applications

[0004] This application is based on and claims priority to Korean Patent Application No. 10 - 2018 - 0136255, filed on November 8, 2018. The entire content of the Korean patent application is incorporated herein by reference.

[0005] This application is based on and claims priority to Korean Patent Application No. 10 - 2018 - 0148890, filed on November 27, 2018. The entire content of the Korean patent application is incorporated herein by reference.

[0006] This application is based on and claims priority to Korean Patent Application No. 10 - 2018 - 0149064, filed on November 27, 2018. The entire content of the Korean patent application is incorporated herein by reference. Technical Field

[0007] The present invention relates to a video signal encoding / decoding method and an apparatus for the method. Background Art

[0008] As display panels become larger and larger, there is an increasing need for video services with higher image quality. The biggest problem with high-definition video services is the substantial increase in data volume. To solve this problem, research on improving video compression rates is being actively conducted. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union - Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013, and its compression performance is approximately twice that of H.264 / AVC. With the rapid development of high-definition video services, the limitations of HEVC's performance have gradually emerged. Summary of the Invention

[0009] Technical Problem to be Solved

[0010] An object of the present invention is to provide a method for dividing an encoding block into a plurality of prediction blocks when encoding / decoding a video signal, and a device for performing the method.

[0011] An object of the present invention is to provide a method for deriving motion information of each of a plurality of prediction blocks when encoding / decoding a video signal, and a device for performing the method.

[0012] An object of the present invention is to provide a method for deriving merge candidates using an inter-frame motion information list when encoding / decoding a video signal, and a device for performing the method.

[0013] The technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs through the following description.

[0014] Technical Solution

[0015] The video signal encoding / decoding method of the present invention includes the following steps: determining whether to divide an encoding block into a first prediction unit and a second prediction unit; when it is determined to divide the encoding block, determining the division type of the encoding block; deriving first motion information of the first prediction unit of the encoding block and second motion information of the second prediction unit; and obtaining prediction samples in the encoding block based on the first motion information and the second motion information. In this case, the first motion information of the first prediction unit can be obtained according to a first merge candidate, which is specified from among a plurality of merge candidates included in a merge candidate list by first index information, and the second motion information of the second prediction unit can be obtained according to a second merge candidate, which is specified from among the plurality of merge candidates included in the merge candidate list by second index information.

[0016] In the video signal encoding / decoding method of the present invention, when the value of the second index information is equal to or greater than the value of the first index information, the second merge candidate may have a value obtained by adding 1 to the value of the second index information as an index.

[0017] In the video signal encoding / decoding method of the present invention, when the value of the second index information is less than the value of the first index information, the value of the second index information may be used as the index of the second merge candidate.

[0018] In the video signal encoding / decoding method of the present invention, when the prediction samples are included in a boundary region between the first prediction unit and the second prediction unit, the prediction samples may be derived by performing a weighted sum operation on a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information.

[0019] In the video signal encoding / decoding method of the present invention, a first weighting value applied to the first prediction sample may be determined based on the x-axis coordinate and y-axis coordinate of the prediction samples.

[0020] In the video signal encoding / decoding method of the present invention, a second weighting value applied to the second prediction sample may be derived by subtracting the first weighting value from a constant.

[0021] In the video signal encoding / decoding method of the present invention, the size of the boundary region may be determined based on at least one of the size of the encoding block or the shape of the encoding block.

[0022] The features briefly outlined above for the present invention are merely exemplary embodiments of the detailed description of the present invention to be described later and do not limit the scope of the present invention.

[0023] Advantages of the Invention

[0024] According to the present invention, the inter-frame prediction efficiency can be improved by providing a method for dividing a coding block into a plurality of prediction blocks and deriving motion information for each of the plurality of prediction blocks.

[0025] According to the present invention, the inter-frame prediction efficiency can be improved by providing a method for deriving merge candidates using an inter-frame motion information list.

[0026] The effects obtainable in the present invention are not limited to the above effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention pertains through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0028] Figure 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0029] Figure 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0030] Figure 4 is a diagram showing various division types of a coding block.

[0031] Figure 5 is a diagram showing an example of the division of a coding tree unit.

[0032] Figure 6 is a flowchart of an inter-frame prediction method according to an embodiment of the present invention.

[0033] Figure 7 is a diagram showing the non-linear motion of an object.

[0034] Figure 8 is a flowchart showing an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0035] Figure 9 is a diagram showing an example of an affine seed vector for each affine motion model.

[0036] Figure 10 is a diagram showing an example of an affine vector of a sub-block under a 4-parameter motion model.

[0037] Figure 11 is a diagram showing adjacent blocks that can be used to derive merge candidates.

[0038] Figure 12 is a diagram showing the derivation of an affine seed vector of a current block based on the affine seed vectors of affine adjacent blocks.

[0039] Figure 13 It is a diagram showing an example of setting the motion vector of a sub-block to the affine seed vector of an affine neighboring block.

[0040] Figures 14 to 16 It is a diagram showing the position of a reference sample.

[0041] Figure 17 It is a diagram showing an example of applying a modified affine merge vector derivation method.

[0042] Figure 18 It is a diagram showing an example of deriving an affine seed vector of an affine merge candidate based on multiple motion vectors of multiple sub-blocks belonging to neighboring blocks.

[0043] Figure 19 It is a diagram showing an example of deriving an affine seed vector of an affine merge candidate based on the motion vectors of multiple sub-blocks located to the left of the current block.

[0044] Figure 20 It is a diagram showing an example of deriving an affine seed vector of an affine merge candidate based on the motion information of non-neighboring blocks or neighboring blocks located to the left of the current block.

[0045] Figure 21 It is a diagram showing the position of a block for deriving an affine seed vector of an affine merge candidate.

[0046] Figure 22 It is a diagram for explaining an example of deriving a combined merge candidate by combining multiple motion vectors of multiple neighboring blocks.

[0047] Figure 23 It is a diagram showing unavailable neighboring blocks.

[0048] Figure 24 It is a flowchart of the process of deriving the motion information of the current block in the merge mode.

[0049] Figure 25 It is a diagram for explaining an example of updating the inter-frame motion information list.

[0050] Figure 26 It is a diagram showing an embodiment of updating the inter-frame merge candidate list.

[0051] Figure 27 It is a diagram showing an example in which the index of a previously stored inter-frame merge candidate is updated.

[0052] Figure 28 It is a diagram showing the position of a representative sub-block.

[0053] Figure 29 It shows an example of generating an inter-frame motion information list for different inter-frame prediction modes.

[0054] Figure 30 A diagram showing an example of adding an inter-frame merge candidate included in a long-term motion information list to a merge candidate list.

[0055] Figure 31 A diagram showing an example of performing a redundancy check only on some of the merge candidates.

[0056] Figure 32 A diagram showing an example of omitting a redundancy check for a specific merge candidate.

[0057] Figure 33 A diagram showing an example of using a diagonal line to divide an encoded block into multiple prediction units.

[0058] Figure 34 A diagram showing an example of dividing an encoded block into two prediction units.

[0059] Figure 35 A diagram showing an example of dividing an encoded block into multiple prediction blocks of different sizes.

[0060] Figure 36 A diagram showing adjacent blocks for deriving triangular merge candidates.

[0061] Figure 37 A diagram for describing an example of determining the availability of adjacent blocks for each triangular prediction unit.

[0062] Figure 38 and Figure 39 A diagram showing an example of deriving a prediction sample based on a weighted sum operation of a first prediction sample and a second prediction sample.

[0063] Figure 40 A flowchart showing an intra-frame prediction method according to an embodiment of the present invention.

[0064] Figure 41 A diagram showing intra-frame prediction modes.

[0065] Figure 42 and Figure 43 A diagram showing an example of a one-dimensional arrangement in which reference samples are arranged in a row.

[0066] Figure 44 A diagram showing an angle formed between an angular intra-frame prediction mode and a line parallel to the x-axis.

[0067] Figure 45 A diagram showing an example of obtaining a prediction sample when a current block is non-square.

[0068] Figure 46 A diagram showing a wide-angle intra-frame prediction mode.

[0069] Figure 47 is a flowchart showing a process of determining block strength.

[0070] Figure 48 is a diagram showing predefined filter candidates. Detailed Description of the Invention

[0071] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0072] Video encoding and decoding are performed in units of blocks. For example, encoding / decoding processes such as transformation, quantization, prediction, in-loop filtering, or reconstruction can be performed on encoding blocks, transform blocks, or prediction blocks.

[0073] Hereinafter, a block to be encoded / decoded is referred to as a "current block". For example, depending on the current encoding / decoding processing step, the current block can represent an encoding block, a transform block, or a prediction block.

[0074] In addition, the term "unit" used in this specification represents a basic unit for performing a specific encoding / decoding process, and a "block" can be understood to represent an array of samples of a predetermined size. Unless otherwise specified, "block" and "unit" can be used interchangeably. For example, in the embodiments described later, an encoding block and an encoding unit can be understood to have the same meaning.

[0075] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0076] Referring to Figure 1 , the video encoding device 100 may include an image partitioning unit 110, prediction units 120, 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0077] Figure 1 Each component shown is separately shown to represent mutually different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, for ease of explanation, each structural element is listed and included as a representative structural element, and at least two structural elements can be combined into one structural element or one structural element can be divided into multiple structural elements and thereby perform functions. As long as the essence of the present invention is not departed from, embodiments integrating these structural elements and embodiments separating these structural elements also fall within the scope of the claims of the present invention.

[0078] In addition, some structural elements are not essential structural elements for performing the essential functions in the present invention, but are optional structural elements only for improving performance. The present invention can be implemented by including only the components necessary for realizing the essence of the present invention (excluding the structural elements for improving performance), and the structure including only the essential structural elements (excluding the structural elements for improving performance) also falls within the scope of the claims of the present invention.

[0079] The image partitioning unit 110 may partition the input image into at least one processing unit. In this case, the processing unit may be a Prediction Unit (PU), a Transform Unit (TU), or a Coding Unit (CU). The image partitioning unit 110 partitions an image into a combination of multiple coding units, prediction units, and transform units, and may select a combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the image.

[0080] For example, an image may be partitioned into multiple coding units. To partition an image into coding units, a recursive tree structure such as a Quad Tree Structure may be used. A video or the largest coding unit may be used as the root, and the coding unit may be partitioned into other coding units. The coding unit may have as many child nodes as the number of partitioned coding units. The coding unit that is no longer partitioned according to certain restrictions will become a leaf node. That is, when assuming that a coding unit can only achieve square partitioning, a coding unit can be partitioned into at most 4 other coding units.

[0081] Hereinafter, in the embodiments of the present invention, the coding unit may mean a unit for performing encoding and may also mean a unit for performing decoding.

[0082] The prediction units within a coding unit may be partitioned into at least one square or rectangle of the same size, or a prediction unit within a coding unit may be partitioned into a shape and / or size different from another prediction unit.

[0083] When the prediction unit for performing intra prediction based on the coding unit is not the smallest coding unit, intra prediction may be performed without partitioning into multiple N×N prediction units.

[0084] The prediction units 120 and 125 may include an inter-frame prediction unit 120 that performs inter-frame prediction and an intra-frame prediction unit 125 that performs intra-frame prediction. It is possible to determine whether to use inter-frame prediction or intra-frame prediction for a prediction unit, and to determine specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and the specific content. For example, the prediction method and prediction mode, etc. may be determined by the prediction unit, and the prediction may be performed by the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit 130. In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When using a specific coding mode, it is also possible to directly encode the original block and transmit it to the decoder without generating a prediction block through the prediction units 120 and 125.

[0085] The inter-frame prediction unit 120 may predict a prediction unit based on information of at least one of the previous image or the next image of the current image, and in some cases, may also predict a prediction unit based on information of a part of the region that has been encoded within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0086] The reference image interpolation unit receives reference image information from the memory 155, and may generate pixel information of integer pixels or fractional pixels from the reference image. For luminance pixels, in order to generate pixel information of fractional pixels in units of 1 / 4 pixels, a DCT-based 8th order interpolation filter with different filter coefficients (DCT-based Interpolation Filter) may be used. For chrominance signals, in order to generate pixel information of fractional pixels in units of 1 / 8 pixels, a DCT-based 4th order interpolation filter with different filter coefficients (DCT-based Interpolation Filter) may be used.

[0087] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Methods for calculating the motion vector can use a variety of methods such as the full search block matching algorithm (FBMA), the three-step search method (TSS), the new three-step search algorithm (NTS), etc. Based on the interpolated pixels, the motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel. In the motion prediction unit, the current prediction unit can be predicted by adopting different motion prediction methods. The motion prediction methods can use a variety of methods such as the Skip method, the Merge method, the Advanced Motion Vector Prediction (AMVP) method, the Intra Block Copy method, etc.

[0088] The intra prediction unit 125 can generate a prediction unit based on the reference pixel information around the current block (the reference pixel information is the pixel information within the current image). In the case where the adjacent block of the current prediction unit is a block that has performed inter prediction and the reference pixel is a pixel that has performed inter prediction, the reference pixel included in the block that has performed inter prediction can be used as the reference pixel information for the adjacent block that has performed intra prediction around. That is, in the case where the reference pixel is not available, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0089] In intra prediction, the prediction mode can have an angular prediction mode that uses the reference pixel information according to the prediction direction and a non-angular mode that does not use the direction information when performing prediction. The mode for predicting the luminance information and the mode for predicting the chrominance information can be different. To predict the chrominance information, the intra prediction mode information for predicting the luminance information or the predicted luminance signal information can be used.

[0090] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixels located on the left side of the prediction unit, the pixels located in the upper left, and the pixels located above. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed based on the reference pixels of the transform unit. In addition, intra prediction using N×N partitioning can be applied only to the smallest coding unit.

[0091] After applying an Adaptive IntraSmoothing (AIS) filter to reference pixels according to a prediction mode, an intra prediction method may generate a prediction block. The type of the adaptive intra smoothing filter applied to the reference pixels may be different. To perform the intra prediction method, the intra prediction mode of a current prediction unit may be predicted based on the intra prediction modes of prediction units located around the current prediction unit. In a case where the prediction mode of the current prediction unit is predicted using the mode information predicted from the surrounding prediction units, if the intra prediction mode of the current prediction unit is the same as that of the surrounding prediction units, predetermined flag information may be used to transmit information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same. If the intra prediction mode of the current prediction unit is different from that of the surrounding prediction units, the prediction mode information of the current block may be encoded by performing entropy coding.

[0092] In addition, a residual block including residual information may be generated, where the residual information is a difference between a prediction unit that performs prediction based on the prediction units generated in the prediction units 120 and 125 and the original block of the prediction unit. The generated residual block may be input to the transform unit 130.

[0093] In the transform unit 130, a transform method such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST) may be used to transform the residual block, which includes residual information between the original block and the prediction units generated by the prediction units 120 and 125. Among them, the DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST to transform the residual block may be determined based on the intra prediction mode information of the prediction unit used to generate the residual block. Transformation of the residual block may also be skipped. A flag indicating whether to skip the transformation of the residual block may be encoded. For a residual block, a luminance component, or a chrominance component (below 4:4:4 format) whose size is below a threshold, transformation skip may be allowed.

[0094] The quantization unit 135 may quantize the values transformed into the frequency domain in the transform unit 130. The quantization coefficient may be changed according to the importance of the block or the image. The value calculated in the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0095] The rearrangement unit 160 may perform rearrangement of the coefficient values on the quantized residual values.

[0096] The rearrangement unit 160 can change the two-dimensional block shape coefficients into a one-dimensional vector form by means of coefficient scanning. For example, the rearrangement unit 160 can use the Zig-Zag Scan method to scan the DC coefficients and even the coefficients in the high-frequency domain, and change them into a one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of the Zig-Zag Scan, a vertical scan that scans the two-dimensional block shape coefficients along the column direction and a horizontal scan that scans the two-dimensional block shape coefficients along the row direction can also be used. That is, it is possible to determine which one of the Zig-Zag Scan, the vertical direction scan, and the horizontal direction scan to use according to the size of the transform unit and the intra prediction mode.

[0097] The entropy encoding unit 165 can perform entropy encoding based on the values calculated by the rearrangement unit 160. For example, the entropy encoding can use various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0098] The entropy encoding unit 165 can encode various information such as the residual value coefficient information, block type information, prediction mode information, partitioning unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding units originating from the rearrangement unit 160 and the prediction units 120, 125.

[0099] The entropy encoding unit 165 can perform entropy encoding on the coefficient values of the coding units input from the rearrangement unit 160.

[0100] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the multiple values quantized by the quantization unit 135, and perform inverse transformation on the values transformed by the transform unit 130. A reconstructed block can be generated by combining the residual values generated in the inverse quantization unit 140 and the inverse transform unit 145 with the prediction units predicted by the motion prediction unit, motion compensation unit, and intra prediction unit included in the prediction units 120, 125.

[0101] The filter unit 150 can include at least one of a deblocking filter, an offset correction unit, and an Adaptive Loop Filter (ALF).

[0102] The deblocking filter can remove the block distortion generated in the reconstructed image due to the boundaries between blocks. To determine whether to perform deblocking, it can be judged whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. In the case of applying the deblocking filter to the block, a strong filter or a weak filter can be applied according to the required deblocking filtering strength. In addition, during the use of the deblocking filter, when performing vertical filtering and horizontal filtering, the horizontal direction filtering and the vertical direction filtering can be processed synchronously.

[0103] The offset correction unit can correct the offset between the deblocked image and the original image on a pixel-by-pixel basis. The offset correction of the specified image can be performed in the following manner: after dividing the pixels included in the image into a predetermined number of regions, determining the region to perform the offset, and applying the offset to the corresponding region, or applying the offset taking into account the edge information of each pixel.

[0104] Adaptive Loop Filtering (ALF) can be performed based on the value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into predetermined groups, one filter to be used for the corresponding group can be determined, and filtering can be performed differentially for each group. Information related to whether to apply adaptive loop filtering can be transmitted through the luminance signal for each Coding Unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied can be different according to each block. In addition, the same type (fixed type) of ALF can also be applied regardless of the characteristics of the blocks to which it is applied.

[0105] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the prediction units 120 and 125 during the execution of inter-frame prediction.

[0106] Figure 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0107] Refer to Figure 2 , the video decoder 200 can include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transformation unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0108] When inputting a video bitstream from a video encoder, the input bitstream can be decoded in the reverse steps of the video encoder.

[0109] The entropy decoding unit 210 can perform entropy decoding in the reverse steps of those in which the entropy encoding unit of the video encoder performs entropy encoding. For example, corresponding to the methods performed in the video encoder, various methods such as Exponential Golomb code, Context-Adaptive Variable-Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied.

[0110] The entropy decoding unit 210 can decode information related to intra prediction and inter prediction performed by the encoder.

[0111] The rearrangement unit 215 can perform rearrangement based on the method of rearranging the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. Multiple coefficients represented in a one-dimensional vector form can be reconstructed into a two-dimensional block-shaped coefficient for rearrangement. The rearrangement unit 215 can perform rearrangement by receiving information related to the coefficient scan performed by the encoding unit and performing reverse scanning based on the scan order performed by the corresponding encoding unit.

[0112] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values of the rearranged blocks.

[0113] The inverse transformation unit 225 can perform inverse discrete cosine transform and inverse discrete sine transform on the quantization result performed by the video encoder. The inverse discrete cosine transform and inverse discrete sine transform belong to the inverse transform of the transform performed in the transformation unit, that is, the inverse transform of discrete cosine transform and discrete sine transform. Among them, the DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transformation unit 225 can not perform the inverse transform. The inverse transform can be performed based on the transport unit determined in the video encoder. The inverse transformation unit 225 of the video decoder can selectively perform a transform method (e.g., DCT or DST) according to multiple information such as the prediction method, the size of the current block, and the prediction direction.

[0114] The prediction units 230, 235 can generate a prediction block based on the information related to the generation of the prediction block provided by the entropy decoding unit 210 and the previously decoded block or image information provided by the memory 245.

[0115] As described above, when performing intra prediction in the same manner as the operations in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixels located to the left of the prediction unit, the pixels located in the upper left, and the pixels located above. If the size of the prediction unit during intra prediction is different from the size of the transform unit, intra prediction can be performed using the reference pixels based on the transform unit. Additionally, intra prediction using N×N partitioning can be applied only to the smallest coding unit.

[0116] The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information such as the prediction unit information input from the entropy decoding unit 210, the prediction mode information of the intra prediction method, and the motion prediction related information of the inter prediction method, classifies the prediction unit according to the current coding unit, and determines whether the prediction unit is performing inter prediction or intra prediction. The inter prediction unit 230 may use the information required to perform the inter prediction of the current prediction unit provided by the video encoder and perform inter prediction on the current prediction unit based on the information included in at least one of the previous image or the next image of the current image to which the current prediction unit belongs. Alternatively, inter prediction may also be performed based on the information of a part of the region that has been reconstructed in the current image to which the current prediction unit belongs.

[0117] To perform inter prediction, it is possible to determine, based on the coding unit, which mode among the skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), and intra block copy mode is the motion prediction method of the prediction unit included in the corresponding coding unit.

[0118] The intra prediction unit 235 may generate a prediction block based on the pixel information within the current image. When the prediction unit is a prediction unit for which intra prediction has been performed, intra prediction can be performed based on the intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra smoothing filter is the part that performs filtering on the reference pixels of the current block and can determine whether to apply the filter according to the prediction mode of the current prediction unit. Adaptive intra smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit and the adaptive intra smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode that does not perform adaptive intra smoothing filtering, the adaptive intra smoothing filter may not be applied.

[0119] If the prediction mode of a prediction unit performs intra prediction based on pixel values obtained by interpolating reference pixels, the reference pixel interpolation unit may generate reference pixels in pixel units with integer values or fractional values by interpolating the reference pixels. If the prediction mode of the current prediction unit generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. If the prediction mode of the current block is the DC mode, the DC filter may generate a prediction block by filtering.

[0120] A reconstructed block or image may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0121] Information related to whether to apply a deblocking filter to a corresponding block or image and information related to whether to apply a strong filter or a weak filter when applying the deblocking filter may be received from the video encoder. The deblocking filter of the video decoder may receive information related to the deblocking filter provided by the video encoder, and the video decoder may perform deblocking filtering on the corresponding block.

[0122] The offset correction unit may perform offset correction on the reconstructed image based on the type and offset amount information of the offset correction applied to the image during encoding, etc.

[0123] The ALF may be applied to the coding unit based on information related to whether to apply the ALF provided by the encoder, ALF coefficient information, etc. Such ALF information may be provided by being included in a specific parameter set.

[0124] The memory 245 stores the reconstructed image or block such that the image or block can be used as a reference image or reference block, and may provide the reconstructed image to the output unit.

[0125] Figure 3 It is a diagram showing a basic coding tree unit of an embodiment of the present invention.

[0126] The coding block with the largest size may be defined as a coding tree block. An image may be divided into multiple coding tree units (CTUs). A coding tree unit is the coding unit with the largest size and may also be referred to as the largest coding unit (LCU). Figure 3 An example of dividing an image into multiple coding tree units is shown.

[0127] The size of the coding tree unit may be defined at the image level or sequence level. For this, information indicating the size of the coding tree unit may be signaled through an image parameter set or a sequence parameter set.

[0128] For example, the size of the coding tree unit for the entire image within the sequence can be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image can be set to 128×128, and the size of the coding tree unit in the second image can be set to 256×256.

[0129] Coding blocks can be generated by partitioning the coding tree unit. The coding block represents the basic unit for performing encoding / decoding processing. For example, prediction or transformation can be performed on different coding blocks, or the prediction coding mode can be determined for different coding blocks. Among them, the prediction coding mode represents a method for generating a predicted image. For example, the prediction coding mode can include Intra Prediction (Intra Prediction, intra-frame prediction), Inter Prediction (Inter Prediction, inter-frame prediction), Current Picture Referencing (CPR, or Intra Block Copy (IBC)), or Combined Prediction. For a coding block, at least one of the prediction coding modes of intra-frame prediction, inter-frame prediction, current image reference, or combined prediction can be used to generate a prediction block related to the coding block.

[0130] The information indicating the prediction coding mode of the current block can be signaled through the bitstream. For example, the information can be a 1-bit flag indicating whether the prediction coding mode is an intra mode or an inter mode. Current image reference or combined prediction can be used only when the prediction coding mode of the current block is determined to be the inter mode.

[0131] The current image reference is used to set the current image as a reference image and obtain a prediction block for the current block from the encoded / decoded region within the current image. Here, the current image means the image including the current block. The information indicating whether the current image reference is applied to the current block can be signaled through the bitstream. For example, the information can be a 1-bit flag. When the flag is true, the prediction coding mode of the current block can be determined as the current image reference, and when the flag is false, the prediction mode of the current block can be determined as inter-frame prediction.

[0132] Alternatively, the prediction coding mode of the current block can be determined based on the reference image index. For example, when the reference image index points to the current image, the prediction coding mode of the current block can be determined as the current image reference. When the reference image index points to an image other than the current image, the prediction coding mode of the current block can be determined as inter-frame prediction. That is, the current image reference is a prediction method using the information of the encoded / decoded regions in the current image, and inter-frame prediction is a prediction method using the information of other encoded / decoded images.

[0133] Combined prediction represents an encoding mode composed of more than two of intra-frame prediction, inter-frame prediction, and current image reference. For example, in the case of applying combined prediction, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or current image reference, and a second prediction block can be generated based on another one. When generating the first prediction block and the second prediction block, the final prediction block can be generated through the average operation or weighted sum operation of the first prediction block and the second prediction block. Information indicating whether combined prediction is applied can be signaled through the bitstream. The information can be a 1-bit flag.

[0134] Figure 4 is a diagram showing various partitioning types of an encoding block.

[0135] An encoding block can be divided into multiple encoding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. The divided encoding blocks can also be further divided into multiple encoding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0136] Quadtree partitioning refers to a partitioning technique that divides the current block into 4 blocks. As a result of quadtree partitioning, the current block can be divided into 4 square partitions (refer to "SPLIT_QT (Split_Quadtree)" in part (a) of Figure 4 .

[0137] Binary tree partitioning refers to a partitioning technique that divides the current block into 2 blocks. The process of dividing the current block into two blocks along the vertical direction (i.e., using a vertical line crossing the current block) can be called vertical binary tree partitioning, and the process of dividing the current block into two blocks along the horizontal direction (i.e., using a horizontal line crossing the current block) can be called horizontal binary tree partitioning. As a result of binary tree partitioning, the current block can be divided into 2 non-square partitions. Figure 4 "SPLIT_BT_VER (Split_BinaryTree_Vertical)" in part (b) of Figure 4 represents the result of vertical binary tree partitioning, and

[0138] Trinary tree partitioning refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines that cross the current block) can be called vertical trinary tree partitioning, and the process of dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines that cross the current block) can be called horizontal trinary tree partitioning. As a result of trinary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. Figure 4 The “SPLIT_TT_VER (Split Trinary Tree Vertical)” in part (d) of Figure 4 represents the result of vertical trinary tree partitioning, and Figure 4 the “SPLIT_TT_HOR (Split Trinary Tree Horizontal)” in part (e) of Figure 4 represents the result of horizontal trinary tree partitioning.

[0139] The number of times of partitioning of a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Therefore, the maximum partitioning depth of a coding tree unit can vary according to different sequences or pictures.

[0140] Alternatively, the maximum partitioning depth can be determined separately for each of multiple partitioning techniques. For example, the maximum partitioning depth allowed for quadtree partitioning can be different from the maximum partitioning depth allowed for binary tree partitioning and / or trinary tree partitioning.

[0141] The encoder can signal in the bitstream information representing at least one of the partitioning type or partitioning depth of the current block. The decoder can determine the partitioning type and partitioning depth of the coding tree unit based on the information parsed from the bitstream.

[0142] Figure 5 is a diagram showing an example of partitioning of a coding tree unit.

[0143] The process of partitioning a coding block using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or trinary tree partitioning can be called multi-tree partitioning.

[0144] The coding blocks generated by applying multi-tree partitioning to a coding block can be called downstream coding blocks. When the partitioning depth of a coding block is k, the partitioning depths of multiple downstream coding blocks are set to k + 1.

[0145] On the other hand, for a coding block with a partitioning depth of k + 1, the coding block with a partitioning depth of k can be called an upstream coding block.

[0146] The partitioning type of a current coding block can be determined based on at least one of the partitioning type of an upstream coding block or the partitioning type of an adjacent coding block. Herein, the adjacent coding block is adjacent to the current coding block, and it can include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the current coding block. The partitioning type can include at least one of whether to partition a quadtree, whether to partition a binary tree, the binary tree partitioning direction, whether to partition a ternary tree, or the ternary tree partitioning direction.

[0147] To determine the partitioning type of a coding block, information indicating whether the coding block is partitioned can be signaled through a bitstream. The information is a 1-bit flag "split_cu_flag", and the flag being true indicates that the coding block is partitioned by a multi-tree partitioning technique.

[0148] When "split_cu_flag" is true, information indicating whether the coding block is partitioned by a quadtree can be signaled through a bitstream. The information is a 1-bit flag "split_qt_flag", and when the flag is true, the coding block can be partitioned into 4 blocks.

[0149] For example, in Figure 5 the illustrated example shows that when a coding tree unit is partitioned by a quadtree, 4 coding blocks with a partitioning depth of 1 are generated. Additionally, it is illustrated that the first coding block and the fourth coding block among the 4 coding blocks generated as a result of the quadtree partitioning are again applied with quadtree partitioning. Finally, 4 coding blocks with a partitioning depth of 2 can be generated.

[0150] Furthermore, coding blocks with a partitioning depth of 3 can be generated by reapplying quadtree partitioning to the coding blocks with a partitioning depth of 2.

[0151] When a quadtree partitioning is not applied to a coding block, whether to perform a binary tree partitioning or a ternary tree partitioning on the coding block can be determined by considering at least one of the size of the coding block, whether the coding block is located at an image boundary, the maximum partitioning depth, or the partitioning type of an adjacent block. When it is determined to perform a binary tree partitioning or a ternary tree partitioning on the coding block, information indicating the partitioning direction can be signaled through a bitstream. The information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partitioning direction can be determined to be the vertical direction or the horizontal direction based on the flag. Additionally, information indicating which one of a binary tree partitioning or a ternary tree partitioning is applied to the coding block can be signaled through a bitstream. The information can be a 1-bit flag "mtt_split_cu_binary_flag". Whether to apply a binary tree partitioning or a ternary tree partitioning to the coding block can be determined based on the flag.

[0152] For example, in Figure 5In the illustrated example, vertical binary tree partitioning is applied to an encoded block with a partitioning depth of 1, vertical ternary tree partitioning is applied to the left encoded block among the encoded blocks generated as the partitioning result, and vertical binary tree partitioning is applied to the right encoded block.

[0153] Inter-frame prediction refers to predicting the prediction coding mode of the current block using the information of the previous image. For example, a block at the same position as the current block in the previous image (hereinafter referred to as a collocated block) can be set as the prediction block of the current block. Hereinafter, a prediction block generated based on a block at the same position as the current block is referred to as a collocated prediction block.

[0154] On the other hand, if an object existing in the previous image has moved to another position in the current image, the movement of the object can be used to effectively predict the current block. For example, if it is possible to know the movement direction and size of the object by comparing the previous image and the current image, the movement information of the object can be considered to generate a prediction block (or a prediction image) of the current block. Hereinafter, a prediction block generated using movement information can be referred to as a motion prediction block.

[0155] A residual block can be generated by subtracting the prediction block from the current block. In this case, when there is movement of an object, by using a motion prediction block instead of a collocated prediction block, the energy of the residual block can be reduced, and thus the compression performance of the residual block can be improved.

[0156] As described above, the process of generating a prediction block using movement information can be referred to as motion compensation prediction. In most inter-frame predictions, a prediction block can be generated based on motion compensation prediction.

[0157] The movement information can include at least one of a motion vector, a reference image index, a prediction direction, or a bi-directional weighting value index. The motion vector represents the movement direction and size of the object. The reference image index specifies the reference image of the current block among the multiple reference images included in the reference image list. The prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bi-directional prediction (L0 prediction and L1 prediction). At least one of the motion information in the L0 direction or the motion information in the L1 direction can be used according to the prediction direction of the current block. The bi-directional weighting value index specifies the weighting value applied to the L0 prediction block and the weighting value applied to the L1 prediction block.

[0158] Figure 6 It is a flowchart of the inter-frame prediction method according to an embodiment of the present invention.

[0159] Refer to Figure 6, the inter-frame prediction method includes the following steps: determining an inter-frame prediction mode of a current block (S601); obtaining motion information of the current block according to the determined inter-frame prediction mode (S602); and performing motion compensation prediction on the current block based on the obtained motion information (S603).

[0160] Among them, the inter-frame prediction mode represents various techniques for determining the motion information of the current block, and may include an inter-frame prediction mode using translational motion information and an inter-frame prediction mode using affine motion information. For example, the inter-frame prediction mode using translational motion information may include a merge mode and an advanced motion vector prediction mode, and the inter-frame prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. According to the inter-frame prediction mode, the motion information of the current block can be determined based on adjacent blocks adjacent to the current block or information parsed from the bitstream.

[0161] Hereinafter, the inter-frame prediction method using affine motion information will be described in detail.

[0162] Figure 7 is a diagram illustrating the non-linear motion of an example object.

[0163] The motion of an object within a video may be non-linear motion. For example, as Figure 7 shown in the example, non-linear motion of an object such as camera zoom-in, zoom-out, rotation, or affine transformation may occur. When non-linear motion of an object occurs, the motion of the object cannot be effectively represented by a translational motion vector. Therefore, in a portion where non-linear motion of an object occurs, affine motion can be used instead of translational motion, thereby improving the coding efficiency.

[0164] Figure 8 is a flowchart showing an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0165] It can be determined whether to apply an inter-frame prediction technique based on affine motion to the current block based on information parsed from the bitstream. Specifically, it can be determined whether to apply an inter-frame prediction technique based on affine motion to the current block based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0166] When applying the inter-frame prediction technique based on affine motion to a current block, an affine motion model of the current block can be determined (S801). The affine motion model can be determined as at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent affine motion, and the 4-parameter affine motion model uses 4 parameters to represent affine motion.

[0167] Equation 1 is the case of representing affine motion using 6 parameters. The affine motion represents a translational motion with respect to a predetermined region determined by an affine seed vector.

[0168]

Equation 1

[0169] v x = ax - by + e

[0170] v y = cx + dy + f

[0171] When representing affine motion using 6 parameters, complex motion can be represented, but the number of bits required for encoding each parameter increases, thus reducing the encoding efficiency. Therefore, affine motion can also be represented using 4 parameters. Equation 2 is the case of representing affine motion using 4 parameters.

[0172]

Equation 2

[0173] v x = ax - by + e

[0174] v y = bx + ay + f

[0175] The information used to determine the affine motion model of the current block can be encoded and signaled through a bitstream. For example, the information can be a 1-bit flag "affine_type_flag". The value of the flag being 0 indicates the application of the 4-parameter affine motion model, and the value of the flag being 1 indicates the application of the 6-parameter affine motion model. The flag can be encoded in units of a slice, a tile, or a block (e.g., a coded block or a coding tree unit). When signaling the flag at the slice level, the affine motion model determined at the slice level can be applied to all blocks belonging to the slice.

[0176] Alternatively, the affine motion model of the current block may be determined based on the affine inter prediction mode of the current block. For example, when the affine merge mode is applied, the affine motion model of the current block may be determined as a 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, the information for determining the affine motion model of the current block may be encoded and signaled through the bitstream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block may be determined based on a 1-bit flag "affine_type_flag".

[0177] Next, the affine seed vectors of the current block may be derived (s802). When a 4-parameter affine motion model is selected, the motion vectors at two control points of the current block may be derived. On the other hand, when a 6-parameter affine motion model is selected, the motion vectors at three control points of the current block may be derived. The motion vectors at the control points may be referred to as affine seed vectors. The control points may include at least one of the upper left corner, upper right corner, or lower left corner of the current block.

[0178] Figure 9 is a diagram showing examples of affine seed vectors for each affine motion model.

[0179] In the 4-parameter affine motion model, the affine seed vectors related to two of the upper left corner, upper right corner, or lower left corner may be derived. For example, as shown in the example of part (a) of Figure 9 , when a 4-parameter affine motion model is selected, the affine vector may be derived by using the affine seed vector sv0 related to the upper left corner of the current block (e.g., the upper left sample (x0, y0)) and the affine seed vector sv1 related to the upper right corner of the current block (e.g., the upper right sample (x1, y1)). The affine seed vector related to the lower left corner may also be used instead of the affine seed vector related to the upper left corner, or the affine seed vector related to the lower left corner may also be used instead of the affine seed vector related to the upper right corner.

[0180] In the 6-parameter affine motion model, the affine seed vectors related to the upper left corner, upper right corner, and lower left corner may be derived. For example, as shown in the example of part (b) of Figure 9 , when a 6-parameter affine motion model is selected, the affine vector may be derived by using the affine seed vector sv0 related to the upper left corner of the current block (e.g., the upper left sample (x0, y0)), the affine seed vector sv1 related to the upper right corner of the current block (e.g., the upper right sample (x1, y1)), and the affine seed vector sv2 related to the lower left corner of the current block (e.g., the lower left sample (x2, y2)).

[0181] In the embodiments described later, under the 4-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are respectively referred to as the first affine seed vector and the second affine seed vector. In the embodiments using the first affine seed vector and the second affine seed vector described later, at least one of the first affine seed vector and the second affine seed vector can be replaced with the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0182] In addition, under the 6-parameter affine motion model, the affine seed vectors of the upper left control point, the upper right control point, and the lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiments using the first affine seed vector, the second affine seed vector, and the third affine seed vector described later, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with the affine seed vector of the lower right control point (the fourth affine seed vector).

[0183] The affine seed vector can be used to derive the affine vector for different sub-blocks (S803). Among them, the affine vector represents the translational motion vector derived based on the affine seed vector. The affine vector of the sub-block can be referred to as the affine sub-block motion vector or the sub-block motion vector.

[0184] Figure 10 It is a diagram showing an example of the affine vector of a sub-block under the 4-parameter motion model.

[0185] The affine vector of the sub-block can be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine sub-block vector.

[0186]

Equation 3

[0187]

[0188]

[0189] In Equation 3, (x, y) represents the position of the sub-block. Among them, the position of the sub-block represents the position of the reference sample included in the sub-block. The reference sample can be the sample located at the upper left corner of the sub-block, or the sample whose at least one of the x-axis or y-axis coordinates is in the central position. (x0, y0) represents the position of the first control point, and (sv 0x , sv 0y ) represents the first affine seed vector. In addition, (x1, y1) represents the position of the second control point, and (sv 1x , sv 1y ) represents the second affine seed vector.

[0190] When the first control point and the second control point correspond to the upper left corner and the upper right corner of the current block respectively, x1 - x0 can be set to a value equal to the width of the current block.

[0191] After that, motion compensation prediction can be performed on each sub - block using the affine vectors of the respective sub - blocks (S804). After performing the motion compensation prediction, prediction blocks related to the respective sub - blocks can be generated. The prediction block of the sub - block can be set as the prediction block of the current block.

[0192] The affine seed vector of the current block can be derived based on the affine seed vectors of the neighboring blocks adjacent to the current block. When the inter - frame prediction mode of the current block is the affine merge mode, the affine seed vector of the merge candidate included in the merge candidate list can be determined as the affine seed vector of the current block. Additionally, when the inter - frame prediction mode of the current block is the affine merge mode, the motion information including at least one of the reference image index of the current block, the specific direction prediction flag, or the bi - directional weighting value can be set to be the same as that of the merge candidate.

[0193] The merge candidates can be derived based on the neighboring blocks of the current block. The neighboring blocks can include at least one of the spatially neighboring blocks that are spatially adjacent to the current block and the temporally neighboring blocks included in an image different from the current image.

[0194] Figure 11 FIG. is a diagram showing the neighboring blocks that can be used to derive the merge candidates.

[0195] The neighboring blocks of the current block can include at least one of the neighboring block (A) adjacent to the left side of the current block, the neighboring block (B) adjacent to the upper side of the current block, the neighboring block (C) adjacent to the upper right corner of the current block, the neighboring block (D) adjacent to the lower left corner of the current block, or the neighboring block adjacent to the upper left corner of the current block. If the coordinates of the upper - left sample of the current block are (x0, y0), the left neighboring block A includes the sample at the position (x0 - 1, y0 + H - 1), and the upper neighboring block B includes the sample at the position (x0+W - 1, y0 - 1). Here, W and H represent the width and height of the current block respectively. The upper - right neighboring block C includes the sample at the position (x0 + W, y0 - 1), and the lower - left neighboring block D includes the sample at the position (x0 - 1, y0 + H). The upper - left neighboring block E includes the sample at the position (x0 - 1, y0 - 1).

[0196] When encoding the neighboring block in the affine inter - frame prediction mode, the affine seed vector of the merge candidate can be derived based on the affine seed vector of the corresponding neighboring block. Hereinafter, the neighboring block encoded in the affine inter - frame prediction mode is referred to as an affine neighboring block, and the merge candidate derived from the affine neighboring block is referred to as an affine merge candidate.

[0197] Affine merge candidates related to the current block can be generated by searching for adjacent blocks in a predefined scan order. The scan order can be predefined in both the encoder and the decoder. For example, adjacent blocks can be searched in the order of A, B, C, D, E. Additionally, merge candidates can be derived sequentially from the searched affine adjacent blocks. Alternatively, the scan order can be adaptively determined based on at least one of the size, shape, or affine motion model of the current block. That is, the scan order for blocks with at least one of different sizes, shapes, or affine motion models will be different.

[0198] Alternatively, multiple blocks located above the current block are searched sequentially, and an affine merge candidate is derived from the first found affine adjacent block, and multiple blocks located to the left of the current block are searched sequentially, and an affine merge candidate is derived from the first found affine adjacent block. Among them, the multiple adjacent blocks located above the current block include at least one of adjacent block E, adjacent block B, or adjacent block C, and the multiple blocks located to the left of the current block can include at least one of block A or block D. In this case, adjacent block E can also be classified as a block located to the left of the current block.

[0199] Although not illustrated, affine merge candidates can also be derived from the temporal adjacent blocks of the current block. Among them, the temporal adjacent blocks can include blocks located at the same position as the current block in the co-located picture or blocks adjacent thereto. Specifically, if the temporal adjacent blocks of the current block are encoded in the affine inter prediction mode, the affine merge candidates can be derived based on the affine seed vectors of the temporal affine merge candidates.

[0200] A merge candidate list including affine merge candidates can be generated, and one of the affine seed vectors of the merge candidates included in the merge candidate list can be determined as the affine seed vector of the current block. For this purpose, index information identifying any one of the multiple merge candidates can be encoded and transmitted through the bitstream.

[0201] As another example, multiple adjacent blocks can be searched in the scan order, and the affine seed vector of the current block can be derived from the affine seed vector of the first found affine adjacent block.

[0202] As described above, in the affine merge mode, the affine seed vector of the current block can be derived using the affine seed vectors of adjacent blocks.

[0203] When the inter prediction mode of the current block is the affine motion vector prediction mode, the affine seed vector of the motion vector prediction candidate included in the motion vector prediction candidate list can be determined as the affine seed vector prediction value of the current block. The affine seed vector of the current block can be derived by adding the affine seed vector difference to the affine seed vector prediction value.

[0204] An affine seed vector prediction candidate can be derived based on neighboring blocks of a current block. Specifically, a plurality of neighboring blocks located above the current block are searched in a predetermined scan order, and a first affine seed vector prediction candidate can be derived from the first found affine neighboring block. Additionally, a plurality of neighboring blocks located to the left of the current block are searched in a predetermined scan order, and a second affine seed vector prediction candidate can be derived from the first found affine neighboring block.

[0205] Information for determining an affine seed vector difference can be encoded and transmitted through a bitstream. The information can include size information representing the size of the affine seed vector difference and a sign signal representing the sign of the affine seed vector difference. The affine seed vector differences associated with respective control points can be set to be the same. Alternatively, the affine seed vector differences can be set differently for each control point.

[0206] As described above, an affine seed vector of an affine merge candidate or an affine seed vector prediction candidate can be derived from the affine seed vector of an affine neighboring block, and the derived affine seed vector of the affine merge candidate or the affine seed vector prediction candidate can be used to derive the affine seed vector of the current block. Alternatively, after searching for a plurality of affine neighboring blocks in a predetermined scan order, the affine seed vector of the current block can be derived from the affine seed vector of the first found affine neighboring block.

[0207] Hereinafter, a method for deriving the affine seed vector of the current block, an affine merge candidate, or an affine seed vector prediction candidate from the affine seed vector of an affine neighboring block will be described in detail. In the embodiments described later, deriving the affine seed vector of the current block can be understood as deriving the affine seed vector of the affine merge candidate, or can also be understood as deriving the affine seed vector of the affine seed vector prediction candidate.

[0208] Figure 12 FIG. is a diagram showing derivation of the affine seed vector of the current block based on the affine seed vector of an affine neighboring block.

[0209] If the affine neighboring block stores a first affine seed vector nv0 associated with the upper left control point and a second affine seed vector nv1 associated with the upper right control point, a third affine seed vector nv2 associated with the lower left control point of the affine neighboring block can be derived based on the first affine seed vector and the second affine seed vector. Equation 4 shows an example of deriving the third affine seed vector.

[0210]

Equation 4

[0211]

[0212]

[0213] In Equation 4, (nv0x , nv 0y ) represents the first affine seed vector nv0, (nv 1x , nv 1y ) represents the second affine seed vector nv1, and (nv 2x , nv 2y ) represents the third affine seed vector nv2. Additionally, (x n0 , x n0 ) represents the position of the first control point, (x n1 , x n1 ) represents the position of the second control point, and (x n2 , x n2 ) represents the position of the third control point.

[0214] After that, the first affine seed vector, the second affine seed vector, and the third affine seed vector can be used to derive the affine seed vector of the current block. Equation 5 shows an example of deriving the first affine seed vector v0 of the current block, and Equation 6 shows an example of deriving the second affine seed vector v1 of the current block.

[0215]

Equation 5

[0216]

[0217]

[0218]

Equation 6

[0219]

[0220]

[0221] In Equation 5 and Equation 6, (v 0x , v 0y ) represents the first affine seed vector sv0 of the current block, and (v 1x , v 1y ) represents the second affine seed vector sv1 of the current block. Additionally, (x0, y0) represents the position of the first control point, and (x1, y1) represents the position of the second control point. For example, the first control point represents the upper left corner of the current block, and the second control point represents the upper right corner of the current block.

[0222] The example illustrates using 3 affine seed vectors related to affine adjacent blocks to derive multiple affine seed vectors of the current block. As another example, it is also possible to derive the affine seed vector of the current block using only 2 of the multiple affine seed vectors of the affine adjacent blocks.

[0223] Alternatively, instead of using the first affine seed vector at the upper left corner, the second affine seed vector at the upper right corner, or the third affine seed vector at the lower left corner related to the affine adjacent block, a fourth affine seed vector related to the lower right corner can be used to derive multiple affine seed vectors of the current block.

[0224] In particular, when the upper boundary of the current block touches the upper boundary of the coding tree unit and the affine seed vectors of the upper control points (e.g., the upper left corner or the upper right corner) of the affine adjacent block (hereinafter referred to as the upper affine adjacent block) above the current block need to be used, they need to be pre-stored in the memory in advance, which may cause a problem of an increase in the number of line buffers. Therefore, when the upper boundary of the current block touches the upper boundary of the coding tree unit, it can be set to use the affine seed vectors of the lower control points (e.g., the lower left corner or the lower right corner) for the upper affine adjacent block instead of using the affine seed vectors of the upper control points. For example, the third affine seed vector related to the lower left corner of the upper affine adjacent block and the fourth affine seed vector related to the lower right corner can be used to derive multiple affine seed vectors of the current block. In this case, the affine seed vectors related to the lower corner can be derived by copying the affine seed vectors related to the upper corner, or can be derived from the affine seed vectors related to multiple upper corners. For example, the first affine seed vector, the second affine seed vector, or the third affine seed vector can be transformed / replaced with the fourth affine seed vector related to the lower right corner.

[0225] Equations 7 and 8 show examples of using the third affine seed vector related to the lower left control point of the adjacent affine vector and the fourth affine seed vector related to the lower right control point to derive the first affine seed vector and the second affine seed vector of the current block.

[0226]

Equation 7

[0227]

[0228]

[0229]

Equation 8

[0230]

[0231]

[0232] In Equations 7 and 8, (x n2 , y n2 ) represents the coordinates of the lower left control point of the affine adjacent block, and (x n3 , y n3) represents the coordinates of the lower - right control point of the affine adjacent block. (x0, y0) represents the coordinates of the upper - left control point of the current block, and (x1, y1) represents the coordinates of the upper - right control point of the current block. (nv 2x , nv 2y ) represents the affine seed vector of the lower - left control point of the affine adjacent block (i.e., the third affine seed vector), and (nv 3x , nv 3y ) represents the affine seed vector of the lower - right control point of the affine adjacent block (i.e., the fourth affine seed vector). (v 0x , v 0y ) represents the affine seed vector of the upper - left control point of the current block (i.e., the first affine seed vector), and (v 1x , v 1y ) represents the affine seed vector of the upper - right control point of the current block (i.e., the second affine seed vector).

[0233] It is also possible to change the division operations included in Equation 7 and Equation 8 to shift operations. The shift operations can be performed based on the value derived from the width between the lower - left control point and the lower - right control point (i.e., (x n3 - x n2 ).

[0234] In the above example, multiple affine seed vectors of the current block can be derived based on multiple affine seed vectors of the encoded / decoded affine adjacent block. For this purpose, it is necessary to store multiple affine seed vectors of the encoded / decoded affine adjacent block in the memory. However, since in addition to multiple translational motion vectors (i.e., multiple affine vectors) of multiple sub - blocks included in the affine adjacent block, multiple affine seed vectors of the affine adjacent block are also stored in the memory, there is a problem of increased memory usage. To eliminate this problem, the motion vectors of the sub - blocks adjacent to the control points of the affine adjacent block can be used to derive the affine seed vectors of the current block, thereby replacing the affine seed vectors of the affine adjacent block. That is, the motion vectors of the sub - blocks adjacent to the control points of the affine adjacent block can be set as the affine seed vectors of the affine adjacent block. Among them, the sub - block is a block with a size / shape predefined in the encoder and decoder, and can be a block with a basic size / shape for storing motion vectors. For example, the sub - block can be a square block with a size of 4×4. Alternatively, the motion vector specifying the sample position can be set as the affine seed vector of the affine adjacent block.

[0235] Figure 13 is a diagram showing an example of setting the motion vector of a sub - block as the affine seed vector of an affine adjacent block.

[0236] The motion vector of the sub - block adjacent to the control point can be set as the affine seed vector of the corresponding control point. For example, in Figure 13In the example shown, the motion vector (nv 4x , nv 4y ) of the sub-block (lower left sub-block) adjacent to the lower left corner of the affine adjacent block can be set to the affine seed vector (nv 2x , nv 2y ) of the lower left control point, and the motion vector (nv 5x , nv 5y ) of the sub-block (lower right sub-block) adjacent to the lower right corner can be set to the affine seed vector (nv 3x , nv 3y ) of the control point at the lower right corner. Among them, the lower left sub-block refers to a sub-block including samples adjacent to the lower left control point (x n2 , y n2 ) of the adjacent affine block (for example, samples at the position of (x n2 , y n2 - 1)), and the lower right sub-block refers to a block including samples adjacent to the lower right control point (x n3 , y n3 ) of the adjacent affine block (for example, samples at the position of (x n3 - 1, y n3 - 1)). When deriving the affine seed vector of the current block based on Equation 7 and Equation 8, the third affine seed vector of the affine adjacent block can be replaced by the motion vector of the lower left sub-block, and the fourth affine seed vector can be replaced by the motion vector of the lower right sub-block.

[0237] Hereinafter, in the embodiments described later, the sub-block used as the affine seed vector of the affine adjacent block is referred to as an affine sub-block.

[0238] According to an embodiment of the present invention, an affine sub-block can be determined based on samples at a specific position. For example, a sub-block including samples at a specific position can also be set as the affine sub-block. Hereinafter, the samples at the specific position are referred to as affine reference samples. In addition, the reference samples for determining the affine sub-block of the lower left control point are referred to as lower left reference samples, and the reference samples for determining the affine sub-block of the lower right control point are referred to as lower right reference samples.

[0239] The lower left reference sample and the lower right reference sample can be selected from the multiple samples included in the affine adjacent block. For example, at least one of the upper left sample, lower left sample, upper right sample, or lower left sample of the lower left sub-block can be set as the lower left reference sample, and at least one of the upper left sample, lower left sample, upper right sample, or lower left sample of the lower right sub-block can be set as the lower right reference sample. Therefore, the motion vectors of the lower left sub-block including the lower left reference sample and the lower right sub-block including the lower right reference sample can be set as the affine seed vector related to the lower left control point and the affine seed vector related to the lower right control point, respectively.

[0240] As another example, at least one of the lower left reference sample or the lower right reference sample may be set as a sample located outside the affine adjacent block. In this regard, the reference Figures 14 to 16 will be described in detail.

[0241] Figures 14 to 16 is a diagram showing the positions of the reference samples.

[0242] As Figure 14 shown in the (a) part of, for the lower left control point, the upper left sample of the lower left sub-block may be set as the reference sample (x n4 , y n4 ). Therefore, the lower left sub-block including the reference sample (x n4 , y n4 ) may be set as the affine sub-block related to the lower left control point.

[0243] For the lower right control point, the sample to the right of the upper right sample of the lower right sub-block may be set as the reference sample (x n5 , y n5 ). Therefore, the sub-block adjacent to the right of the lower right sub-block including the reference sample (x n5 , y n5 ) may be set as the affine sub-block related to the lower right control point.

[0244] Alternatively, as Figure 14 shown in the (b) part of, for the lower left control point, the sample to the left of the upper left sample of the lower left sub-block may be set as the reference sample (x n4 , y n4 ). Therefore, the sub-block adjacent to the left of the lower left sub-block including the reference sample (x n4 , y n4 ) may be set as the affine sub-block related to the lower left control point.

[0245] For the lower right control point, the upper right sample of the lower right sub-block may be set as the reference sample (x n5 , y n5 ). Therefore, the lower right sub-block including the reference sample (x n5 , y n5 ) may be set as the affine sub-block related to the lower right control point.

[0246] Alternatively, as Figure 15 shown in the (a) part of, for the lower left control point, the lower left sample of the lower left sub-block may be set as the reference sample (x n4 , y n4 ). Therefore, the lower left sub-block including the reference sample (x n4 , y n4 ) may be set as the affine sub-block related to the lower left control point.

[0247] For the lower right control point, the sample to the right of the lower right sample in the lower right sub-block can be set as the reference sample (x n5 , y n5 ). Therefore, the sub-block adjacent to the right of the lower right sub-block including the reference sample (x n5 , y n5 ) can be set as the affine sub-block related to the lower right control point.

[0248] Alternatively, as shown in the example of part (b) of Figure 15 , for the lower left control point, the sample to the left of the lower left sample in the lower left sub-block can be set as the reference sample (x n4 , y n4 ). Therefore, the sub-block adjacent to the left of the lower left sub-block including the reference sample (x n4 , y n4 ) can be set as the affine sub-block related to the lower left control point.

[0249] For the lower right control point, the lower right sample in the lower right sub-block can be set as the reference sample (x n5 , y n5 ). Therefore, the lower right sub-block including the reference sample (x n5 , y n5 ) can be set as the affine sub-block related to the lower right control point.

[0250] Alternatively, as shown in the example of part (a) of Figure 16 , for the lower left control point, the sample between the upper left sample and the lower left sample in the lower left sub-block (e.g., the middle sample on the left) can be set as the reference sample (x n4 , y n4 ). Therefore, the lower left sub-block including the reference sample (x n4 , y n4 ) can be set as the affine sub-block related to the lower left control point.

[0251] For the lower right control point, the sample to the right of the sample between the upper right sample and the lower right sample in the lower right sub-block (e.g., the middle sample on the right) can be set as the reference sample (x n5 , y n5 ). Therefore, the sub-block adjacent to the right of the lower right sub-block including the reference sample (x n5 , y n5 ) can be set as the affine sub-block related to the lower right control point.

[0252] Alternatively, as shown in Figure 16For the example shown in part (b), for the lower left control point, the sample to the left of the samples between the upper left sample and the lower left sample in the lower left sub-block can be set as the reference sample (x n4 , y n4 ). Therefore, the sub-block adjacent to the left of the lower left sub-block including the reference sample (x n4 , y n4 ) can be set as the affine sub-block related to the lower left control point.

[0253] For the lower right control point, the sample between the upper right sample and the lower right sample in the lower right sub-block can be set as the reference sample (x n5 , y n5 ). Therefore, the lower right sub-block including the reference sample (x n5 , y n5 ) can be set as the affine sub-block related to the lower right control point.

[0254] In the case of deriving multiple affine seed vectors of the current block based on Equation 7 and Equation 8, the third affine seed vector of the affine adjacent block can be replaced by the motion vector of the affine sub-block related to the lower left control point, and the fourth affine seed vector can be replaced by the motion vector of the affine sub-block related to the lower right control point. Additionally, the position of the lower left control point can be replaced by the position of the lower left reference sample, and the position of the lower right control point can be replaced by the position of the lower right reference sample.

[0255] Different from Figures 14 to 16 the content described above, a sub-block including a sample adjacent to the reference sample can also be set as the affine sub-block. Specifically, a sample outside the affine adjacent sub-block can be set as the reference sample, and the sub-block included in the affine adjacent block can be set as the affine sub-block. For example, in Figure 14 the example shown in part (a), the sample to the right of the upper right sample in the lower right sub-block can be set as the reference sample (x n5 , y n5 ), and the lower right sub-block can be set as the affine sub-block related to the lower right corner. Alternatively, in Figure 14 the example shown in part (b), the sample to the left of the upper left sample in the lower left sub-block can be set as the reference sample (x n4 , y n4 ), and the lower left sub-block can be set as the affine sub-block related to the lower left corner.

[0256] The embodiments described in Figure 15 and Figure 16 can be equivalently applied. That is, in Figure 15 part (a) or Figure 16In the example shown in part (a), the bottom - right sample of the bottom - right sub - block or the sample to the right of the middle sample on the right can be set as the reference sample (x n5 , y n5 ), and the bottom - right sub - block can be set as the affine sub - block related to the bottom - right corner. Alternatively, in the example shown in part (b) of Figure 15 or Figure 16 , the bottom - left sample of the bottom - left sub - block or the sample to the left of the middle sample on the left can be set as the reference sample (x n4 , y n4 ), and the bottom - left sub - block can be set as the affine sub - block related to the bottom - left corner.

[0257] In the above example, the affine seed vector of the affine adjacent block can be derived by using the motion vector of the affine sub - block. For this purpose, for the encoded / decoded block, the motion vector can be stored in units of sub - blocks.

[0258] As another example, after storing the minimum number of affine seed vectors in the affine adjacent block, the motion vector of the affine sub - block can be derived by using the stored multiple affine seed vectors.

[0259] Equations 9 and 10 represent examples of using the affine seed vectors of the affine adjacent block to derive the motion vector of the affine sub - block.

[0260]

Equation 9

[0261]

[0262]

[0263]

Equation 10

[0264]

[0265]

[0266] In Equations 9 and 10, (nv 4x , nv 4y ) represents the motion vector of the affine sub - block related to the bottom - left control point, and (nv 5x , nv 5y ) represents the motion vector of the affine sub - block related to the bottom - right control point. Since the motion vector of the affine sub - block is set to be the same as the affine seed vector of the control point, (nv 4x , nv 4y ) can be replaced by the affine seed vector (nv 2x , nv 2y ) related to the bottom - left control point, or (nv 5x , nv5y ) can be replaced by the affine seed vector (nv 3x , nv 3y ) related to the lower-right control point.

[0267] (x n4 , y n4 ) represents the position of the reference sample of the lower-left sub-block. Alternatively, instead of this position, the center position of the lower-left sub-block or the position of the lower-left control point can also be used. (x n5 , y n5 ) represents the position of the reference sample of the lower-right sub-block. Alternatively, instead of this position, the center position of the lower-right sub-block or the position of the lower-right control point can also be used.

[0268] Equations 9 and 10 can be applied when the current block does not touch the boundary of the coding tree unit. When the current block touches the upper boundary of the coding tree unit, instead of using Equations 9 and 10, the translational motion vector of the affine sub-block determined based on the lower-left reference sample can be set to the third affine seed vector, and the translational motion vector of the affine sub-block determined based on the lower-right reference sample can be set to the fourth affine seed vector.

[0269] In Equations 7 and 8, (x n3 - x n2 ) represents the width between the lower-left control point and the lower-right control point. As described above, x n3 can be replaced by the position x n5 of the lower-right reference sample, and x n2 can be replaced by the position x n4 of the lower-left reference sample. Hereinafter, (x n3 - x n2 ) or the value obtained by replacing the positions in the foregoing equations with the positions of the reference samples (for example, (x n5 - x n4 )) is defined as the variable W seed , and the variable is called the sub-seed vector width.

[0270] According to the positions of the reference samples, the sub-seed vector width may not be a power series of 2 (for example, 2 n) For example, when setting the lower left sample of the lower left sub-block as the lower left reference sample and the lower right sample of the lower right sub-block as the lower right reference sample, the width of the sub-seed vector is not a multiple of 2. As described above, when the width of the sub-seed vector is not a power series of 2, the width of the sub-seed vector can be transformed into a power series of 2. The transformation may include adding / subtracting an offset to the width of the sub-seed vector, or using the position of a sample adjacent to the reference sample instead of the position of the reference sample. For example, the transformed width of the sub-seed vector can be derived by adding 1 to the width between the lower left reference sample and the lower right reference sample. Alternatively, the width between the adjacent reference sample adjacent to the right side of the lower right reference sample and the lower left reference sample can be set as the transformed width of the sub-seed vector. After that, the affine seed vector of the current block can be derived by substituting the transformed width of the sub-seed vector into Equation 7 and Equation 8.

[0271] The division operations included in Equation 7 and Equation 8 can also be changed to shift operations. The shift operations can be performed based on values derived from the transformed width of the sub-seed vector (i.e., a value expressed as a power series of 2).

[0272] When the reference sample used to determine the affine sub-block does not belong to the affine adjacent block, the affine seed vector of the affine adjacent block can be derived based on the sample adjacent to the reference sample among the multiple samples included in the affine adjacent block. Specifically, the translational motion vector of the sub-block including the sample adjacent to the reference sample in the affine adjacent block (hereinafter referred to as the adjacent reference sample) can be set as the affine seed vector of the affine adjacent block. As described above, the method of using the adjacent reference sample to derive the affine seed vector can be defined as a modified affine merge vector derivation method.

[0273] Figure 17 is a diagram showing an example of applying the modified affine merge vector derivation method.

[0274] When the lower right reference sample (x n5 , y n5 ) of the affine adjacent block E does not belong to the affine adjacent block, the affine seed vector can be derived based on the sample adjacent to the left side of the lower right reference sample among the samples included in the affine adjacent block (x n5 -1, y n5 ). Specifically, the translational motion vector of the sub-block including the adjacent reference sample (x n5 -1, y n5 ) can be set as the affine seed vector of the lower right control point.

[0275] In Figure 17In the illustrated example, the sample adjacent to the right side of the upper - right sample of the lower - right sub - block is shown as the lower - right reference sample. In the case where the sample adjacent to the right side of the lower - right sample of the lower - right sub - block or the sample adjacent to the right side of the middle - right sample of the lower - right sub - block is set as the lower - right reference sample, the affine seed vector can be derived based on the sample adjacent to the left side of the adjacent reference sample.

[0276] In addition, when the lower - left reference sample does not belong to the affine - adjacent block, the affine seed vector can also be derived based on the sample adjacent to the right side of the lower - left reference sample as described in the embodiments.

[0277] By setting the positions of the reference samples and the sub - blocks for deriving the affine seed vector in different ways, the sub - seed vector width can thus be set to a power of 2.

[0278] Alternatively, in the case where the motion vectors of the lower - left sub - block and the lower - right sub - block can be used, multiple affine seed vectors can be derived based on the width of the adjacent block.

[0279] Figure 18 FIG. is an example diagram showing the affine seed vectors for deriving the affine merge candidates based on the multiple motion vectors of the multiple sub - blocks belonging to the adjacent block.

[0280] In the case where the upper boundary of the current block touches the boundary of the coding tree unit, the affine seed vectors for the affine merge candidates can be derived based on the motion vectors of the lower - left sub - block and the lower - right sub - block of the upper - adjacent block located above the current block. Assuming that the position of the upper - left sample of the upper - adjacent block is (xNb, yNb), the width and height of the upper - adjacent block are NbW and NbH respectively, the lower - left sub - block may include the sample at the position (xNb, yNb + yNbH - 1), and the lower - right sub - block may include the sample at the position (xNb+NbW - 1, yNb + yNbH - 1).

[0281] Multiple affine seed vectors for the affine merge candidates can be derived based on the width of the adjacent block and the coordinate difference between the adjacent block and the current block. For example, multiple affine seed vectors for the affine merge candidates can be derived based on the following equations 11 to 13.

[0282]

Equation 11

[0283] v 0x =V LBx <<7+((V RBx -V LBx )<<(7 - log2(NbW))*(xCb - xNb)

[0284] v 0y =V LBy <<7+((V RBy -V LBy) << (7 - log2(NbW)) * (xCb - xNb)

[0285]

Equation 12

[0286] v 1x = V LBx << 7 + ((V RBx - V LBx ) << (7 - log2(NbW)) * (xCb + xCbW - xNb)

[0287] v 1y = V LBy << 7 + ((V RBy - V LBy ) << (7 - log2(NbW)) * (xCb + xCbW - xNb)

[0288]

Equation 13

[0289] v 2x - V LBx << 7 + ((V RBx - V LBx ) << (7 - log2(NbW)) * (xCb + xCbW - xNb) + ((V LBy - V RBy ) << (7 - log2(NbW)) * xCbH

[0290] v 2y = V LBy << 7 + ((V RBy - V LBy ) << (7 - log2(NbW)) * (xCb + xCbW - xNb) + ((V LBx - V RBx ) << (7 - log2(NbW)) * xCbH

[0291] In the said Equations 11 to 13, (v 0x , v 0y ) represents the first affine seed vector, (v 1x , v 1y ) represents the second affine seed vector, and (v 2x , v 2y) represents the third affine seed vector. Vlbx represents the horizontal component motion vector of the lower left sub-block, and Vlby represents the vertical component motion vector of the lower left sub-block. Vrbx represents the horizontal component motion vector of the lower right sub-block, and Vrby represents the vertical component motion vector of the lower right sub-block. NbW represents the width of the adjacent block, and xCbW and xCbH represent the width and height of the current block respectively. xCb represents the x-coordinate of the upper left sample of the current block, and xNb represents the x-coordinate of the upper left sample of the adjacent block.

[0292] An affine merge candidate can also be generated by combining a plurality of affine seed vectors derived from the first adjacent block and the motion vectors of the second adjacent block. For example, in Figure 18 the example shown, instead of deriving the third affine seed vector for the affine merge candidate based on the motion vectors of the lower left sub-block and the lower right sub-block, the third affine seed vector for the affine merge candidate can also be derived based on the motion vectors of the sub-blocks located to the left of the current block.

[0293] Figure 19 FIG. is a diagram showing an example of an affine seed vector for deriving an affine merge candidate based on the motion vectors of a plurality of sub-blocks located to the left of the current block.

[0294] Based on the motion vectors of the lower left sub-block and the lower right sub-block of the upper adjacent block located above the current block, an affine seed vector for the affine merge candidate can be derived. Specifically, the first affine seed vector and the second affine seed vector for the affine merge candidate can be derived based on the motion vectors of the lower left sub-block and the lower right sub-block. For example, the first affine seed vector and the second affine seed vector for the affine merge candidate can be derived based on Equation 11 and Equation 12.

[0295] The third affine seed vector for the affine merge candidate can be derived based on the motion vectors of the adjacent blocks located to the left of the current block. For example, the motion vector of the adjacent block adjacent to the left side of the current block or the adjacent block adjacent to the lower left corner of the current block can be set as the third affine seed vector. The adjacent block adjacent to the left side of the current block may include the sample at the position (xCb - 1, yCb + CbH - 1), and the adjacent block adjacent to the lower left corner of the current block may include the sample at the position (xCb - 1, yCb + CbH). Equation 14 and Equation 15 show examples of setting the motion vector of the adjacent block located to the left of the current block as the third affine seed vector.

[0296]

Equation 14

[0297] v 2x =V Leftx

[0298] v 2y =V Lefty

[0299]

Equation 15

[0300] v 2x = V LBx

[0301] v 2y = V LBy

[0302] In Equation 14, V Leftx represents the horizontal motion vector of the neighboring block adjacent to the left side of the current block, and V Lefty represents the vertical motion vector of the neighboring block adjacent to the left side of the current block. In Equation 15, V LBx represents the horizontal motion vector of the neighboring block adjacent to the lower left corner of the current block, and V LBy represents the vertical motion vector of the neighboring block adjacent to the lower left corner of the current block.

[0303] The motion vector of the neighboring block adjacent to the lower left corner of the current block can be used to derive the third affine seed vector only when the motion vector of the neighboring block adjacent to the left side of the current block is unavailable.

[0304] The method of deriving the third affine seed vector for the affine merge candidate can also be set differently according to whether the motion vector of the neighboring block located on the left side of the current block is available. For example, when the motion vector of the neighboring block located on the left side of the current block (e.g., the sub-block adjacent to the left side of the current block or the sub-block adjacent to the lower left corner of the current block) is available, Equation 14 or Equation 15 can be used to derive the third affine seed vector. On the other hand, when the motion vector of the neighboring block located on the left side of the current block is unavailable, Equation 13 can be used to derive the third affine seed vector.

[0305] The third affine seed vector can also be derived based on the motion vectors of non-neighboring blocks where the reference sample is not adjacent to the current block. To derive the third affine seed vector, it is possible to determine whether to use the motion vector of the neighboring block where the reference sample is adjacent to the current block or the motion vector of the non-neighboring block where the reference sample is not adjacent to the current block based on the position of the upper neighboring block used when deriving the affine merge candidate.

[0306] Figure 20 is a diagram showing an example of deriving the affine seed vector for the affine merge candidate based on the motion information of non-neighboring blocks or neighboring blocks located on the left side of the current block.

[0307] The position of the block for deriving the third affine seed vector can be determined by comparing the position of the upper left sample of the upper neighboring block used for deriving the affine merge candidate and the position of the upper left sample of the current block. For example, as Figure 20In the example shown in part (a), when the x - coordinate (xNb) of the upper - left sample of the upper adjacent block is less than the x - coordinate (xCb) of the upper - left sample of the current block, the third affine seed vector can be derived based on the motion vector of a non - adjacent block that is not adjacent to the left side of the current block. Specifically, the third affine seed vector can be derived based on the motion vector of the left non - adjacent block including the sample (xNb, yCb + CbH - 1) having the same x - coordinate as the upper - left sample of the upper adjacent block or the lower - left non - adjacent block including the sample at the position (xNb, yCb + CbH). Equations 16 and 17 show examples of deriving the third affine seed vector based on the motion vector of the non - adjacent block.

[0308]

Equation 16

[0309] v 2x =V Left2x

[0310] v 2y =V Left2y

[0311]

Equation 17

[0312] v 2x =V LB2x

[0313] v 2y =V LB2y

[0314] In Equation 16, V Left2x represents the horizontal - direction motion vector of the left non - adjacent block, and V Left2y represents the vertical - direction motion vector of the left non - adjacent block. In Equation 17, V LB2x represents the horizontal - direction motion vector of the lower - left non - adjacent block, and V LB2y represents the vertical - direction motion vector of the lower - left non - adjacent block.

[0315] The motion vector of the lower - left non - adjacent block can be used to derive the third affine seed vector only when the motion vector of the left non - adjacent block is not available.

[0316] As Figure 20In the example shown in part (b), when the x - coordinate (xNb) of the upper - left sample of the upper - adjacent block is the same as the x - coordinate (xCb) of the upper - left sample of the current block or the x - coordinate (xNb) of the upper - left sample of the upper - adjacent block is greater than the x - coordinate (xCb) of the upper - left sample of the current block, the third affine seed vector can be derived based on the motion vector of the adjacent block adjacent to the left side or the lower - left corner of the current block. Specifically, the third affine seed vector can be derived based on the motion vector of the left - adjacent block including the sample (xCb, yCb + CbH - 1) adjacent to the left side of the current block or the lower - left - adjacent block including the sample at the position of the sample (xCb, yCb + CbH) adjacent to the lower - left corner of the current block.

[0317] It is also possible to set the position of the reference sample of the left - adjacent block or the non - adjacent block on the left side in a different way as Figures 18 to 20 shown. For example, the block including the sample (xCb - 1, yCb + subH - 1) adjacent to the current block can be set as the left - adjacent block, and the block including the sample (xNb, yCb + subH - 1) not adjacent to the current block can be set as the non - adjacent block on the left side. Here, subH represents the minimum height of the block storing motion information, which can be an integer such as 2, 4, or 8.

[0318] Figure 21 is a diagram showing the position of the block for deriving the affine seed vector for the affine merge candidate.

[0319] For example, in the example shown in part (a) of Figure 21 , when the x - coordinate (xNb) of the upper - left sample of the upper - adjacent block is less than the x - coordinate (xCb) of the upper - left sample of the current block, the third affine seed vector can be derived based on the motion vector of the non - adjacent block not adjacent to the left side of the current block. Specifically, the third affine seed vector can be derived based on the motion vector of the non - adjacent block on the left side having the same x - coordinate as the upper - left sample of the upper - adjacent block and including the sample (xNb, yCb + subH - 1) separated from the lower - left sample of the upper - adjacent block by a predetermined distance (for example, subH). Equation 18 shows an example of deriving the third affine seed vector based on the motion vector of the non - adjacent block.

[0320]

Equation 18

[0321] v 2x = V LeftT2x

[0322] v 2y = V LeftT2y

[0323] In Equation 18, V LeftT2x represents the horizontal - direction motion vector of the non - adjacent block on the left side including the sample separated from the lower - left sample of the upper - adjacent block by a predetermined distance. VLeftT2y Represents a vertical motion vector of a left non - adjacent block including a sample separated by a predetermined distance from the lower - left sample of the upper - adjacent block.

[0324] Alternatively, as Figure 21 shown in part (b) of, in the case where the x - coordinate (xNb) of the upper - left sample of the upper - adjacent block is the same as the x - coordinate (xCb) of the upper - left sample of the current block or the x - coordinate (xNb) of the upper - left sample of the upper - adjacent block is greater than the x - coordinate (xCb) of the upper - left sample of the current block, a third affine seed vector can be derived based on the motion vector of the left - adjacent block adjacent to the left side of the current block and including a sample separated by a predetermined distance (e.g., subH - 1) in the vertical direction from the upper - left sample of the current block (xCb - 1, yCb+subH - 1). Equation 19 represents an example of deriving a third affine seed vector based on the motion vector of an adjacent block.

[0325]

Equation 19

[0326] v 2x =V LeftTx

[0327] v 2y =V LeftTy

[0328] In Equation 19, V LeftTx represents the horizontal motion vector of the left - adjacent block adjacent to the left side of the current block and including a sample separated by a predetermined distance in the vertical direction from the upper - left sample of the current block. V LeftTy represents the vertical motion vector of the left - adjacent block adjacent to the left side of the current block and including a sample separated by a predetermined distance in the vertical direction from the upper - left sample of the current block.

[0329] Alternatively, a reference sample can be set at a position different from the position shown in Figures 18 to 21 . For example, a sample (xCb - 1, yCb) adjacent to the left side of the current block and having the same y - coordinate as the upper - left sample of the current block or a sample (xNb, yCb) not adjacent to the left side of the current block and having the same y - coordinate as the upper - left sample of the current block can be set as the reference sample.

[0330] As described in the example, the first and second affine seed vectors of the affine merge candidate can be derived from the adjacent block above the current block, and the third affine seed vector of the affine merge candidate can be derived from the adjacent block to the left of the current block. The first and third affine seed vectors of the affine merge candidate can be derived from the adjacent block to the left of the current block, and the second affine seed vector of the affine merge candidate can be derived from the adjacent block above the current block.

[0331] As shown in Equations 11 to 13, an affine seed vector of an affine merge candidate can be derived based on the difference between the motion vector V of the lower left sub-block of the upper adjacent block LB and the motion vector V of the lower right sub-block. RB In this case, instead of the motion vector of the lower left sub-block or the motion vector of the lower right sub-block, the motion vector of other blocks can be used to derive the affine seed vector of the affine merge candidate. For example, for the motion vector V of the lower left sub-block of the upper adjacent block LB and the motion vector V of the lower right sub-block RB with the same value, the motion vector of a block including samples located to the right or below the lower right sub-block can be used to replace V. RB . For example, instead of the motion vector V of the lower right sub-block RB , the motion vector of a block including samples at the position (xNb + NbW, yNb + NbH - 1) can be used to derive the motion vector of the affine merge candidate.

[0332] Alternatively, for the motion vector V of the lower left sub-block of the upper adjacent block LB and the motion vector V of the lower right sub-block RB with the same value, the motion vector of a block including samples located to the left or below the lower left sub-block can be used to replace V. LB . For example, instead of the motion vector V of the lower left sub-block LB , the motion vector of a block including samples at the position (xNb, yNb + NbH - 1 + offset) can be used to derive the motion vector of the affine merge candidate. The offset can be an integer greater than or equal to 1.

[0333] Alternatively, the motion vector of a block including samples located to the left or below the lower left sub-block can be used to replace V RB , or the motion vector of a block including samples located to the right or below the lower right sub-block can be used to replace V LB .

[0334] A merge candidate can also be derived by combining multiple motion vectors of multiple adjacent blocks adjacent to the current block. A merge candidate generated by combining multiple motion vectors of multiple adjacent blocks can be referred to as a combined merge candidate. When deriving a combined merge candidate, an affine seed vector of a control point can be derived from one of the multiple adjacent blocks adjacent to the control point.

[0335] Figure 22 FIG. is a diagram for explaining an example of deriving a combined merge candidate by combining multiple motion vectors of multiple adjacent blocks.

[0336] The adjacent blocks may include an upper adjacent block located above the current block and a left adjacent block located to the left of the current block. The upper adjacent block may be at least one of the following blocks, i.e., block B0 including the sample at the position (xCb+CbW, yCb-1), block B1 including the sample at the position (xCb+CbW-1, yCb-1), block B2 including the sample at the position (xCb-1, yCb-1), or block B3 including the sample at the position (xCb, yCb-1). Herein, (xCb, yCb) represents the position of the upper left sample of the current block, and CbW represents the width of the current block. The left adjacent block may be at least one of the following blocks, i.e., block A0 including the sample at the position (xCb-1, yCb+CbH), block A1 including the sample at the position (xCb-1, yCb+CbH-1), or block A2 including the sample at the position (xCb-1, yCb). Herein, CbH represents the height of the current block.

[0337] In addition, the adjacent blocks may further include the temporal adjacent blocks of the current block.

[0338] The combined merge candidate may be derived by combining the motion information of two or more adjacent blocks among the adjacent blocks. The combined merge candidate may be derived by combining the motion information of two or three adjacent blocks. Specifically, the combined merge candidate may also be derived by combining two or more of the motion information of the adjacent blocks adjacent to the first control point, the motion information of the adjacent blocks adjacent to the second control point, or the motion information of the adjacent blocks adjacent to the third control point.

[0339] For example, the combined merge candidate may be derived by combining the motion vectors of two or more adjacent blocks among block A2, B2, or B3 adjacent to the upper left control point CP0, block B1 or B0 adjacent to the upper right control point CP1, or the temporal adjacent block T related to the lower left control point CP2.

[0340] In this case, when scanning multiple adjacent blocks adjacent to the control point in a predefined scan order, the motion vector of the first available adjacent block found may be set as the affine seed vector of the control point. For example, the first affine seed vector of the combined merge candidate related to CP0 may be set as the motion vector of the first available adjacent block found when searching multiple adjacent blocks in the order of B2, B3, and A2. The second affine seed vector of the combined merge candidate related to CP1 may be set as the motion vector of the first available adjacent block found when searching multiple adjacent blocks in the order of B1 and B0. The third affine seed vector of the combined merge candidate related to CP2 may be set as the motion vector of the first available adjacent block found when searching multiple adjacent blocks in the order of A1 and A0.

[0341] When exporting combined merge candidates, adjacent blocks encoded with an affine motion model or adjacent blocks derived from affine merge candidates can be set as unavailable.

[0342] Figure 23 FIG. is a diagram showing unavailable adjacent blocks.

[0343] When deriving an affine merge candidate from an adjacent block B1 located above the current block, it can be determined that the adjacent block B1 is unavailable for deriving combined merge candidates. Therefore, the motion vector of B1 may not be used when deriving the affine seed vector of the combined merge candidate related to CP1.

[0344] Alternatively, when encoding an adjacent block with an affine motion model, a combined merge candidate can be derived based on the affine vectors of sub-blocks including reference samples. For example, if an adjacent block including samples at the B1 position is encoded with an affine motion model, the affine vectors of sub-blocks including samples at the B1 position of the adjacent block can be used to derive a combined merge candidate.

[0345] Alternatively, the scanning order can be determined by considering whether an adjacent block is encoded with an affine motion model or whether an adjacent block is derived from an affine merge candidate. For example, the scanning order of adjacent blocks encoded with an affine motion model or derived from affine merge candidates can be set to the last. For example, when encoding an adjacent block at the B1 position with an affine motion model, the second affine seed vector of the combined merge candidate related to CP2 can be derived by searching for multiple adjacent blocks in the order of B0 and B1.

[0346] When the reference image indices among multiple adjacent blocks are different, the motion vector can be scaled based on the reference image with the smallest index or the reference image with the largest index. The scaling can be performed based on the difference in the output order (POC, Picture Order Count) between the reference image and the current image.

[0347] Alternatively, only adjacent blocks with the same reference image index among multiple adjacent blocks can be used to perform the combination of multiple adjacent blocks. For example, when at least one of the reference image indices among multiple adjacent blocks is different, the combination can be set as unavailable for combined merge candidates. Additionally, the availability of the combination can be determined regardless of whether the motion information or motion vectors of the adjacent blocks are the same or not. For example, when the motion vectors of multiple adjacent blocks (e.g., CP0 affine seed vector and CP1 affine seed vector) are the same, a combined merge candidate can also be derived by combining multiple adjacent blocks with the same reference image index. Alternatively, the scanning order of multiple adjacent blocks can be determined by considering the reference image indices of multiple adjacent blocks. Alternatively, only multiple adjacent blocks with the same prediction direction among multiple adjacent blocks can be used to perform the combination.

[0348] Merge candidates can be generated according to the set combination order. For example, the combination order can be set as follows.

[0349] 1. {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector}

[0350] 2. {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector}

[0351] 3. {CP0 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0352] 4. {CP1 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0353] 5. {CP0 affine seed vector, CP1 affine seed vector}

[0354] 6. {CP0 affine seed vector, CP2 affine seed vector}

[0355] Although six combination examples are shown, combination examples with fewer or more than this number can also be used.

[0356] The combination merge candidates generated by combining three affine seed vectors can be called six-parameter set combination merge candidates, and the combination merge candidates generated by combining two affine seed vectors can be called four-parameter set combination merge candidates.

[0357] The combination order can be predefined in the encoder and decoder. Alternatively, the combination order can be determined based on at least one of the size, shape, partition type, affine motion model, position of the current block within the coding tree unit, or output order of the reference image of the current block. For example, when applying a four-parameter affine motion model to the current block, multiple combination examples for four-parameter set combination merge candidates can be set to have a higher priority than multiple combination examples for six-parameter set combination merge candidates.

[0358] Combination merge candidates can be generated according to the combination order, but it can be determined that only the merge candidates generated by combining multiple adjacent blocks with the same reference image are available. Alternatively, when at least one of the reference images in multiple adjacent blocks is different, the motion vectors can be scaled based on the reference image with the largest index or the smallest index to derive the merge candidates. The scaling can be performed based on the output order difference between the current image and the reference image. Alternatively, when two of the reference images in multiple adjacent blocks are the same but another reference image is different, the motion vector of another adjacent block can be scaled based on the reference image commonly applied to the two adjacent blocks to derive the combination merge candidates.

[0359] The number of combined merge candidates added to the merge candidate list can be determined based on at least one of the number of merge candidates already included in the merge candidate list or the maximum number of merge candidates. Alternatively, according to the affine motion model of the current block, only 6-parameter set merge candidates can be added to the merge candidate list, or only 4-parameter set merge candidates can be added to the combined merge candidate list.

[0360] For example, when the number of combined merge candidates to be added to the merge candidate list is 1 and a 6-parameter motion model is applied to the current block, one 6-parameter set merge candidate based on the set combination order can be added to the merge candidate list. Specifically, when the CP0 affine seed vector, CP1 affine seed vector, and CP2 affine seed vector are all available, the combined merge candidate {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector} can be added to the merge candidate list. Alternatively, when the CP2 affine seed vector is unavailable and the CP3 affine seed vector is available, the combined merge candidate {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector} can be added to the merge candidate list.

[0361] Alternatively, when the number of combined merge candidates to be added to the merge candidate list is 1 and a 6-parameter motion model is applied to the current block, one 4-parameter set merge candidate based on the set combination order can be added to the merge candidate list.

[0362] As another example, according to the affine motion model of the current block, the number of multiple combined merge candidates to be added to the merge candidate list can be set differently. For example, when a 6-parameter motion model is applied to the current block, 1 combined merge candidate can be added to the merge candidate list. On the other hand, when a 4-parameter motion model is applied to the current block, 2 combined merge candidates can be added to the merge candidate list.

[0363] According to the number of available affine merge candidates, the combined merge candidates to be added to the merge candidate list can be selected. For example, when the number of available affine merge candidates is more than 2, only 4-parameter set merge candidates can be added to the merge candidate list. On the other hand, when the number of available affine merge candidates is 1 or less, only 6-parameter set merge candidates can be added to the merge candidate list. Alternatively, when the number of available affine merge candidates is 1 or less, N 6-parameter set merge candidates and M 4-parameter set merge candidates can be added to the merge candidate list. Here, N and M are integers, and M can be derived based on the value obtained by subtracting N from the maximum number of merge candidates. Alternatively, when the number of available affine merge candidates is 1 or less, only 4-parameter set merge candidates can be added to the merge candidate list.

[0364] Alternatively, the combination order can be determined based on the availability of multiple affine seed vectors. For example, the availability of multiple affine seed vectors can be considered to add combination merge candidates to the merge candidate list according to the following order.

[0365] 1. The case where the CP0 affine seed vector, CP1 affine seed vector, and CP2 affine seed vector are available

[0366] {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector}

[0367] 2. The case where the CP0 affine seed vector, CP1 affine seed vector, and CP3 affine seed vector are available

[0368] {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector}

[0369] 3. The case where the CP0 affine seed vector, CP2 affine seed vector, and CP3 affine seed vector are available

[0370] {CP0 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0371] 4. The case where the CP0 affine seed vector, CP2 affine seed vector, and CP3 affine seed vector are available

[0372] {CP1 affine seed vector, CP2 affine seed vector, CP3 affine seed vector}

[0373] 5. The case where the CP0 affine seed vector and CP1 affine seed vector are available

[0374] {CP0 affine seed vector, CP1 affine seed vector}

[0375] 6. The case where the CP0 affine seed vector and CP2 affine seed vector are available

[0376] {CP0 affine seed vector, CP2 affine seed vector}

[0377] When the number of combination merge candidates that can be added to the merge candidate list is 1, the combination merge candidate that satisfies the condition first among the conditions 1 to 6 can be added to the merge candidate list. In the case where the conditions 1 to 6 are not satisfied, the combination merge candidate may not be added to the merge candidate list.

[0378] As another example, the maximum number of combination merge candidates that can be added to the merge candidate list can also be determined according to the number of available affine merge candidates.

[0379] When the number of merge candidates included in the merge candidate list is less than the maximum number, a zero merge candidate with a motion vector of 0 can also be added to the merge candidate list. Therefore, in the affine merge mode, the merge candidates can be derived in the following order.

[0380] 1. Derive affine merge candidates

[0381] 2. Combine merge candidates

[0382] 3. Zero merge candidates

[0383] Next, the inter prediction method using translational motion information will be described in detail.

[0384] The motion information of the current block can be derived from the motion information of other blocks of the current block. Among them, the other blocks can be blocks that have been encoded / decoded by inter prediction before the current block. The case where the motion information of the current block is set to be the same as the motion information of the other blocks is defined as the merge mode. In addition, the case where the motion vector of the other blocks is set to the predicted value of the motion vector of the current block is defined as the motion vector prediction mode.

[0385] Figure 24 is a flowchart of the process of deriving the motion information of the current block in the merge mode.

[0386] The merge candidates of the current block can be derived (S2401). The merge candidates of the current block can be derived from the blocks that have been encoded / decoded by inter prediction before the current block.

[0387] The candidate blocks used to derive the merge candidates can include adjacent blocks, that is, samples adjacent to the current block. For example, if the coordinates of the upper left sample of the current block are (0, 0), then at least one of the blocks including the reference sample at the position (-1, H - 1), the block including the reference sample at the position (W - 1, -1), the block including the reference sample at the position (W, -1), the block including the reference sample at the position (-1, H), or the block including the reference sample at the position (-1, -1) can be used as a candidate block.

[0388] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block can be set as not available as merge candidates. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, the candidate block including the reference sample can be set as not available as a merge candidate.

[0389] Merge candidates can also be derived from temporally adjacent blocks included in an image different from the current block. For example, merge candidates can be derived from co-located blocks included in a co-located image. Any one of the multiple reference images included in the reference picture list can be set as the co-located image. Index information for identifying the co-located image among the multiple reference images can be signaled via a bitstream. Alternatively, a reference image with a predefined index among the multiple reference images can be determined as the co-located image.

[0390] The motion information of the merge candidate can be set to be the same as the motion information of the candidate block. For example, at least one of the motion vector, reference picture index, prediction direction, or bi-predictive weight value index of the candidate block can be set as the motion information of the merge candidate.

[0391] A merge candidate list including the merge candidates can be generated (S2402).

[0392] Indices of the multiple merge candidates within the merge candidate list can be assigned in a predetermined order. For example, indices can be assigned in the order of merge candidates derived from the left adjacent block, the upper adjacent block, the upper-right adjacent block, the lower-left adjacent block, the upper-left adjacent block, and the temporally adjacent block.

[0393] When the merge candidate includes multiple merge candidates, at least one of the multiple merge candidates can be selected (S2403). Specifically, information for specifying any one of the multiple merge candidates can be signaled via a bitstream. For example, information merge_idx representing the index of any one of the multiple merge candidates included in the merge candidate list can be signaled via a bitstream.

[0394] When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter-frame motion information list can be added to the merge candidate list. Here, the threshold can be the value of the maximum number of merge candidates that the merge candidate list can include or the value of the maximum number of merge candidates minus an offset. The offset can be an integer such as 1 or 2. The inter-frame motion information list can include merge candidates derived based on blocks encoded / decoded before the current block.

[0395] The inter-frame motion information list includes merge candidates derived from blocks encoded / decoded based on inter-frame prediction in the current image. For example, the motion information of the merge candidates included in the inter-frame motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on inter-frame prediction. Here, the motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-predictive weight value index.

[0396] For ease of explanation, the merge candidates included in the inter - frame motion information list are referred to as inter - frame merge candidates.

[0397] The maximum number of merge candidates that can be included in the inter - frame motion information list can be predefined in the encoder and decoder. For example, the maximum number of merge candidates that can be included in the inter - frame motion information list can be 1, 2, 3, 4, 5, 6, 7, 8 or greater (e.g., 16).

[0398] Alternatively, information representing the maximum number of merge candidates of the inter - frame motion information list can be signaled via the bitstream. The information is signaled at the sequence level, picture level or slice level.

[0399] Alternatively, the maximum number of merge candidates of the inter - frame motion information list can be determined according to the size of the picture, the size of the slice or the size of the coding tree unit.

[0400] The inter - frame motion information list can be initialized on a picture, slice, tile, brick, coding tree unit or coding tree unit row (row or column) basis. For example, when initializing a slice, the inter - frame motion information list is also initialized, and the inter - frame motion information list may not include any merge candidates.

[0401] Alternatively, information indicating whether to initialize the inter - frame motion information list can also be signaled via the bitstream. The information can be signaled at the slice level, tile level, brick level or block level. Before the information indicates that the inter - frame motion information list is initialized, the configured inter - frame motion information list can be used.

[0402] Alternatively, information related to the inter - frame merge candidates can be signaled via the picture parameter set or slice header. Even if a slice is initialized, the inter - frame motion information list can include initial inter - frame merge candidates. Thus, the inter - frame merge candidates can be used for the blocks that are first encoded / decoded within the slice.

[0403] According to the encoding / decoding order, blocks are encoded / decoded, and multiple blocks encoded / decoded based on inter - frame prediction can be sequentially set as inter - frame merge candidates according to the encoding / decoding order.

[0404] Figure 25 is a diagram for explaining an example of updating the inter - frame motion information list.

[0405] When performing inter - frame prediction on the current block (S2501), inter - frame merge candidates can be derived based on the current block (S2502). The motion information of the inter - frame merge candidates can be set to be the same as the motion information of the current block.

[0406] When the inter-frame motion information list is empty (S2503), the inter-frame merge candidate derived based on the current block can be added to the inter-frame motion information list (S2504).

[0407] When the inter-frame motion information list already includes the inter-frame merge candidate (S2503), a redundancy check can be performed on the motion information of the current block (or the inter-frame merge candidate derived based on the current block) (S2505). The redundancy check is used to determine whether the motion information of the inter-frame merge candidate already stored in the inter-frame motion information list is the same as the motion information of the current block. All the inter-frame merge candidates already stored in the inter-frame motion information list can be used as objects to perform the redundancy check. Alternatively, the inter-frame merge candidates with indices above or below a threshold among the inter-frame merge candidates already stored in the inter-frame motion information list can be used as objects to perform the redundancy check.

[0408] When there is no inter-frame merge candidate having the same motion information as the motion information of the current block, the inter-frame merge candidate derived based on the current block can be added to the inter-frame motion information list (S2508). It can be determined whether the inter-frame merge candidates are the same based on whether the motion information of the inter-frame merge candidates (e.g., motion vector and / or reference image index, etc.) is the same.

[0409] At this time, when the maximum number of inter-frame merge candidates has been stored in the inter-frame motion information list (S2506), the earliest inter-frame merge candidate is deleted (S2507), and the inter-frame merge candidate derived based on the current block can be added to the inter-frame motion information list (S2508).

[0410] Multiple inter-frame merge candidates can be identified according to their respective indices. When adding the inter-frame merge candidate derived from the current block to the inter-frame motion information list, the lowest index (e.g., 0) is assigned to the inter-frame merge candidate, and the indices of the already stored inter-frame merge candidates are each incremented by 1. In this case, when the maximum number of inter-frame merge candidates has been stored in the inter-frame motion information list, the inter-frame merge candidate with the largest index is removed.

[0411] Alternatively, when adding the inter-frame merge candidate derived from the current block to the inter-frame motion information list, the largest index can be assigned to the inter-frame merge candidate. For example, when the number of inter-frame merge candidates already stored in the inter-frame motion information list is less than the maximum value, an index having the same value as the number of already stored inter-frame merge candidates can be assigned to the inter-frame merge candidate. Alternatively, when the number of inter-frame merge candidates already stored in the inter-frame motion information list is equal to the maximum value, an index obtained by subtracting 1 from the maximum value can be assigned to the inter-frame merge candidate. In addition, the inter-frame merge candidate with the smallest index is removed, and the indices of the remaining multiple already stored inter-frame merge candidates are each decremented by 1.

[0412] Figure 26 It is a diagram showing an embodiment of updating an inter-frame merge candidate list.

[0413] Suppose that an inter-frame merge candidate derived from the current block is added to the inter-frame merge candidate list, and the maximum index is assigned to the inter-frame merge candidate. Additionally, assume that the maximum number of inter-frame merge candidates has been stored in the inter-frame merge candidate list.

[0414] When adding the inter-frame merge candidate HmvpCand[n + 1] derived from the current block to the inter-frame merge candidate list HmvpCandList, the inter-frame merge candidate HmvpCand[0] with the smallest index is deleted from the stored inter-frame merge candidates, and the indices of the remaining multiple inter-frame merge candidates are each decreased by 1. Additionally, the index of the inter-frame merge candidate HmvpCand[n + 1] derived from the current block can be set to the maximum value (n in the example shown). Figure 26 as shown in the example).

[0415] When a stored inter-frame merge candidate identical to the inter-frame merge candidate derived based on the current block exists (S2505), the inter-frame merge candidate derived based on the current block may not be added to the inter-frame motion information list (S2509).

[0416] Alternatively, as the inter-frame merge candidate derived based on the current block is added to the inter-frame motion information list, the stored inter-frame merge candidate identical to the inter-frame merge candidate can also be removed. In this case, the effect of updating the indices of the stored inter-frame merge candidates will occur.

[0417] Figure 27 It is a diagram showing an example of updating the index of a previously stored inter-frame merge candidate.

[0418] When the index of the stored inter-frame merge candidate identical to the inter-frame merge candidate mvCand derived based on the current block is hIdx, deleting the stored inter-frame merge candidate, the indices of the inter-frame merge candidates with indices greater than hIdx can each be decreased by 1. For example, in Figure 27 the example shown, it is illustrated that HmvpCand[2] identical to mvCand is deleted from the inter-frame motion information list HvmpCandList, and the indices of HmvpCand[3] to HmvpCand[n] are each decreased by 1.

[0419] Additionally, the inter-frame merge candidate mvCand derived based on the current block can be added to the end of the inter-frame motion information list.

[0420] Alternatively, the index of a stored inter - merge candidate that is the same as an inter - merge candidate derived based on the current block may be updated. For example, the index of the stored inter - merge candidate may be changed to the minimum value or the maximum value.

[0421] The motion information of a block included in a predetermined region may be set such that it cannot be added to the inter - frame motion information list. For example, an inter - merge candidate derived based on the motion information of a block included in the merge - processing region cannot be added to the inter - frame motion information list. Since the coding / decoding order of the blocks included in the merge - processing region is not defined, it is not appropriate to use the motion information of any of these blocks for inter - frame prediction of other blocks. Therefore, an inter - merge candidate derived based on the blocks included in the merge - processing region may not be added to the inter - frame motion information list.

[0422] When performing motion - compensated prediction based on sub - blocks, an inter - merge candidate may be derived based on the motion information of a representative sub - block among the multiple sub - blocks included in the current block. For example, when using a sub - block merge candidate for the current block, an inter - merge candidate may be derived based on the motion information of the representative sub - block in the sub - blocks.

[0423] The motion vector of a sub - block may be derived in the following order. First, any one of the merge candidates included in the merge - candidate list of the current block may be selected, and an initial shift vector (shVector) may be derived based on the motion vector of the selected merge candidate. Additionally, by adding the initial shift vector to the position (xSb, ySb) of the reference sample (e.g., the top - left sample or the middle - position sample) of each sub - block of the coded block, the position of the shifted sub - block with the reference - sample position (xColSb, yColSb) can be derived. Equation 20 below represents the equation for deriving the shifted sub - block.

[0424]

Equation 20

[0425] (xColSb, yColSb)=(xSb + shVector[0]>>4, ySb + shVector[1]>>4)

[0426] Next, the motion vector of the co - located block corresponding to the center position of the sub - block including (xColSb, yColSb) is set as the motion vector of the sub - block including (xSb, ySb).

[0427] The representative sub - block may mean a sub - block including the top - left sample or the center sample of the current block.

[0428] Figure 28 is a diagram showing the position of the representative sub - block.

[0429] Figure 28(a) shows an example in which a sub-block located in the upper left of the current block is set as a representative sub-block, and Figure 28 (b) shows an example in which a sub-block located at the center of the current block is set as a representative sub-block. When performing motion compensation prediction in units of sub-blocks, based on the motion vector of the sub-block including the upper left sample of the current block or the sub-block including the center sample of the current block, the inter-frame merge candidate of the current block can be derived.

[0430] It is also possible to determine whether to use the current block as an inter-frame merge candidate based on the inter-frame prediction mode of the current block. For example, a block encoded / decoded based on an affine motion model can be set as not available as an inter-frame merge candidate. Therefore, even if the current block is encoded / decoded by inter-frame prediction, when the inter-frame prediction mode of the current block is the affine prediction mode, the inter-frame prediction motion information list is not updated based on the current block.

[0431] Alternatively, the inter-frame merge candidate can also be derived based on at least one sub-block vector in the sub-blocks included in the block encoded / decoded based on the affine motion model. For example, the sub-block located in the upper left of the current block, the sub-block located at the center, or the sub-block located in the upper right can be used to derive the inter-frame merge candidate. Alternatively, the average value of the sub-block vectors of multiple sub-blocks can also be set as the motion vector of the inter-frame merge candidate.

[0432] Alternatively, the inter-frame merge candidate can also be derived based on the average value of the affine seed vectors of the block encoded / decoded based on the affine motion model. For example, the average value of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block can be set as the motion vector of the inter-frame merge candidate.

[0433] Alternatively, the inter-frame motion information list can be configured for different inter-frame prediction modes. For example, at least one of the inter-frame motion information list for the block encoded / decoded by intra-block copy, the inter-frame motion information list for the block encoded / decoded based on the translational motion model, or the inter-frame motion information list for the block encoded / decoded based on the affine motion model can be defined. According to the inter-frame prediction mode of the current block, any one of the multiple inter-frame motion information lists can be selected.

[0434] Figure 29 An example of generating the inter-frame motion information list for different inter-frame prediction modes is shown.

[0435] When encoding / decoding a block based on a non-affine motion model, an inter-frame merge candidate mvCand derived from the block can be added to the inter-frame non-affine motion information list HmvpCandList. On the other hand, when encoding / decoding a block based on an affine motion model, an inter-frame merge candidate mvAfCand derived from the block can be added to the inter-frame affine motion information list HmvpAfCandList.

[0436] The affine seed vector of a block can be stored in an inter-frame merge candidate derived from a block encoded / decoded based on an affine motion model. Therefore, the inter-frame merge candidate can be used as a merge candidate for deriving the affine seed vector of the current block.

[0437] In addition to the described inter-frame motion information lists, other inter-frame motion information lists can be defined. In addition to the described inter-frame motion information lists (hereinafter referred to as the first inter-frame motion information list), a long-term motion information list (hereinafter referred to as the second inter-frame motion information list) can be defined. Among them, the long-term motion information list includes long-term merge candidates.

[0438] When both the first inter-frame motion information list and the second inter-frame motion information list are in an empty state, first, an inter-frame merge candidate can be added to the second inter-frame motion information list. After the number of available inter-frame merge candidates in the second inter-frame motion information list reaches the maximum number, an inter-frame merge candidate can be added to the first inter-frame motion information list.

[0439] Alternatively, an inter-frame merge candidate can be added to both the second inter-frame motion information list and the first inter-frame motion information list.

[0440] In this case, the second inter-frame motion information list that has been configured may no longer be updated. Alternatively, when the decoding area is above a predetermined ratio of a strip, the second inter-frame motion information list can be updated. Alternatively, the second inter-frame motion information list can be updated every N coding tree unit rows.

[0441] On the other hand, the first inter-frame motion information list can be updated whenever a block encoded / decoded by inter-frame prediction is generated. However, the inter-frame merge candidate added to the second inter-frame motion information list can also be set not to be used for updating the first inter-frame motion information list.

[0442] Information for selecting either the first inter-frame motion information list or the second inter-frame motion information list can be signaled through a bitstream. When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter-frame motion information list indicated by the information can be added to the merge candidate list.

[0443] Alternatively, the inter-frame motion information list may also be selected based on the size, shape, inter-frame prediction mode, bi-directional prediction or not, motion vector refinement or not, or triangle partitioning or not of the current block.

[0444] Alternatively, if the number of merge candidates included in the merge candidate list is still less than the maximum merge number even after adding the inter-frame merge candidates included in the first inter-frame motion information list, the inter-frame merge candidates included in the second inter-frame motion information list may be added to the merge candidate list.

[0445] Figure 30 FIG. is an example showing adding the inter-frame merge candidates included in the long-term motion information list to the merge candidate list.

[0446] When the number of merge candidates included in the merge candidate list is less than the maximum number, the inter-frame merge candidates included in the first inter-frame motion information list HmvpCandList may be added to the merge candidate list. If the number of merge candidates included in the merge candidate list is still less than the maximum number even after adding the inter-frame merge candidates included in the first inter-frame motion information list to the merge candidate list, the inter-frame merge candidates included in the long-term motion information list HmvpLTCandList may be added to the merge candidate list.

[0447] Table 1 shows the process of adding the inter-frame merge candidates included in the long-term motion information list to the merge candidate list.

[0448]

Table 1

[0449]

[0450]

[0451] The inter-frame merge candidate may be set to include additional information in addition to the motion information. For example, the size, shape, or block partitioning information of the storage block may be added to the inter-frame merge candidate. When constructing the merge candidate list of the current block, only the inter-frame merge candidates with the same or similar size, shape, or partitioning information as the current block are used in the inter-frame merge candidate, or the inter-frame merge candidates with the same or similar size, shape, or partitioning information as the current block may be preferentially added to the merge candidate list.

[0452] Alternatively, the inter-frame motion information list may be generated for different block sizes, shapes, or partitioning information. The merge candidate list of the current block may be generated using the inter-frame motion information list corresponding to the shape, size, or partitioning information of the current block in the multiple inter-frame motion information lists.

[0453] When the number of merge candidates included in the merge candidate list of the current block is less than the threshold, the inter-frame merge candidates included in the inter-frame motion information list can be added to the merge candidate list. The addition process is performed in ascending or descending order of the indices. For example, the inter-frame merge candidate with the largest index can be added to the merge candidate list.

[0454] When adding the inter-frame merge candidates included in the inter-frame motion information list to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidates and the multiple merge candidates already stored in the merge candidate list.

[0455] For example, Table 2 shows the process of adding inter-frame merge candidates to the merge candidate list.

[0456]

Table 2

[0457]

[0458] Redundancy checks can also be performed only on some of the inter-frame merge candidates included in the inter-frame motion information list. For example, redundancy checks can be performed only on the inter-frame merge candidates with indices above or below the threshold. Alternatively, redundancy checks can be performed only on the N merge candidates with the largest indices or the N merge candidates with the smallest indices.

[0459] Alternatively, redundancy checks can be performed only on some of the merge candidates already stored in the merge candidate list. For example, redundancy checks can be performed only on the merge candidates with indices above or below the threshold or the merge candidates derived from blocks at specific positions. Among them, the specific positions can include at least one of the left adjacent block, the upper adjacent block, the upper right adjacent block, or the lower left adjacent block of the current block.

[0460] Figure 31 is a diagram showing an example of performing redundancy checks only on some of the merge candidates.

[0461] When adding the inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the two merge candidates with the largest indices mergeCandList[NumMerge - 2] and mergeCandList[NumMerge - 1]. Among them, NumMerge can represent the number of available spatial merge candidates and temporal merge candidates.

[0462] Unlike the example shown in the figure, when adding the inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can also be performed between the inter-frame merge candidate and the two merge candidates with the smallest indices. For example, it can be confirmed whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, the redundancy check can be performed only on the merge candidates derived from a specific position. For example, the redundancy check can be performed on at least one of the merge candidates derived from the adjacent block to the left of the current block or the adjacent block above the current block. When there is no merge candidate derived from a specific position in the merge candidate list, the inter-frame merge candidate can be added to the merge candidate list without performing a redundancy check.

[0463] In the case of finding a merge candidate that is the same as the first inter-frame merge candidate, when performing a redundancy check on the second inter-frame merge candidate, the redundancy check for the merge candidate that is the same as the first inter-frame merge candidate can be omitted.

[0464] Figure 32 It is a diagram showing an example of omitting a redundancy check for a specific merge candidate.

[0465] When adding the inter-frame merge candidate HmvpCand[i] with index i to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the merge candidates already stored in the merge candidate list. In this case, in the case of finding a merge candidate mergeCandList[j] that is the same as the inter-frame merge candidate HmvpCand[i], the inter-frame merge candidate HmvpCand[i] will not be added to the merge candidate list, and a redundancy check can be performed between the inter-frame merge candidate HmvpCand[i - 1] with index i - 1 and the merge candidate. In this case, the redundancy check between the inter-frame merge candidate HmvpCand[i - 1] and the merge candidate mergeCandList[j] can be omitted.

[0466] For example, in Figure 32 the example shown, it is determined that HmvpCand[i] is the same as mergeCandList[2]. Therefore, HmvpCand[i] is not added to the merge candidate list, and a redundancy check can be performed on HmvpCand[i - 1]. In this case, the redundancy check between HvmpCand[i - 1] and mergeCandList[2] can be omitted.

[0467] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold, in addition to inter-frame merge candidates, it may also include at least one of paired merge candidates or zero merge candidates. A paired merge candidate refers to a merge candidate that uses the average of the motion vectors of two or more merge candidates as the motion vector, and a zero merge candidate refers to a merge candidate with a motion vector of 0.

[0468] The merge candidate list of the current block can add merge candidates in the following order.

[0469] Spatial merge candidate - Temporal merge candidate - Inter-frame merge candidate - (Inter-frame affine merge candidate) - Paired merge candidate - Zero merge candidate

[0470] A spatial merge candidate refers to a merge candidate derived from at least one of adjacent blocks or non-adjacent blocks, and a temporal merge candidate refers to a merge candidate derived from a previous reference image. The inter-frame affine merge candidate list represents inter-frame merge candidates derived from blocks encoded / decoded using an affine motion model.

[0471] In the advanced motion vector prediction mode, an inter-frame motion information list can be used. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold, the inter-frame merge candidates included in the inter-frame motion information list are set as the motion vector prediction candidates related to the current block. Specifically, the motion vector of the inter-frame merge candidate is set as the motion vector prediction candidate.

[0472] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as the motion vector prediction value of the current block. After decoding the motion vector residual value of the current block, the motion vector of the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0473] The motion vector prediction candidate list of the current block can be constructed in the following order.

[0474] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Inter-frame decoded region merge candidate - (Inter-frame decoded region affine merge candidate) - Zero motion vector prediction candidate

[0475] A spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one of adjacent blocks or non-adjacent blocks, and a temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from a previous reference image. The inter-frame affine merge candidate list represents inter-frame motion vector prediction candidates derived from blocks encoded / decoded using an affine motion model. A zero motion vector prediction candidate represents a candidate with a motion vector value of 0.

[0476] A coding block can be divided into multiple prediction units, and prediction can be performed on the divided prediction units separately. Herein, a prediction unit represents a basic unit for prediction.

[0477] At least one of a vertical line, a horizontal line, a slant line, or a diagonal line can be used to divide a coding block. Information for determining at least one of the number, angle, or position of the lines for dividing the coding block can be signaled via a bitstream. For example, information representing any one of the division type candidates of the coding block can be signaled via the bitstream, or information specifying any one of multiple line candidates for dividing the coding block can be signaled via the bitstream. Alternatively, information for determining the number or type of the line candidates for dividing the coding block can be signaled via the bitstream. For example, using a 1-bit flag, it can be determined whether a slant line with an angle greater than the diagonal and / or a slant line with an angle less than the diagonal can be used as a line candidate.

[0478] Alternatively, based on at least one of the intra prediction mode, the inter prediction mode, the positions of merge candidates that can be used, or the division type of adjacent blocks of the coding block, at least one of the number, angle, or position of the lines for dividing the coding block can be adaptively determined.

[0479] If a coding block is divided into multiple prediction units, then intra prediction or inter prediction can be performed on each of the divided prediction units.

[0480] Figure 33 FIG. is an example showing using a diagonal line to divide a coding block into multiple prediction units.

[0481] As Figure 33 shown in the (a) part and (b) part of, a coding block can be divided into two triangular prediction units using a diagonal line.

[0482] In Figure 33 the (a) part and (b) part of, it is shown that a coding block is divided into two prediction units using a diagonal line connecting two vertices of the coding block. However, a coding block can be divided into two prediction units using a slant line where at least one end of the line does not pass through the vertex of the coding block.

[0483] Figure 34 FIG. is an example showing dividing a coding block into two prediction units.

[0484] As Figure 34 shown in the (a) part and (b) part of, a coding block can be divided into two prediction units using a slant line where both ends are in contact with the upper boundary and the lower boundary of the coding block respectively.

[0485] Alternatively, as Figure 34As shown in parts (c) and (d), a coding block can be divided into two prediction units by using a slant line that touches the left and right boundaries of the coding block at both ends.

[0486] Alternatively, the coding block can be divided into two prediction blocks of different sizes. For example, the coding block can be divided into two prediction units of different sizes by setting the slant line that divides the coding block to touch two side faces that form a vertex.

[0487] Figure 35 A diagram showing an example of dividing a coding block into multiple prediction blocks of different sizes.

[0488] As in Figure 35 parts (a) and Figure 35 part (b) shown, the coding block can be divided into two prediction units of different sizes by setting the diagonal line connected at the upper left or lower right corner of the coding block to pass through the left, right, upper, or lower boundary of the coding block, rather than through the upper left or lower right corner of the coding block.

[0489] Alternatively, as in Figure 35 parts (c) and Figure 35 part (d) shown, the coding block can be divided into two prediction units of different sizes by setting the diagonal line connected at the upper right or lower left corner of the coding block to pass through the left, right, upper, or lower boundary of the coding block, rather than through the upper left or lower right corner of the coding block.

[0490] Each prediction unit generated by dividing the coding block is called the "Nth prediction unit". For example, as in Figures 33 to 35 the example shown, PU1 can be defined as the first prediction unit, and PU2 can be defined as the second prediction unit. The first prediction unit can refer to a prediction unit that includes samples located in the lower left or upper left of the coding block, and the second prediction unit can refer to a prediction unit that includes samples located in the upper right or lower right of the coding block.

[0491] Conversely, a prediction unit that includes samples located in the upper right or lower right of the coding block can be defined as the first prediction unit, and a prediction unit that includes samples located in the lower left or upper left of the coding block can be defined as the second prediction unit.

[0492] The embodiments described later mainly illustrate examples of dividing using a diagonal line. In particular, the process of dividing a coding block into two prediction units using a diagonal line is called diagonal division or triangular division, and the prediction unit generated based on diagonal division is called a triangular prediction unit. However, of course, examples of dividing using an oblique line at an angle different from a vertical line, a horizontal line, or a diagonal line can also be used in the embodiments described later.

[0493] Whether to apply diagonal division to a coding block can be determined based on at least one of the slice type, the maximum number of merge candidates that the merge candidate list can include, the size of the coding block, the shape of the coding block, the predictive coding mode of the coding block, or the division type of the parent node.

[0494] For example, based on whether the current slice is of type B, it can be determined whether to apply diagonal division to the coding block. Diagonal division may be allowed only when the current slice is of type B.

[0495] Alternatively, it can be determined whether to apply diagonal division to a coding block based on whether the maximum number of merge candidates included in the merge candidate list is more than two. Diagonal division may be allowed only when the maximum number of merge candidates included in the merge candidate list is more than two.

[0496] Alternatively, when at least one of the width or height of the hardware is greater than 64, there is a disadvantage that a data processing unit of size 64×64 is accessed redundantly. Therefore, when at least one of the width or height of the coding block is greater than a threshold value, dividing the coding block into multiple prediction blocks may not be allowed. For example, when at least one of the height or width of the coding block is greater than 64, diagonal division may not be used.

[0497] Alternatively, based on at least one of whether the number of samples included in the coding block is below a first threshold value or whether the number of samples included in the coding block is above a second threshold value, it can be determined whether to apply diagonal division to the coding block. For example, when the number of samples included in the coding block is below the first threshold value or when the number of samples included in the coding block is above the second threshold value, it is set not to apply diagonal division to the coding block. Alternatively, based on whether the width-to-height ratio of the coding block is less than a first threshold value or whether the width-to-height ratio of the coding block is greater than a second threshold value, it can be determined whether to apply diagonal division to the coding block. Among them, the width-to-height ratio whRatio of the coding block can be determined as the ratio of the width CbW to the height CbH of the coding block, as shown in Equation 21 below.

[0498]

Equation 21

[0499] whRatio = CbW / CbH

[0500] The second threshold value can be the reciprocal of the first threshold value. For example, when the first threshold value is k, the second threshold value can be 1 / k.

[0501] Only when the width-to-height ratio of the coding block exists between the first threshold value and the second threshold value, can diagonal partitioning be applied to the coding block.

[0502] Alternatively, triangular partitioning can be used only when the width-to-height ratio of the coding block is less than the first threshold value or greater than the second threshold value. For example, when the first threshold value is 16, for coding blocks of sizes 64×4 or 4×64, diagonal partitioning may not be allowed.

[0503] Alternatively, based on the partitioning type of the parent node, it can be determined whether diagonal partitioning is allowed. For example, when the coding block as the parent node is partitioned based on quadtree partitioning, diagonal partitioning can be applied to the coding block as the leaf node. On the other hand, when the coding block as the parent node is partitioned based on binary tree or ternary tree partitioning, the coding block as the leaf node can be set to not allow diagonal partitioning.

[0504] Alternatively, based on the predictive coding mode of the coding block, it can be determined whether diagonal partitioning is allowed. For example, when the coding block is encoded with intra prediction, diagonal partitioning can be allowed only when the coding block is encoded with inter prediction or the coding block is encoded with a predefined inter prediction mode. Among them, the predefined inter prediction mode can represent at least one of a merge mode, an advanced motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0505] Alternatively, based on the size of the parallel processing area, it can be determined whether diagonal partitioning is allowed. For example, when the size of the coding block is larger than the size of the parallel processing area, diagonal partitioning may not be used.

[0506] Two or more of the listed conditions can also be considered to determine whether to apply diagonal partitioning to the coding block.

[0507] Furthermore, information indicating whether diagonal partitioning is allowed can be signaled through the bitstream. The information can be signaled at the sequence level, picture level, slice level, or block level. Diagonal partitioning can be applied to the coding block only when the information indicates that diagonal partitioning is allowed and at least one of the listed conditions is satisfied.

[0508] When it is determined to apply diagonal partitioning to the coding block, information indicating the number of lines or the positions of the lines for partitioning the coding block can be signaled through the bitstream.

[0509] For example, when a coding block is partitioned by a diagonal line, information representing the direction of the diagonal line partitioning the coding block can be signaled via a bitstream. For example, a flag triangle_partition_type_flag representing the direction of the diagonal line can be signaled via the bitstream. The flag indicates whether the coding block is partitioned by a diagonal line connecting the upper left and lower right, or by a diagonal line connecting the upper right and lower left. Partitioning the coding block by a diagonal line connecting the upper left and lower right can be referred to as a left triangle partition type, and partitioning the coding block by a diagonal line connecting the upper right and lower left can be referred to as a right triangle partition type. For example, a value of 0 for the flag indicates that the partition type of the coding block is the left triangle partition type, and a value of 1 for the flag indicates that the partition type of the coding block is the right triangle partition type.

[0510] Information representing the partition type of a coding block can be signaled at the coding block level. Thus, the partition type can be determined for different coding blocks to which diagonal partitioning is applied.

[0511] As another example, for a sequence, picture, slice, tile, or coding tree unit, information representing the partition type can be signaled. In this case, the partition type of the coding blocks to which diagonal partitioning is applied can be set to be the same within the sequence, picture, slice, tile, or coding tree unit.

[0512] Alternatively, for a first coding unit to which diagonal partitioning is applied within a coding tree unit, information used to determine the partition type is encoded and signaled, and the second and subsequent coding units to which diagonal partitioning is applied are set to use the same partition type as the first coding unit.

[0513] As another example, the partition type of a coding block can be determined based on the partition types of adjacent blocks. The adjacent blocks can include at least one of an adjacent block adjacent to the upper left corner of the coding block, an adjacent block adjacent to the upper right corner, an adjacent block adjacent to the lower left corner, an adjacent block above, or an adjacent block to the left. For example, the partition type of the current block can be set to be the same as the partition types of the adjacent blocks. Alternatively, based on whether the upper left adjacent block applies the left triangle partition type, and whether the upper right adjacent block or the lower left adjacent block applies the right triangle partition type, the partition type of the current block can be determined.

[0514] In order to perform motion prediction compensation on the first triangular prediction unit and the second triangular prediction unit, the motion information of each of the first triangular prediction unit and the second triangular prediction unit can be derived. In this case, the motion information of the first triangular prediction unit and the second triangular prediction unit can be derived from the merge candidates included in the merge candidate list. To distinguish the general merge candidate list from the merge candidate list used when deriving the motion information of the triangular prediction unit, the merge candidate list used to derive the motion information of the triangular prediction unit is referred to as a triangular merge candidate list, and the merge candidates included in the triangular merge candidate list are referred to as triangular merge candidates. However, applying the foregoing merge candidate derivation method and merge candidate list construction method to the triangular merge candidate and triangular merge candidate list construction method is also included within the spirit of the present invention.

[0515] Information for determining the maximum number of triangular merge candidates that the triangular merge candidate list can include can be signaled via a bitstream. The information can represent the difference between the maximum number of merge candidates that the merge candidate list can include and the maximum number of triangular merge candidates that the triangular merge candidate list can include.

[0516] Triangular merge candidates can be derived from spatially adjacent blocks and temporally adjacent blocks of an encoded block.

[0517] Figure 36 FIG. is a diagram showing adjacent blocks for deriving triangular merge candidates.

[0518] At least one of an adjacent block located above the encoded block, an adjacent block located to the left of the encoded block, or a co-located block included in a different picture from the encoded block can be used to derive triangular merge candidates. The upper adjacent block can be a block including at least one of the samples (xCb+CbW−1, yCb−1) located above the encoded block, the samples (xCb+CbW, yCb−1) located above the encoded block, or the samples (xCb−1, yCb−1) located above the encoded block. The left adjacent block can be a block including at least one of the samples (xCb−1, yCb+CbH−1) located to the left of the encoded block or the samples (xCb−1, yCb+CbH) located to the left of the encoded block. The co-located block can be determined to be either a block including the samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the encoded block within the co-located picture or a block including the samples (xCb / 2, yCb / 2) located at the center of the encoded block.

[0519] Adjacent blocks can be searched in a predefined order, and triangle merge candidates can be constructed into a triangle merge candidate list in a predefined order. For example, triangle merge candidates can be searched in the order of B1, A1, B0, A0, C0, B2, and C1 to construct a triangle merge candidate list.

[0520] The motion information of the triangle prediction unit can be derived based on the triangle merge candidate list. That is, the triangle prediction unit can share a triangle merge candidate list.

[0521] To derive the motion information of the triangle merge unit, information for specifying at least one of the triangle merge candidates included in the triangle merge candidate list can be signaled via a bitstream. For example, index information merge_triangle_idx for specifying at least one of the triangle merge candidates can be signaled via a bitstream.

[0522] The index information can specify a combination of the merge candidates of the first triangle prediction unit and the merge candidates of the second triangle prediction unit. For example, Table 3 below is an example showing combinations of merge candidates based on the index information merge_triangle_idx.

[0523]

Table 3

[0524]

[0525]

[0526] A value of 1 for the index information merge_triangle_idx indicates that the motion information of the first triangle prediction unit is derived from the merge candidate at index 1, and the motion information of the second triangle prediction unit is derived from the merge candidate at index 0. The triangle merge candidate for deriving the motion information of the first triangle prediction unit and the triangle merge candidate for deriving the motion information of the second triangle prediction unit can be determined by the index information merge_triangle_idx.

[0527] The partition type of the coded block to which diagonal partitioning is applied can also be determined according to the index information. That is, the index information can specify a combination of the merge candidates of the first triangle prediction unit, the merge candidates of the second triangle prediction unit, and the partition direction of the coded block. When determining the partition type of the coded block according to the index information, the information triangle_partition_type_flag indicating the direction of the diagonal for partitioning the coded block may not be coded. Table 4 shows the partition type of the coded block with respect to the index information merge_triangle_idx.

[0528]

Table 4

[0529] merge_triangle_idx 0 1 2 3 4 5 6 7 8 TriangleDir 0 1 1 0 0 1 1 1 0 merge_triangle_idx 9 10 11 12 13 14 15 16 17 TriangleDir 0 0 0 1 0 0 0 0 1 merge_triangle_idx 18 19 20 21 22 23 24 25 26 TriangleDir 1 1 1 0 0 1 1 1 1 merge_triangle_idx 27 28 29 30 31 32 33 34 35 TriangleDir 1 1 1 0 0 1 0 1 0 merge_triangle_idx 36 37 38 39 TriangleDir 0 1 0 0

[0530] The variable TriangleDir being 0 indicates that the coding block applies the left triangle partitioning type, and the variable TriangleDir being 1 indicates that the coding block applies the right triangle partitioning type. By combining Table 3 and Table 4, it can be set to specify the combination of the merge candidates of the first triangle prediction unit, the merge candidates of the second triangle prediction unit, and the partitioning direction of the coding block according to the index information merge_triangle_idx.

[0531] As another example, the index information for only one of the first triangle prediction unit and the second triangle prediction unit can be signaled, and the index of the triangle merge candidate for the other of the first triangle prediction unit and the second triangle prediction unit can be determined based on the index information. For example, the triangle merge candidate of the first triangle prediction unit can be determined based on the index information merge_triangle_idx representing any one of the triangle merge candidates. Additionally, the triangle merge candidate of the second triangle prediction unit can be specified based on the merge_triangle_idx. For example, the triangle merge candidate of the second triangle prediction unit can be derived by adding or subtracting an offset to the merge_triangle_idx. The offset can be an integer such as 1 or 2. For example, the triangle merge candidate of the second triangle prediction unit can be determined as the triangle merge candidate having the value obtained by adding 1 to merge_traingle_idx as the index. When merge_triangle_idx indicates the triangle merge candidate with the largest index value among the triangle merge candidates, the motion information of the second triangle prediction unit can be derived from the triangle merge candidate with the index 0 or the triangle merge candidate with the value obtained by subtracting 1 from merge_triangle_idx as the index.

[0532] Alternatively, the motion information of the second triangle prediction unit can be derived from the triangle merge candidate having the same reference image as the triangle merge candidate of the first triangle prediction unit specified according to the index information. Among them, the triangle merge candidate having the same reference image as the triangle merge candidate of the first triangle prediction unit can represent at least one of the L0 reference image or the L1 reference image having the same triangle merge candidate as the triangle merge candidate of the first triangle prediction unit. When there are multiple triangle merge candidates having the same reference image as the triangle merge candidate of the first triangle prediction unit, any one can be selected based on at least one of whether the merge candidate includes bidirectional motion information or the difference between the index of the merge candidate and the index information.

[0533] As another example, for the first triangle prediction unit and the second triangle prediction unit, the index information may be signaled separately. For example, the first index information 1st_merge_idx for determining the triangle merge candidate of the first triangle prediction unit and the second index information 2nd_merge_idx for determining the triangle merge candidate of the second triangle prediction unit may be signaled via a bitstream. The motion information of the first triangle prediction unit may be derived from the triangle merge candidate determined based on the first index information 1st_merge_idx, and the motion information of the second triangle prediction unit may be derived from the triangle merge candidate determined based on the second index information 2nd_merge_idx.

[0534] The first index information 1st_merge_idx may represent any index among the triangle merge candidates included in the triangle merge candidate list. The triangle merge candidate of the first triangle prediction unit may be determined as the triangle merge candidate pointed to by the first index information 1st_merge_idx.

[0535] The triangle merge candidate indicated by the first index information 1st_merge_idx may be set as not available as the triangle merge candidate of the second triangle prediction unit. Therefore, the second index information 2nd_merge_idx of the second triangle prediction unit may indicate the index of any one of the remaining triangle merge candidates other than the triangle merge candidate indicated by the first index information. When the

[0536] value of the second index information 2nd_merge_idx is less than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangle prediction unit may be determined as the triangle merge candidate having the index information represented by the second index information 2nd_merge_idx. On the other hand, when the value of the second index information 2nd_merge_idx is the same as or greater than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangle prediction unit may be determined as the triangle merge candidate having a value obtained by adding 1 to the value of the second index information 2nd_merge_idx as an index.

[0537] Alternatively, according to the number of triangle merge candidates included in the triangle merge candidate list, it can be determined whether to signal the second index information. For example, when the maximum number of triangle merge candidates that the triangle merge candidate list can include does not exceed 2, signaling the second index information can be omitted. When signaling the second index information is omitted, the second triangle merge candidate can be derived by adding or subtracting an offset from the first index information. For example, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information is index 0, the second triangle merge candidate can be derived by adding 1 to the first index information. Alternatively, when the maximum number of triangle merge candidates that the triangle merge candidate list can include is 2 and the first index information is 1, the second triangle merge candidate can be derived by subtracting 1 from the first index information.

[0538] Alternatively, when signaling the second index information is omitted, the second index information can be set to a default value. Among them, the default value can be 0. By comparing the first index information and the second index information, the second triangle merge candidate can be derived. For example, when the second index information is less than the first index information, the merge candidate with index 0 is set as the second triangle merge candidate, and when the second index information is the same as or greater than the first index information, the merge candidate with index 1 is set as the second triangle merge candidate.

[0539] When the triangle merge candidate has unidirectional motion information, the unidirectional motion information of the triangle merge candidate is set as the motion information of the triangle prediction unit. On the other hand, when the triangle merge candidate has bidirectional motion information, only either the L0 motion information or the L1 motion information is set as the motion information of the triangle prediction unit. It can be determined which of the L0 motion information or the L1 motion information to obtain based on the index of the triangle merge candidate or the motion information of another triangle prediction unit.

[0540] For example, when the index of the triangle merge candidate is even, set the L0 motion information of the triangle prediction unit to 0, and set the L1 motion information of the triangle merge candidate to the L1 motion information of the triangle prediction unit. On the other hand, when the index of the triangle merge candidate is odd, set the L1 motion information of the triangle prediction unit to 0, and set the L0 motion information of the triangle merge candidate to 0. Conversely, when the index of the triangle merge candidate is even, the L0 motion information of the triangle merge candidate can be set to the L0 motion information of the triangle prediction unit, and when the index of the triangle merge candidate is odd, the L1 motion information of the triangle merge candidate can be set to the L1 motion information of the triangle prediction unit. Alternatively, for the first triangle prediction unit, when the triangle merge candidate is even, the L0 motion information of the triangle merge candidate can be set to the L0 motion information of the first triangle prediction unit. On the other hand, for the second triangle prediction unit, when the triangle merge candidate is odd, the L1 motion information of the triangle merge candidate can be set to the L1 motion information of the second triangle prediction unit.

[0541] Alternatively, when the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit can be set to 0, and the L1 motion information of the triangle merge candidate can be set to the L1 information of the second triangle prediction unit. On the other hand, when the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit can be set to 0, and the L0 motion information of the triangle merge candidate can be set to the L0 motion signal of the second triangle prediction unit.

[0542] The triangle merge candidate list for deriving the motion information of the first triangle prediction unit and the triangle merge candidate list for deriving the motion information of the second triangle prediction unit can also be set differently.

[0543] For example, when specifying the triangle merge candidate for deriving the motion information of the first triangle prediction unit in the triangle merge candidate list based on the index information related to the first triangle prediction unit, a triangle merge list including the remaining triangle merge candidates other than the triangle merge candidate indicated by the index information can be used to derive the motion information of the second triangle prediction unit. Specifically, the motion information of the second triangle prediction unit can be derived from any one of the remaining triangle merge candidates.

[0544] Therefore, the maximum number of triangle merge candidates included in the triangle merge candidate list of the first triangle prediction unit and the maximum number of triangle merge candidates included in the triangle merge candidate list of the second triangle prediction unit will be different. For example, when the triangle merge candidate list of the first triangle prediction unit includes M merge candidates, the triangle merge candidate list of the second triangle prediction unit may include M - 1 merge candidates other than the triangle merge candidate indicated by the index information of the first triangle prediction unit.

[0545] As another example, merge candidates for each triangle prediction unit can be derived based on neighboring blocks adjacent to the coding block, and the availability of the neighboring blocks can be determined by considering the shape or position of the triangle prediction unit.

[0546] Figure 37 is a diagram for describing an example of determining the availability of neighboring blocks for each triangle prediction unit.

[0547] Neighboring blocks that are not adjacent to the first triangle prediction unit can be set as unavailable neighboring blocks for the first triangle prediction unit, and neighboring blocks that are not adjacent to the second triangle prediction unit can be set as unavailable neighboring blocks for the second triangle prediction unit.

[0548] For example, as shown in the example of Figure 37 (a), when the left triangle partition type is applied to the coding block, it can be determined that among the neighboring blocks adjacent to the coding block, blocks A1, A0, and A2 adjacent to the first triangle prediction unit are available for the first triangle prediction unit, while blocks B0 and B1 are not available for the first triangle prediction unit. Therefore, the triangle merge candidate list related to the first triangle prediction unit includes triangle merge candidates derived from blocks A1, A0, and A2, and does not include triangle merge candidates derived from blocks B0 and B1.

[0549] As shown in the example of Figure 37 (b), when the left triangle partition type is applied to the coding block, it can be determined that blocks B0 and B1 adjacent to the second triangle prediction unit are available for the second triangle prediction unit, while blocks A1, A0, and A2 are not available for the second triangle prediction unit. Therefore, the triangle merge candidate list related to the second triangle prediction unit includes triangle merge candidates derived from blocks B0 and B1, and does not include triangle merge candidates derived from blocks A1, A0, and A2.

[0550] Therefore, the number or range of triangle merge candidates that a triangle prediction unit can use can be determined based on at least one of the position of the triangle prediction unit or the partition type of the coding block.

[0551] As another example, the merge mode may be applied to only one of the first triangular prediction unit and the second triangular prediction unit. Additionally, the motion information of the other one of the first triangular prediction unit and the second triangular prediction unit may be set to be the same as the motion information of the triangular prediction unit to which the merge mode is applied, or the motion information of the triangular prediction unit to which the merge mode is applied may be refined to derive it.

[0552] For example, the motion vector and reference picture index of the first triangular prediction unit may be derived based on triangular merge candidates, and the motion vector of the first triangular prediction unit may be refined to derive the motion vector of the second triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding or subtracting the fine motion vector {Rx, Ry} to or from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0553] The information for determining the fine motion vector representing the difference between the motion vector of the first triangular prediction unit and the motion vector of the second triangular prediction unit may be signaled through the bitstream. The information may include at least one of the information representing the magnitude of the fine motion vector or the information representing the sign of the fine motion vector.

[0554] Alternatively, the sign of the fine motion vector may be derived based on at least one of the position, index, or partition type applied to the coded block of the triangular prediction unit.

[0555] As another example, the motion vector and reference picture index of either the first triangular prediction unit or the second triangular prediction unit may be signaled. The motion vector of the other one of the first triangular prediction unit and the second triangular prediction unit may be derived by refining the signaled motion vector.

[0556] For example, based on the information signaled from the bitstream, the motion vector and reference picture index of the first triangular prediction unit may be determined. Additionally, the motion vector of the second triangular prediction unit may be derived by refining the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding or subtracting the fine motion vector {Rx, Ry} to or from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0557] Motion prediction compensation prediction can be performed on the coded block based on the motion information of the first triangular prediction unit and the motion information of the second triangular prediction unit respectively. In this case, image quality degradation may occur in the boundary portion between the first triangular prediction unit and the second triangular prediction unit. For example, near the edge existing on the boundary between the first triangular prediction unit and the second triangular prediction unit, the continuity of the image quality will deteriorate. To reduce the image quality degradation in the boundary portion, the predicted sample can be derived by smoothing filtering or weighted prediction.

[0558] The predicted sample for applying diagonal partitioning to the coded block can be derived based on the weighted sum operation of the first predicted sample obtained based on the motion information of the first triangular prediction unit and the second predicted sample obtained based on the motion information of the second triangular prediction unit. Alternatively, the predicted sample of the first triangular prediction unit is derived from the first predicted block determined based on the motion information of the first triangular prediction unit, the predicted sample of the second triangular prediction unit is derived from the second predicted block determined based on the motion information of the second triangular prediction unit, and the predicted sample in the boundary region between the first triangular prediction unit and the second triangular prediction unit can be derived based on the weighted sum operation of the first predicted sample included in the first predicted block and the second predicted sample included in the second predicted block. For example, the following Equation 22 shows an example of deriving the predicted samples of the first triangular prediction unit and the second triangular prediction unit.

[0559]

Equation 22

[0560] P(x, y) = w1 * P1(x, y) + (1 - w1) * P2(x, y)

[0561] In Equation 22, P1 represents the first predicted sample, and P2 represents the second predicted sample. w1 represents the weighting value applied to the first predicted sample, and (1 - w1) represents the weighting value applied to the second predicted sample. As shown in the example of Equation 22, the weighting value applied to the second predicted sample can be derived by subtracting the weighting value applied to the first predicted sample from a constant.

[0562] When applying the left triangular partitioning type to the coded block, the boundary region may include predicted samples having the same x-axis coordinate and y-axis coordinate. On the other hand, when applying the right triangular partitioning type to the coded block, the boundary region may include predicted samples whose sum of the x-axis coordinate and the y-axis coordinate is equal to or greater than a first threshold and equal to or less than a second threshold.

[0563] The size of the boundary region can be determined based on at least one of the size of the coded block, the shape of the coded block, the motion information of the triangular prediction unit, the motion vector difference of the triangular prediction unit, the output order of the reference image, or the difference between the first predicted sample and the second predicted sample in the diagonal boundary.

[0564] Figure 38 and Figure 39 is a diagram showing an example of deriving a prediction sample based on a weighted sum operation of a first prediction sample and a second prediction sample. Figure 38 shows a case where a left triangular partitioning type is applied to an encoding block, and Figure 39 shows a case where a right triangular partitioning type is applied to an encoding block. Additionally, Figure 38 (a) of Figure 39 is a diagram showing a prediction state related to a luminance component, and Figure 38 (b) of Figure 39 is a diagram showing a prediction state related to a chrominance component.

[0565] In the diagrams shown, the numbers written in the prediction samples near the boundary between the first prediction unit and the second prediction unit represent the weighting values applied to the first prediction sample. For example, when the number written in the prediction sample is N, the prediction sample can be derived by applying a weighting value of N / 8 to the first prediction sample and a weighting value of (1 - (N / 8)) to the second prediction sample.

[0566] In a non-boundary region, either the first prediction sample or the second prediction sample can be determined as the prediction sample. Referring to Figure 38 's example, in a region belonging to the first triangular prediction unit where the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, the first prediction sample derived based on the motion information of the first triangular prediction unit can be determined as the prediction sample. On the other hand, in a region belonging to the second triangular prediction unit where the difference between the x-axis coordinate and the y-axis coordinate is greater than the threshold, the second prediction sample derived based on the motion information of the second triangular prediction unit can be determined as the prediction sample.

[0567] Referring to Figure 39 's example, in a region where the sum of the x-axis coordinate and the y-axis coordinate is less than a first threshold, the first prediction sample derived based on the motion information of the first triangular prediction unit can be determined as the prediction sample. On the other hand, in a region where the sum of the x-axis coordinate and the y-axis coordinate is greater than a second threshold, the second prediction sample derived based on the motion information of the second triangular prediction unit can be determined as the prediction sample.

[0568] The threshold for determining the non-boundary region can be determined based on at least one of the size of the encoding block, the shape of the encoding block, or the color component. For example, when the threshold related to the luminance component is set to N, the threshold related to the chrominance component can be set to N / 2.

[0569] The predicted samples included in the boundary region can be derived based on the weighted sum operation of the first predicted sample and the second predicted sample. In this case, the weighting values applied to the first predicted sample and the second predicted sample can be determined based on at least one of the position of the predicted sample, the size of the coding block, the shape of the coding block, or the color component.

[0570] For example, in the example shown in Figure 38 (a), the predicted sample at the position with the same x-axis coordinate and y-axis coordinate can be derived by applying the same weighting value to the first predicted sample and the second predicted sample. The ratio of the weighting values applied to the first predicted sample and the second predicted sample can be set to (3:1) or (1:3) to derive the predicted sample with the absolute value of the difference between the x-axis coordinate and the y-axis coordinate being 1. Additionally, the ratio of the weighting values applied to the first predicted sample and the second predicted sample can be set to (7:1) or (1:7) to derive the predicted sample with the absolute value of the difference between the x-axis coordinate and the y-axis coordinate being 2.

[0571] Alternatively, in the example shown in Figure 38 (b), the predicted sample at the position with the same x-axis coordinate and y-axis coordinate can be derived by applying the same weighting value to the first predicted sample and the second predicted sample, and the predicted sample with the absolute value of the difference between the x-axis coordinate and the y-axis coordinate being 1 can be derived by setting the ratio of the weighting values applied to the first predicted sample and the second predicted sample to (7:1) or (1:7).

[0572] For example, in the example shown in Figure 39 (a), the predicted sample with the sum of the x-axis coordinate and the y-axis coordinate being 1 less than the width or height of the coding block can be derived by applying the same weighting value to the first predicted sample and the second predicted sample. The ratio of the weighting values applied to the first predicted sample and the second predicted sample can be set to (3:1) or (1:3) to derive the predicted sample with the sum of the x-axis coordinate and the y-axis coordinate being the same as or 2 less than the width or height of the coding block. The ratio of the weighting values applied to the first predicted sample and the second predicted sample can be set to (7:1) or (1:7) to derive the predicted sample with the sum of the x-axis coordinate and the y-axis coordinate being 1 greater than or 3 less than the width or height of the coding block.

[0573] Alternatively, in the example shown in Figure 39 (b), the predicted sample with the sum of the x-axis coordinate and the y-axis coordinate being 1 less than the width or height of the coding block can be derived by applying the same weighting value to the first predicted sample and the second predicted sample. The ratio of the weighting values applied to the first predicted sample and the second predicted sample can be set to (7:1) or (1:7) to derive the predicted sample with the sum of the x-axis coordinate and the y-axis coordinate being the same as or 2 less than the width or height of the coding block.

[0574] As another example, the position of the prediction sample or the shape of the coding block can be considered to determine the weighting value. Equations 23 to 25 represent examples of deriving the weighting value when applying the left triangular partitioning type to the coding block. Equation 23 represents an example of deriving the weighting value applied to the first prediction sample when the coding block is square.

[0575]

Equation 23

[0576] w1 = (x - y + 4) / 8

[0577] In Equation 23, x and y represent the position of the prediction sample. When the coding block is non-square, the weighting value applied to the first prediction sample can be derived as shown in Equation 24 or 25. Equation 24 shows the case where the width of the coding block is greater than the height, and Equation 25 shows the case where the width of the coding block is less than the height.

[0578]

Equation 24

[0579] w1 = ((x / whRatio) - y + 4) / 8

[0580]

Equation 25

[0581] w1 = (x - (y * whRatio) + 4) / 8

[0582] When applying the right triangular partitioning type to the coding block, the weighting value applied to the first prediction sample can be determined as shown in Equations 26 to 28. Equation 26 represents an example of deriving the weighting value applied to the first prediction sample when the coding block is square.

[0583]

Equation 26

[0584] w1 = (CbW - 1 - x - y) + 4) / 8

[0585] In Equation 26, CbW represents the width of the coding block. When the coding block is non-square, the weighting value applied to the first prediction sample can be derived as shown in Equation 27 or 28. Equation 27 shows the case where the width of the coding block is greater than the height, and Equation 28 shows the case where the width of the coding block is less than the height.

[0586]

Equation 27

[0587] w1 = (CbH - 1 - (x / whRatio) - y) + 4) / 8

[0588]

Equation 28

[0589] w1 = (CbW - 1 - x - (y * whRatio) + 4) / 8

[0590] In Equation 27, CbH represents the height of the coding block.

[0591] As shown in the example, for prediction samples within the boundary region, samples included in the first triangular prediction unit can be derived by assigning a greater weighting value to the first prediction sample than to the second prediction sample, and samples included in the second triangular prediction unit can be derived by assigning a greater weighting value to the second prediction sample than to the first prediction sample.

[0592] When applying diagonal partitioning to an encoding block, the encoding block can be set to a combined prediction mode that does not apply a combination of an intra prediction mode and a merge mode.

[0593] Intra prediction uses the reconstructed samples that have been encoded / decoded around the current block to predict the current block. In this case, the intra prediction of the current block can use the reconstructed samples before applying the in-loop filter.

[0594] Intra prediction techniques include matrix-based intra prediction and general intra prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra prediction technique of the current block can be signaled via a bitstream. The information can be a 1-bit flag. Alternatively, the intra prediction technique of the current block can be determined based on at least one of the position, size, shape of the current block, or the intra prediction technique of adjacent blocks. For example, when the current block exists across the image boundary, the current block can be set not to apply matrix-based intra prediction.

[0595] Matrix-based intra prediction is a method of obtaining a prediction block of the current block based on matrix multiplication between a matrix stored in an encoder and a decoder and the reconstructed samples around the current block. Information for specifying any one of the stored multiple matrices can be signaled via a bitstream. The decoder can determine the matrix for intra prediction of the current block based on the information and the size of the current block.

[0596] General intra prediction is a method of obtaining a prediction block of the current block based on a non-angle intra prediction mode or an angle intra prediction mode. Hereinafter, the process of performing intra prediction based on general intra prediction will be described in more detail with reference to the accompanying drawings.

[0597] Figure 40 is a flowchart showing an intra prediction method according to an embodiment of the present invention.

[0598] A reference sample line of the current block can be determined (S4001). The reference sample line refers to a set of reference samples included in the Kth line deviated from above and / or to the left of the current block. The reference samples can be derived from the reconstructed samples that have been encoded / decoded around the current block.

[0599] Index information of a reference sample line that identifies the current block among multiple reference sample lines can be signaled via a bitstream. For example, index information intra_luma_ref_idx for specifying the reference sample line of the current block can be signaled via a bitstream. The index information can be signaled on a per-coded block basis.

[0600] The multiple reference sample lines can include at least one of the first, second, third, or fourth line above and / or to the left of the current block. A reference sample line formed by a row adjacent to the upper side of the current block and a column adjacent to the left side of the current block among the multiple reference sample lines can be referred to as an adjacent reference sample line, and the other reference sample lines can also be referred to as non-adjacent reference sample lines.

[0601] Only some of the multiple reference sample lines can be selected as the reference sample line of the current block. For example, the remaining reference sample lines among the multiple reference sample lines except for the third non-adjacent reference sample line can be set as candidate reference sample lines. Table 5 shows the indices assigned to each candidate reference sample line.

[0602]

Table 5

[0603] Index (intra_luma_ref_idx) Reference sample line 0 Adjacent reference sample line 1 First non - adjacent reference sample line 2 Second non - adjacent reference sample line

[0604] More candidate reference sample lines than those described can be set, or fewer candidate reference sample lines can also be set. Additionally, the number or position of non-adjacent reference sample lines set as candidate reference sample lines is not limited to the examples described. For example, the first non-adjacent reference sample line and the third non-adjacent reference sample line can be set as candidate reference sample lines, or the second non-adjacent reference sample line and the third non-adjacent reference sample line can be set as candidate reference sample lines. Alternatively, the first non-adjacent reference sample line, the second non-adjacent reference sample line, and the third non-adjacent reference sample line can all be set as candidate reference sample lines.

[0605] The number or type of candidate reference sample lines can be determined based on at least one of the size, shape, position, sub-block partitioning, or intra prediction mode of the current block.

[0606] The reference sample line of the current block can also be determined based on at least one of the position, size, shape, or predictive coding mode of adjacent blocks of the current block. For example, when the current block touches the boundary of an image, tile, stripe, or coding tree unit, the first reference sample line can be determined as the reference sample line of the current block.

[0607] Alternatively, when the current block is not square, the adjacent reference sample lines can be set as the reference sample lines of the current block. Alternatively, when the width-to-height ratio of the current block is above or below a threshold, the adjacent reference sample lines can be determined as the reference sample lines of the current block.

[0608] The reference sample lines can include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper reference sample and the left reference sample can be derived from the reconstructed samples around the current block. The reconstructed samples can be in a state before applying the loop filter.

[0609] Next, the intra prediction mode of the current block can be determined (S4002). For the intra prediction mode of the current block, at least one of a non-angular intra prediction mode or an angular intra prediction mode can be determined as the intra prediction mode of the current block. The non-angular intra prediction mode includes Planer and DC, and the angular intra prediction mode includes 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

[0610] Figure 41 is a diagram showing the intra prediction mode.

[0611] Figure 41 (a) of shows 35 intra prediction modes, and Figure 41 (b) of shows 67 intra prediction modes.

[0612] More or fewer intra prediction modes can also be defined compared to Figure 41 shown.

[0613] The most probable mode (MPM) can be set based on the intra prediction modes of the adjacent blocks adjacent to the current block. Among them, the adjacent blocks can include a left adjacent block adjacent to the left of the current block and an upper adjacent block adjacent to the upper of the current block.

[0614] The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information representing the number of MPMs can be signaled via a bitstream. Alternatively, the number of MPMs can be determined based on at least one of the prediction coding mode of an adjacent block, the size or shape of the current block. For example, N MPMs are used when an adjacent reference sample line is determined as the reference sample line of the current block, and M MPMs can be used when a non-adjacent reference sample line is determined as the reference sample line of the current block. M is an integer less than N. For example, N can be 6, and M can be 5, 4, or 3. Therefore, when the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra prediction mode of the current block can be determined as any one of 6 candidate intra prediction modes, and when the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra prediction mode of the current block can be determined as any one of 5 candidate intra prediction modes.

[0615] Alternatively, a fixed number (e.g., 6 or 5) of MPM candidates can also be used, regardless of the index of the reference sample line of the current block.

[0616] An MPM list including multiple MPMs can be generated, and information indicating whether an MPM identical to the intra prediction mode of the current block is included in the MPM list can be signaled via a bitstream. The information is a 1-bit flag, which can be referred to as the MPM flag. When the MPM flag indicates that an MPM identical to the current block is included in the MPM list, index information identifying one of the MPMs can be signaled via a bitstream. For example, index information mpm_idx specifying any one of the multiple MPMs can be signaled via a bitstream. The MPM specified by the index information can be set as the intra prediction mode of the current block. When the MPM flag indicates that an MPM identical to the current block is not included in the MPM list, residual mode information indicating any one of the residual intra prediction modes other than the MPMs can be signaled via a bitstream. The residual mode information represents the index value corresponding to the intra prediction mode of the current block when reassigning indices to the residual intra prediction modes other than the MPMs. The decoder can sort the MPMs in ascending order and determine the intra prediction mode of the current block by comparing the residual mode information with the MPMs. For example, when the residual mode information is the same as or smaller than the MPM, the intra prediction mode of the current block can be derived by adding 1 to the residual mode information.

[0617] When exporting the intra prediction mode of the current block, some comparisons with the residual mode information in the MPM can be omitted. For example, the MPM of the non - angular intra prediction mode can be excluded from the comparison objects. When the non - angular intra prediction mode is set as the MPM, since the residual mode information clearly indicates the angular intra prediction mode, the intra prediction mode of the current block can be derived by comparing the remaining MPMs other than the non - angular intra prediction mode with the residual mode information. Instead of excluding the non - angular intra prediction mode from the comparison objects, the result value after adding the number of the non - angular intra prediction mode to the residual mode information can be compared with the MPM.

[0618] Instead of the operation of setting the default mode as the MPM, information indicating whether the intra prediction mode of the current block is the default mode can be signaled through the bitstream. The information is a 1 - bit flag, and the flag can be referred to as the default mode flag. The default mode flag can be signaled only when the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode can include at least one of planar, DC, vertical direction mode, or horizontal direction mode. For example, when planar is set as the default mode, the default mode flag can indicate whether the intra prediction mode of the current block is planar. When the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information can be set as the intra prediction mode of the current block.

[0619] When using the default mode flag, the intra prediction mode same as the default mode can not be set as the MPM. For example, when the default mode flag indicates whether the intra prediction mode of the current block is planar, the intra prediction mode of the current block can be derived by using 5 MPMs other than the MPM corresponding to planar.

[0620] When multiple intra prediction modes are set as the default mode, index information indicating any one of the default modes can be further signaled. The intra prediction mode of the current block can be set as the default mode indicated by the index information.

[0621] When the index of the reference sample line of the current block is not 0, the default mode is set not to be used. For example, when a non - adjacent reference sample line is determined as the reference sample line of the current block, non - angular intra prediction modes such as the DC mode or the planar mode can be set not to be used. Therefore, when the index of the reference sample line is not 0, the default mode flag is not signaled, and the value of the default mode flag can be set to a predefined value (i.e., false).

[0622] If the intra prediction mode of the current block is determined, prediction samples related to the current block can be obtained based on the determined intra prediction mode (S4003).

[0623] When the DC mode is selected, prediction samples related to the current block can be generated based on the average value of reference samples. Specifically, the values of all samples of the prediction block can be generated based on the average value of reference samples. At least one of the upper reference samples located above the current block and the left reference samples located to the left of the current block can be used to derive the average value.

[0624] The number or range of reference samples used to derive the average value may vary depending on the shape of the current block. For example, when the current block is a non-square block with a width greater than the height, only the upper reference samples can be used to calculate the average value. On the other hand, when the current block is a non-square block with a width less than the height, only the left reference samples can be used to calculate the average value. That is, when the width and height of the current block are different, only the reference samples adjacent to the longer length can be used to calculate the average value. Alternatively, it can be determined whether to calculate the average value using only the upper reference samples or only the left reference samples based on the width-to-height ratio of the current block.

[0625] When the planar mode is selected, horizontal direction prediction samples and vertical direction prediction samples can be used to obtain prediction samples. Among them, the horizontal direction prediction samples are obtained based on the left reference samples and the right reference samples located on the same horizontal line as the prediction samples, and the vertical direction prediction samples are obtained based on the upper reference samples and the lower reference samples located on the same vertical line as the prediction samples. Among them, the right reference samples can be generated by copying the reference samples adjacent to the upper right corner of the current block, and the lower reference samples can be generated by copying the reference samples adjacent to the lower left corner of the current block. The horizontal direction prediction samples can be obtained based on the weighted sum operation of the left reference samples and the right reference samples, and the vertical direction prediction samples can be obtained based on the weighted sum operation of the upper reference samples and the lower reference samples. In this case, the weighting values assigned to each reference sample can be determined according to the position of the prediction samples. The prediction samples can be obtained based on the average operation or the weighted sum operation of the horizontal direction prediction samples and the vertical direction prediction samples. When performing the weighted sum operation, the weighting values assigned to the horizontal direction prediction samples and the vertical direction prediction samples can be determined based on the position of the prediction samples.

[0626] When the angular prediction mode is selected, a parameter representing the prediction direction (or prediction angle) of the selected angular prediction mode can be determined. Table 6 below shows the intra-prediction parameter intraPredAng for each intra-frame prediction mode.

[0627]

Table 6

[0628]

[0629] Table 6 shows the intra - prediction direction parameters for each intra - prediction mode having an index from 2 to 34 when 35 intra - prediction modes are defined. When more than 33 angular intra - prediction modes are defined, Table 6 is further subdivided to set the intra - prediction direction parameters for each angular intra - prediction mode.

[0630] After arranging the upper reference sample and the left reference sample of the current block in a row, the predicted sample can be obtained based on the value of the intra - prediction direction parameter. In this case, when the value of the intra - prediction direction parameter is negative, the left reference sample and the upper reference sample can be arranged in a row.

[0631] Figure 42 and Figure 43 are diagrams showing examples of one - dimensional arrangements in which reference samples are arranged in a row.

[0632] Figure 42 shows an example of a one - dimensional array in the vertical direction in which reference samples are arranged vertically, and Figure 43 shows an example of a one - dimensional array in the horizontal direction in which reference samples are arranged horizontally. The embodiments of Figure 42 and 43 will be described under the assumption that 35 intra - prediction modes are defined.

[0633] When the intra - prediction mode index is any one of 11 to 18, a one - dimensional arrangement in the horizontal direction in which the upper reference sample is rotated counter - clockwise can be applied, and when the intra - prediction mode index is any one of 19 to 25, a one - dimensional arrangement in the vertical direction in which the left reference sample is rotated clockwise can be applied. When arranging the reference samples in a row, the intra - prediction mode angle can be considered.

[0634] Based on the intra - prediction direction parameter, a reference - sample determination parameter can be determined. The reference - sample determination parameter can include a reference - sample index for specifying a reference sample and a weighting - value parameter for determining a weighting value applied to the reference sample.

[0635] The reference - sample index iIdx and the weighting - value parameter ifact can be obtained through Equation 30 and Equation 31 respectively.

[0636]

Equation 30

[0637] iIdx=(y + 1)*P ang / 32

[0638]

Equation 31

[0639] i fact =[(y + 1)*P ang &31

[0640] In Equations 30 and 31, P angIndicates the intra - direction parameter. The reference sample specified according to the reference sample index iIdx corresponds to an integer pixel (Integer pel).

[0641] To derive a prediction sample, more than one reference sample can be specified. Specifically, the position of the reference sample used to derive the prediction sample can be specified considering the slope of the prediction mode. For example, the reference sample index iIdx can be used to specify the reference sample for deriving the prediction sample.

[0642] In this case, when the slope of the intra - prediction mode cannot be represented by one reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, when the slope of the intra - prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the first reference sample and the second reference sample can be interpolated to obtain the prediction sample. That is, when the angular line following the intra - prediction angle does not pass through the reference sample located at an integer pixel, the reference samples adjacent to the left or right or above or below the position where the angular line passes can be interpolated to obtain the prediction sample.

[0643] The following Equation 32 shows an example of obtaining a prediction sample based on a reference sample.

[0644]

Equation 32

[0645] P(x, y) = ((32 - i fact ) / 32) * Ref_1D(x + iIdx + 1)+(i fact / 32) * Ref_1D(x + iIdx + 2)

[0646] In Equation 32, P represents the prediction sample, and Ref_1D represents any one of the reference samples arranged in one - dimension. In this case, the position of the reference sample can be determined according to the position (x, y) of the prediction sample and the reference sample index iIdx.

[0647] When the slope of the intra - prediction mode can be represented by one reference sample, the weighting value parameter i fact can be set to 0. Therefore, Equation 32 can be simplified as shown in the following Equation 33.

[0648]

Equation 33

[0649] P(x, y) = Ref_1D(x + iIdx + 1)

[0650] Intra - prediction can also be performed on the current block based on multiple intra - prediction modes. For example, intra - prediction modes can be derived for different prediction samples, and prediction samples can be derived based on the intra - prediction modes assigned to each prediction sample.

[0651] Alternatively, intra prediction modes can be derived for different regions, and intra prediction can be performed on each region based on the intra prediction modes assigned to the respective regions. Herein, the region may include at least one sample. At least one of the size or shape of the region can be adaptively determined based on at least one of the size, shape, or intra prediction mode of the current block. Alternatively, at least one of the size or shape of the region can be predefined in the encoder and decoder, regardless of the size or shape of the current block.

[0652] Alternatively, intra prediction can be performed based on multiple intra predictions respectively, and a final predicted sample can be derived based on an average operation or weighted sum operation of multiple predicted samples obtained through multiple intra predictions. For example, intra prediction can be performed based on a first intra prediction mode to obtain a first predicted sample, and intra prediction can be performed based on a second intra prediction mode to obtain a second predicted sample. Thereafter, a final predicted sample can be obtained based on an average operation or weighted sum operation between the first predicted sample and the second predicted sample. In this case, the weighting values assigned to the first predicted sample and the second predicted sample can be determined by considering at least one of whether the first intra prediction mode is a non - angular / angular prediction mode, whether the second intra prediction mode is a non - angular / angular prediction mode, or the intra prediction mode of an adjacent block.

[0653] Multiple intra prediction modes can be a combination of non - angular intra prediction modes and angular prediction modes, a combination of angular prediction modes, or a combination of non - angular prediction modes.

[0654] Figure 44 is a diagram showing the angle formed between an angular intra prediction mode and a line parallel to the x - axis.

[0655] As Figure 44 In the example shown, the angular prediction mode can exist between the lower - left diagonal direction and the upper - right diagonal direction. When described as the angle formed between the x - axis and the angular prediction mode, the angular prediction mode can exist between 45 degrees (lower - left diagonal direction) and - 135 degrees (upper - right diagonal direction).

[0656] If the current block is non - square in shape, according to the intra prediction mode of the current block, a reference sample farther from the predicted sample rather than a reference sample closer to the predicted sample among the reference samples on the line following the intra prediction angle is used to derive the predicted sample.

[0657] Figure 45 is a diagram showing an example of obtaining a predicted sample when the current block is non - square in shape.

[0658] For example, as in Figure 45In the example shown in (a), it is assumed that the current block is a non-square shape with a width greater than the height, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between 0 degrees and 45 degrees. In this case, when deriving the prediction sample A near the right column of the current block, the left reference sample L that is far from the prediction sample among the reference samples on the angular mode at the said angle is used to replace the upper reference sample T that is close to the prediction sample.

[0659] As another example, as in the example shown in Figure 45 (b), it is assumed that the current block is a non-square shape with a height greater than the width, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between -90 degrees and -135 degrees. In the above case, when deriving the prediction sample A near the lower row of the current block, the upper reference sample T that is far from the prediction sample among the reference samples on the angular mode at the said angle is used to replace the left reference sample L that is close to the prediction sample.

[0660] To solve the above problem, when the current block is non-square, the intra prediction mode of the current block can be replaced with an intra prediction mode in the opposite direction. Therefore, for a non-square shaped block, an angular prediction mode with an angle larger or smaller than the angle of the angular prediction mode shown in Figure 41 can be used. This angular intra prediction mode can be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode represents an angular intra prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0661] Figure 46 is a diagram showing the wide-angle intra prediction mode.

[0662] In the example shown in Figure 46 , the intra prediction modes with indices from -1 to -14 and the intra prediction modes with indices from 67 to 80 represent the wide-angle intra prediction mode.

[0663] Although in Figure 46 14 wide-angle intra prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) with angles less than -135 degrees are shown, more or fewer wide-angle intra prediction modes can be defined.

[0664] When using the wide-angle intra prediction mode, the length of the upper reference sample is set to 2W + 1, and the length of the left reference sample is set to 2H + 1.

[0665] When using the wide-angle intra prediction mode, the sample A shown in Figure 45 (a) can be predicted using the reference sample T, and...Figure 45 Sample A shown in (b) thereof.

[0666] By adding the existing intra prediction mode and N wide-angle intra prediction modes, a total of 67 + N intra prediction modes can be used. For example, Table 7 shows the intra prediction direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0667]

Table 7

[0668] PredModeIntra -10 -9 -8 -7 -6 -5 -4 -3 -2 intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -1 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 11 12 13 14 15 16 17 18 intraPredAngle 13 11 9 7 5 3 2 1 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -1 -2 -3 -5 -7 -9 -11 -13 -15 PredModeIntra 28 29 30 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -11 -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -1 0 1 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 11 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

[0669] When the current block is non-square shaped and the intra prediction mode of the current block obtained in step S4002 falls within the transformation range, the intra prediction mode of the current block can be transformed into a wide-angle intra prediction mode. The transformation range can be determined based on at least one of the size, shape, or ratio of the current block. Wherein, the ratio can represent the ratio between the width and height of the current block.

[0670] When the current block is a non-square with a width greater than the height, the transformation range can be set to the intra prediction mode index in the upper right diagonal direction (e.g., 66) to (the index of the intra prediction mode in the upper right diagonal direction - N). Wherein, N can be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transformation range, the intra prediction mode can be transformed into a wide-angle intra prediction mode. The transformation can be performed by subtracting a predefined value from the intra prediction mode, and the predefined value can be the total number of intra prediction modes other than the wide-angle intra prediction mode (e.g., 67).

[0671] According to the embodiment, the intra prediction modes between the 66th and the 53rd can be respectively transformed into the wide-angle intra prediction modes between the -1st and the -14th.

[0672] When the current block is a non-square with a height greater than the width, the transformation range can be set to the intra prediction mode index in the lower left diagonal direction (e.g., 2) to (the index of the intra prediction mode in the lower left diagonal direction + M). Wherein, M can be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transformation range, the intra prediction mode can be transformed into a wide-angle intra prediction mode. The transformation can be performed by adding a predefined value to the intra prediction mode, and the predefined value can be the total number of angular intra prediction modes other than the wide-angle intra prediction mode (e.g., 65).

[0673] According to the embodiment, the intra prediction modes between the 2nd and the 15th can be respectively transformed into the wide-angle intra prediction modes between the 67th and the 80th.

[0674] Hereinafter, an intra prediction mode falling within the transform range is referred to as a wide-angle intra substitution prediction mode.

[0675] The transform range can be determined based on the ratio of the current block. For example, Table 8 and Table 9 each show the transform range when 35 intra prediction modes and 67 intra prediction modes other than the wide-angle intra prediction mode are defined.

[0676]

Table 8

[0677] Condition Replace intra prediction mode W / H = 2 Modes 2, 3, 4 W / H > 2 Modes 2, 3, 4, 5, 6 W / H = 1 None H / W = 1 / 2 Modes 32, 33, 34 H / W < 1 / 2 Modes 30, 31, 32, 33, 34

[0678]

Table 9

[0679] Condition Replace Intra Prediction Mode W / H = 2 Modes 2, 3, 4, 5, 6, 7 W / H > 2 Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H = 1 None H / W = 1 / 2 Modes 61, 62, 63, 64, 65, 66 H / W < 1 / 2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0680] As shown in the examples in Table 8 and Table 9, the number of wide-angle intra substitution prediction modes falling within the transform range can vary according to the ratio of the current block.

[0681] By subdividing the ratio of the current block, the transform range can be set as shown in Table 10.

[0682]

Table 10

[0683] Condition Replace Intra Prediction Mode W / H = 16 Modes 12, 13, 14, 15 W / H = 8 Modes 12, 13 W / H = 4 Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 H / W = 2 Modes 2, 3, 4, 5, 6, 7 H / W = 1 None W / H = 1 / 2 Modes 61, 62, 63, 64, 65, 66 W / H = 1 / 4 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66 W / H = 1 / 8 Modes 55, 56 H / W = 1 / 16 Modes 53, 54, 55, 56

[0684] When a non-adjacent reference sample line is determined as the reference sample line of the current block, or when a multi-line intra prediction coding method for selecting any one of multiple reference sample lines is used, the wide-angle intra prediction mode can be set not to be used. That is, even if the current block is non-square and the intra prediction mode of the current block falls within the transform range, the intra prediction mode of the current block may not be transformed into a wide-angle intra prediction mode.

[0685] Alternatively, when the intra prediction mode of the current block is determined as a wide-angle intra prediction mode, a non-adjacent reference sample line can be set as not available as the reference sample line of the current block, or a multi-line intra prediction coding method for selecting any one of multiple reference sample lines can be set not to be used. When the multi-line intra prediction coding method is not used, an adjacent reference sample line can be set as the reference sample line of the current block.

[0686] When the wide-angle intra prediction mode is not used, refW and refH are set to the sum of nTbW and nTbH. Therefore, except for the top-left reference sample, the non-adjacent reference samples at a distance of i from the current block may include (nTbW + nTbH + offsetX[i]) upper reference samples and (nTbW + nTbH + offsetY[i]) left reference samples. That is, the non-adjacent reference samples at a distance of i from the current block may include (2nTbW + 2nTbH + offsetX[i] + offsetY[i] + 1) reference samples. For example, when the value of whRatio is greater than 1, the value of offsetX is set to be greater than the value of offsetY. For example, the value of offsetX is set to 1 and the value of offsetY is set to 0. On the other hand, when the value of whRatio is less than 1, the value of offsetY is set to be greater than the value of offsetX. For example, the value of offsetX is set to 0 and the value of offsetY is set to 1.

[0687] As the wide-angle intra prediction mode is used in addition to the existing intra prediction modes, the resources required for encoding the wide-angle intra prediction mode increase, so the encoding efficiency may be reduced. Therefore, instead of directly encoding the wide-angle intra prediction mode, a replacement intra prediction mode related to the wide-angle intra prediction mode is encoded, thereby improving the encoding efficiency.

[0688] For example, when encoding the current block using the 67th wide-angle intra prediction mode, the number 2, which is the 67th wide-angle replacement intra prediction mode, can be encoded as the intra prediction mode of the current block. Additionally, when encoding the current block using the -1st wide-angle intra prediction mode, the number 66, which is the -1st wide-angle replacement intra prediction mode, can be encoded as the intra prediction mode of the current block.

[0689] The decoder can decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode is included in the transform range. When the decoded intra prediction mode is a wide-angle replacement intra prediction mode, the intra prediction mode can be transformed into a wide-angle intra prediction mode.

[0690] Alternatively, when encoding the current block in the wide-angle intra prediction mode, the wide-angle intra prediction mode can also be directly encoded.

[0691] The encoding of the intra prediction mode can be implemented based on the MPM list. Specifically, when encoding an adjacent block in the wide-angle intra prediction mode, the MPM can be set based on the wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode.

[0692] The derived residual image can be obtained by subtracting the predicted image from the original image. In this case, when the residual image is transformed into the frequency domain, even if the high-frequency components in the frequency components are removed, the subjective image quality of the video will not be significantly reduced. Therefore, if the values of the high-frequency components are converted to smaller values or set to 0, it has the effect of improving the compression efficiency without causing obvious visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. Transformation techniques such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST) can be used to perform the transformation.

[0693] DCT uses the cosine transform to decompose (or transform) the residual image into two-dimensional frequency components, while DST uses the sine transform to decompose (or transform) the residual image into two-dimensional frequency components. As a result of the transformation of the residual image, the frequency components can be represented as basic images. For example, when performing a DCT transform on a block of size N×N, N 2 basic pattern components can be obtained. The size of each basic pattern component included in a block of size N×N can be obtained through the transformation. Depending on the transformation technique used, the size of the basic pattern component can be referred to as a DCT coefficient or a DST coefficient.

[0694] The transformation technique DCT is mainly used for transforming images with a relatively large distribution of non-zero low-frequency components. The transformation technique DST is mainly used for images with a relatively large distribution of high-frequency components.

[0695] It is also possible to use transformation techniques other than DCT or DST to transform the residual image.

[0696] Hereinafter, the process of transforming the residual image into two-dimensional frequency components is referred to as two-dimensional image transformation. In addition, the size of the basic pattern component obtained through the transformation is referred to as a transformation coefficient. For example, the transformation coefficient can refer to a DCT coefficient or a DST coefficient. When the main transformation and the secondary transformation to be described later are applied simultaneously, the transformation coefficient can represent the size of the basic pattern component generated as a result of the secondary transformation.

[0697] The transformation technique can be determined on a block-by-block basis. The transformation technique can be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, when encoding the current block in the intra-frame prediction mode and the size of the current block is smaller than N×N, the transformation can be performed using the transformation technique DST. On the other hand, when the above conditions are not met, the transformation can be performed using the transformation technique DCT.

[0698] In the residual image, two-dimensional image transformation may not be performed on a part of the blocks. Not performing two-dimensional image transformation may be referred to as Transform Skip. When Transform Skip is applied, quantization may be applied to the residual values for which transformation is not performed.

[0699] After the current block is transformed using DCT or DST, the transformed current block may be transformed again. In this case, the transformation based on DCT or DST may be defined as the main transformation, and the process of transforming the block to which the main transformation is applied again may be referred to as the secondary transformation.

[0700] The main transformation may be performed using any one of a plurality of transform kernel candidates. For example, the main transformation may be performed using any one of DCT2, DCT8, or DCT7.

[0701] Different transform kernels may also be used for the horizontal direction and the vertical direction. Information representing a combination of the transform kernel for the horizontal direction and the transform kernel for the vertical direction may also be signaled through the bitstream.

[0702] The execution units of the main transformation and the secondary transformation may be different. For example, the main transformation may be performed on an 8×8 block, and the secondary transformation may be performed on a 4×4 sub-block within the transformed 8×8 block. In this case, the transform coefficients of the remaining region where the secondary transformation is not performed may also be set to 0.

[0703] Alternatively, the main transformation may be performed on a 4×4 block, and the secondary transformation may be performed on an 8×8 region including the transformed 4×4 block.

[0704] Information representing whether the secondary transformation is performed may be signaled through the bitstream.

[0705] The inverse transformation of the secondary transformation (second inverse transformation) may be performed in the decoder, and the inverse transformation of the main transformation (first inverse transformation) may be performed on the result thereof. As a result of the execution of the second inverse transformation and the first inverse transformation, the residual signal of the current block may be obtained.

[0706] Quantization is used to reduce the energy of the block, and the quantization process includes a process of dividing the transform coefficients by a specific constant. The constant may be derived from a quantization parameter, and the quantization parameter may be defined as a value between 1 and 63.

[0707] If transformation and quantization are performed in the encoder, the decoder may obtain the residual block through inverse quantization and inverse transformation. The decoder may obtain the reconstructed block of the current block by adding the prediction block and the residual block.

[0708] If a reconstructed block of a current block is obtained, information loss generated during quantization and encoding can be reduced through in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset (SAO) filter, or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter is referred to as a first reconstructed block, and the reconstructed block after applying the in-loop filter is referred to as a second reconstructed block.

[0709] At least one of a deblocking filter, SAO, or ALF may be applied to the first reconstructed block to obtain the second reconstructed block. In this case, SAO or ALF may be applied after applying the deblocking filter.

[0710] The deblocking filter is used to mitigate image quality degradation (blocking artifact) generated at the boundary of a block when quantization is performed on a block-by-block basis. To apply the deblocking filter, the blocking strength (BS) between the first reconstructed block and an adjacent reconstructed block may be determined.

[0711] Figure 47 is a flowchart illustrating a process of determining the block strength.

[0712] As Figure 47 shown in the example, P represents the first reconstructed block, and Q represents the adjacent reconstructed block. Among them, the adjacent reconstructed block may be adjacent to the left or above of the current block.

[0713] In Figure 47 the shown example, it is shown that the block strength is determined by considering the predictive coding mode of P and Q, whether non-zero transform coefficients are included, whether inter prediction is performed using the same reference image, and whether the difference in motion vectors is greater than or equal to a threshold.

[0714] Based on the block strength, it may be determined whether the deblocking filter is applied. For example, when the block strength is 0, filtering may not be performed.

[0715] SAO is used to mitigate the ringing artifact generated when performing quantization in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. The method for determining the offset includes Edge Offset (EO) or Band Offset. EO represents a method of determining the offset of the current sample based on the pattern of neighboring pixels. BO represents a method of applying a common offset to a set of pixels having similar luminance values within a region. Specifically, the pixel luminance can be divided into 32 equal intervals, and pixels having similar luminance values can be set as a set. For example, four adjacent bands out of the 32 bands can be set as a group, and the same offset can be applied to the samples belonging to the four bands.

[0716] ALF is a method of generating a second reconstructed image by applying a filter of a predefined size / shape to the first reconstructed image or the reconstructed image to which a deblocking filter has been applied. The following Equation 34 represents an example of applying ALF.

[0717]

Equation 34

[0718]

[0719] Any one of the predefined filter candidates can be selected in units of an image, a coding tree unit, a coding block, a prediction block, or a transform block. Any one of the size or shape of each filter candidate can be different.

[0720] Figure 48 is a diagram showing the predefined filter candidates.

[0721] As in the Figure 48 example shown, at least one of a 5×5, 7×7, and 9×9 rhombus can be selected.

[0722] For the chrominance component, only a 5×5 rhombus can be used.

[0723] Using the embodiments described focusing on the decoding process or the encoding process for the encoding process or the decoding process is also included within the scope of the present invention. Changing the multiple embodiments described in a predetermined order in a different order from that described is also included within the scope of the present invention.

[0724] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention, and they can be executed simultaneously or in a different order as needed. Additionally, in the above embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagrams can also be implemented as hardware devices or software respectively, and multiple structural elements can be combined and implemented as a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc. individually or in combination. Examples of the computer-readable recording medium can include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store program instructions and execute the instructions such as ROMs, RAMs, flash memories, etc. The hardware devices can be configured to operate as one or more software modules to execute the processing according to the present invention, and vice versa.

[0725]

Industrial Applicability

[0726] The present invention can be applied to electronic devices for encoding / decoding videos.

Claims

1. A video decoding method, comprising the following steps: Determine a first prediction unit and a second prediction unit in a coding block; Derive first motion information of the first prediction unit and second motion information of the second prediction unit; And Based on the first motion information and the second motion information, obtain prediction samples in the coding block, wherein, Obtain the first motion information of the first prediction unit according to a first merge candidate, and the first merge candidate is specified from among a plurality of merge candidates included in a merge candidate list by first index information; Obtain the second motion information of the second prediction unit according to a second merge candidate, and the second merge candidate is specified from among a plurality of merge candidates included in the merge candidate list by second index information; wherein, the index of the second merge candidate is determined according to a comparison between the second index information and the first index information.

2. The video decoding method according to claim 1, wherein, When the second index information is equal to or greater than the first index information, the second merge candidate has a value obtained by adding 1 to the second index information as an index.

3. The video decoding method according to claim 1, wherein, When the second index information is less than the first index information, use the second index information as the index of the second merge candidate.

4. The video decoding method according to claim 1, wherein, When the prediction samples are included in a boundary region between the first prediction unit and the second prediction unit, derive the prediction samples according to a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information.

5. The video decoding method according to claim 4, wherein, Determine a first weight value applied to the first prediction sample based on the x-axis coordinate and y-axis coordinate of the prediction sample.

6. The video decoding method according to claim 5, wherein, Derive a second weight value applied to the second prediction sample by subtracting the first weight value from a constant.

7. The video decoding method according to claim 4, wherein, Determine the size of the boundary region based on at least one of the size of the coding block or the shape of the coding block.

8. A video encoding method, comprising the following steps: Determine a first prediction unit and a second prediction unit in a coding block; Derive first motion information of the first prediction unit and second motion information of the second prediction unit; And Based on the first motion information and the second motion information, obtain prediction samples in the coding block, wherein, Obtain the first motion information of the first prediction unit according to a first merge candidate among a plurality of merge candidates included in a merge candidate list; Obtain the second motion information of the second prediction unit according to a second merge candidate among a plurality of merge candidates included in the merge candidate list, Encode first index information for specifying the first merge candidate and second index information for specifying the second merge candidate respectively; wherein, the index of the second merge candidate is determined according to a comparison between the second index information and the first index information.

9. The video encoding method according to claim 8, wherein, when the index of the second merge candidate is greater than the index of the first merge candidate, the second index information is encoded using a value obtained by subtracting 1 from the index of the second merge candidate.

10. The video encoding method according to claim 8, wherein, when the index of the second merge candidate is less than the index of the first merge candidate, the second index information is encoded as the index value of the second merge candidate.

11. The video encoding method according to claim 8, wherein, when the predicted sample is included in the boundary region between the first prediction unit and the second prediction unit, the predicted sample is derived based on a weighted sum operation of a first predicted sample derived from the first motion information and a second predicted sample derived from the second motion information.

12. The video encoding method according to claim 11, wherein, a first weight value applied to the first predicted sample is determined based on the x-axis coordinate and y-axis coordinate of the predicted sample.

13. The video encoding method according to claim 12, wherein, a second weight value applied to the second predicted sample is derived by subtracting the first weight value from a constant.

14. The video encoding method according to claim 11, wherein, the size of the boundary region is determined based on at least one of the size or shape of the coding block.

15. A video decoding device, comprising a processor and a memory for storing a computer program that can run on the processor, the processor being configured to: determine a first prediction unit and a second prediction unit in a coding block; derive first motion information of the first prediction unit and second motion information of the second prediction unit; and obtain a predicted sample in the coding block based on the first motion information and the second motion information, obtain the first motion information of the first prediction unit according to a first merge candidate, the first merge candidate being specified from among a plurality of merge candidates included in a merge candidate list by first index information; Obtain the second motion information of the second prediction unit according to the second merge candidate, where the second merge candidate is specified from among multiple merge candidates included in the merge candidate list by second index information; wherein, the index of the second merge candidate is determined based on a comparison between the second index information and the first index information.

16. The video decoding device according to claim 15, wherein, when the second index information is equal to or greater than the first index information, the second merge candidate has a value obtained by adding 1 to the second index information as an index.

17. The video decoding device according to claim 15, wherein, the processor is configured to: when the second index information is less than the first index information, use the second index information as the index of the second merge candidate.

18. The video decoding device according to claim 15, wherein, the processor is configured to: when the predicted sample is included in the boundary region between the first prediction unit and the second prediction unit, derive the predicted sample based on a weighted sum operation of a first predicted sample derived from the first motion information and a second predicted sample derived from the second motion information.

19. The video decoding device according to claim 18, wherein, the processor is configured to: determine a first weighting value applied to the first prediction sample based on the x-axis coordinate and the y-axis coordinate of the prediction sample.

20. The video decoding device according to claim 19, wherein, the processor is configured to: derive a second weighting value applied to the second prediction sample by subtracting the first weighting value from a constant.

21. The video decoding device according to claim 18, wherein, the processor is configured to: determine the size of the boundary region based on at least one of the size or the shape of the coding block.

22. A video encoding device, comprising: A processor and a memory for storing a computer program capable of running on the processor, the processor being configured to: determine a first prediction unit and a second prediction unit in a coding block; derive first motion information of the first prediction unit and second motion information of the second prediction unit; and obtain a prediction sample in the coding block based on the first motion information and the second motion information, wherein, the first motion information of the first prediction unit is obtained according to a first merge candidate among a plurality of merge candidates included in a merge candidate list; the second motion information of the second prediction unit is obtained according to a second merge candidate among the plurality of merge candidates included in the merge candidate list, encode first index information for specifying the first merge candidate and second index information for specifying the second merge candidate respectively; wherein, the index of the second merge candidate is determined according to a comparison between the second index information and the first index information.

23. The video coding device according to claim 22, wherein, the processor is configured to: when the index of the second merge candidate is greater than the index of the first merge candidate, encode the second index information using a value obtained by subtracting 1 from the index of the second merge candidate.

24. The video coding device according to claim 22, wherein, the processor is configured to: when the index of the second merge candidate is less than the index of the first merge candidate, encode the second index information as the index value of the second merge candidate.

25. The video coding device according to claim 22, wherein, the processor is configured to: when the prediction sample is included in a boundary region between the first prediction unit and the second prediction unit, derive the prediction sample according to a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information.

26. The video coding device according to claim 25, wherein, the processor is configured to: determine a first weighting value applied to the first prediction sample based on the x-axis coordinate and the y-axis coordinate of the prediction sample.

27. The video coding device according to claim 26, wherein, the processor is configured to: derive a second weighting value applied to the second prediction sample by subtracting the first weighting value from a constant.

28. The video encoding device according to claim 25, wherein the processor is configured to: determine the size of the boundary region based on at least one of the size or the shape of the coding block.

29. A storage medium storing an executable program, which when executed by a processor, implements the video decoding method according to any one of claims 1 to 7.

30. A storage medium storing an executable program, which when executed by a processor, implements the video encoding method according to any one of claims 8 to 14.

Images