Video decoding / encoding methods, video decoders / encoders, and bitstream transmission methods

Abstract

This invention relates to a video decoding / encoding method, a video decoding / encoder, and a bitstream transmission method. The video decoding method includes: generating a merging candidate list for a current block; specifying a merging candidate from a plurality of merging candidates included in the merging candidate list; deriving first and second affine seed vectors for the current block based on first and second affine seed vectors of a merging candidate; deriving affine vectors for sub-blocks in the current block using the first and second affine seed vectors of the current block, wherein the sub-blocks are regions smaller than the current block; and performing motion compensation prediction on the sub-blocks based on the affine vectors.

Description

[0001] Cross-references

[0002] This application is a divisional application of Chinese Patent Application No. 201980054695.2, filed on September 20, 2019, under the national phase of PCT international patent application PCT / KR2019 / 012290, entitled "Video Signal Encoding / Decoding Method and Apparatus for the Method". The PCT international patent application claims priority to Korean Patent Application Nos. 10-2018-0114342, 10-2018-0114343, and 10-2018-0114344, filed on September 21, 2018, with the Korean Patent Office. All of the above patent applications are incorporated herein by reference in their entirety. Technical Field

[0003] This invention relates to video decoding methods, video encoding methods, video decoders, video encoders, and bitstream transmission methods. Background Technology

[0004] As display panels become larger, there is a growing need for higher-quality video services. The biggest challenge with high-definition video services is the dramatic increase in data volume. To address this, research is actively underway to improve video compression rates. As a representative example, in 2009, the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013, boasting compression performance approximately twice that of H.264 / AVC. However, with the rapid development of high-definition video services, the limitations of HEVC have gradually become apparent. Summary of the Invention

[0005] Technical problems to be solved

[0006] The purpose of this invention is to provide an inter-frame prediction method using an affine model when encoding / decoding video signals, and an apparatus for the inter-frame prediction method.

[0007] The object of this invention is to provide a method for deriving an affine seed vector using the translational motion vector of a sub-block when encoding / decoding a video signal, and an apparatus for performing said method.

[0008] Another object of the present invention is to provide a method for deriving an affine seed vector by transforming the distance between an adjacent block and the current block into a power of 2 when encoding / decoding a video signal, and an apparatus for performing said method.

[0009] The technical problems to be solved by the present invention are not limited to those mentioned above, and those skilled in the art to which the present invention pertains will clearly understand other technical problems not mentioned through the following description.

[0010] Technical solution

[0011] The video signal decoding / encoding method according to the present invention includes: generating a merging candidate list for a current block; specifying any one of a plurality of merging candidates included in the merging candidate list; deriving a first affine seed vector and a second affine seed vector for the current block based on a first affine seed vector and a second affine seed vector of the specified merging candidate; deriving an affine vector of a sub-block in the current block using the first affine seed vector and the second affine seed vector of the current block; and performing motion compensation prediction on the sub-block based on the affine vector. In this case, the sub-block is a region smaller than the size of the current block. Alternatively, the first affine seed vector and the second affine seed vector of the merging candidate can be derived based on motion information of neighboring blocks adjacent to the current block.

[0012] In the video signal decoding / encoding method according to the present invention, when the adjacent block is included in a coding tree unit different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merging candidate can be derived based on the motion vectors of the lower left sub-block and the lower right sub-block of the adjacent block.

[0013] In the video signal decoding / encoding method according to the present invention, the lower left sub-block may include a lower left reference sample located at the lower left corner of the adjacent block, and the lower right sub-block may include a lower right reference sample located at the lower right corner of the adjacent block.

[0014] In the video signal decoding / encoding method according to the present invention, the first affine seed vector and the second affine seed vector of the merging candidate can be derived using a scaling factor based on a value obtained by shifting the difference between the motion vectors of the lower left sub-block and the lower right sub-block, and the scaling factor can be derived based on a value obtained by adding the horizontal distance between the lower left reference sample and the lower right reference sample to the offset.

[0015] In the video signal decoding / encoding method according to the present invention, the first affine seed vector and the second affine seed vector of the merging candidate can be derived using a scaling factor based on a value obtained by shifting the difference between the motion vectors of the lower left sub-block and the lower right sub-block, and the scaling factor can be derived based on the distance between the adjacent sample to the right of the lower right reference sample and the lower left reference sample.

[0016] In the video signal decoding / encoding method according to the present invention, the merging candidate list includes a first merging candidate and a second merging candidate. The first merging candidate is derived from the upper neighboring block of the first available block determined to be above the current block, and the second merging candidate is derived from the left neighboring block of the first available block determined to be to the left of the current block.

[0017] In the video signal decoding / encoding method according to the present invention, when the adjacent block is included in the same coding tree unit as the current block, the first affine seed vector and the second affine seed vector of the merging candidate can be derived based on the first affine seed vector and the second affine seed vector of the adjacent block.

[0018] The features briefly outlined above are merely exemplary embodiments of the invention as described in the detailed description to follow, and do not limit the scope of the invention.

[0019] Invention Effects

[0020] According to the present invention, the prediction efficiency is improved by using an inter-frame prediction method based on an affine model.

[0021] According to the present invention, the encoding efficiency is improved by deriving the affine seed vector using the translational motion vector of the sub-block.

[0022] According to the present invention, the coding efficiency is improved by transforming the distance between the adjacent block and the current block into a power of 2 to derive the affine seed vector.

[0023] The effects that can be obtained in this invention are not limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art through the following description. Attached Figure Description

[0024] Figure 1 This is a block diagram of a video encoder according to an embodiment of the present invention.

[0025] Figure 2 This is a block diagram of a video decoder according to an embodiment of the present invention.

[0026] Figure 3 This is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0027] Figure 4 This is a diagram illustrating the various partitioning types of coded blocks.

[0028] Figure 5 This is a diagram illustrating an example of how coding tree units are divided.

[0029] Figure 6 This is a flowchart of the inter-frame prediction method according to an embodiment of the present invention.

[0030] Figure 7 It is a diagram showing the nonlinear motion of an object.

[0031] Figure 8 This is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0032] Figure 9 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0033] Figure 10 This is a diagram showing an example of the affine vectors of a sub-block under a 4-parameter motion model.

[0034] Figure 11 This is a diagram showing adjacent blocks that can be used to derive merge candidates.

[0035] Figure 12 This is a diagram illustrating how the affine seed vector of the current block is derived based on the affine seed vectors of neighboring affine blocks.

[0036] Figure 13 This is a diagram illustrating an example of setting the motion vector of a sub-block to an affine seed vector that affines the adjacent block.

[0037] Figures 14 to 16 This is a diagram showing the location of the reference sample.

[0038] Figure 17 This is a diagram illustrating an example of applying a modified affine merging vector derivation method.

[0039] Figure 18 This is a diagram illustrating an example of deriving the affine seed vector of the current block based on non-affine neighboring blocks.

[0040] Figure 19 This is a diagram illustrating an example of replacing an adjacent block with another adjacent block.

[0041] Figure 20 and Figure 21 This is a flowchart illustrating a motion compensation prediction method using multiple merged candidates.

[0042] Figure 22 This is a flowchart of an intra-frame prediction method according to an embodiment of the present invention.

[0043] Figure 23 This is a diagram showing the reference samples included in each reference sample line.

[0044] Figure 24 This is a diagram illustrating the intra-frame prediction mode.

[0045] Figure 25 and Figure 26 This is a diagram illustrating an example of a one-dimensional arrangement of reference samples in a row.

[0046] Figure 27 This is a diagram showing the angle formed between the prediction pattern within the angular frame and a straight line parallel to the x-axis.

[0047] Figure 28 This is a diagram showing an example of obtaining a predicted sample when the current block is not a square.

[0048] Figure 29 This is a diagram illustrating the wide-angle intra-frame prediction mode.

[0049] Figure 30 This is a flowchart illustrating the process of determining block strength.

[0050] Figure 31 This is a graph showing predefined filter candidates.

[0051] Figure 32 An example of dividing an image into multiple tiles is shown.

[0052] Figure 33 This is a diagram illustrating the image partitioning pattern based on the flexible tile technique.

[0053] Figure 34 This is a diagram illustrating an example of assigning tile IDs to each coding tree unit.

[0054] Figure 35 This is a diagram illustrating an example of selectively determining whether to apply a loop filter to each tile. Detailed Implementation

[0055] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0056] Video encoding and decoding are performed on a block-by-block basis. For example, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, or reconstruction can be performed on encoded blocks, transform blocks, or prediction blocks.

[0057] Hereinafter, the block to be encoded / decoded will be referred to as the "current block". For example, depending on the current encoding / decoding process step, the current block can represent an encoded block, a transform block, or a prediction block.

[0058] Additionally, as used herein, the term "unit" refers to a basic unit used to perform a particular encoding / decoding process, and "block" can be understood as representing a sample array of a predetermined size. Unless otherwise stated, "block" and "unit" are used interchangeably. For example, in the embodiments described later, encoding block and encoding unit can be understood to have the same meaning.

[0059] Figure 1 This is a block diagram of a video encoder according to an embodiment of the present invention.

[0060] Reference Figure 1 The video encoding device 100 may include an image segmentation unit 110, a prediction unit 120 and 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy coding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0061] Figure 1 The components shown are illustrated individually to represent the distinct functionalities of the video encoding device, and do not imply that each component is composed of separate hardware or a single software component. That is, for ease of explanation, each structural element is listed and included as a representative structural element, and at least two structural elements can be combined into one structural element or a structural element can be divided into multiple structural elements to perform functions. Such embodiments integrating structural elements and embodiments separating structural elements are also within the scope of this invention, provided they do not depart from the spirit of the invention.

[0062] Furthermore, some structural elements are not essential for performing the essential functions of this invention, but are optional structural elements used only to improve performance. This invention can be implemented by including only the components necessary to achieve the essence of the invention (excluding structural elements used to improve performance), and structures including only the essential structural elements (excluding structural elements used to improve performance) are also within the scope of this invention.

[0063] The image partitioning unit 110 can divide the input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transformation unit (TU), or a coding unit (CU). The image partitioning unit 110 divides an image into a combination of multiple coding units, prediction units, and transformation units, and can select the combination of coding units, prediction units, and transformation units to encode the image based on a predetermined criterion (e.g., a cost function).

[0064] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quadtree structure can be used. A video or the largest coding unit can be used as the root to divide the image into other coding units, which can have as many child nodes as the number of coding units into which they are divided. Coding units that are no longer divided according to certain constraints become leaf nodes. That is, when it is assumed that a coding unit can only be divided into squares, a maximum of four other coding units can be divided into one coding unit.

[0065] In the embodiments of the present invention, the encoding unit may refer to a unit that performs encoding or a unit that performs decoding.

[0066] A prediction unit within a coding unit can be divided into at least one square or rectangle of the same size, or a prediction unit within a coding unit can be divided into units with a different shape and / or size than another prediction unit.

[0067] Intra-prediction can be performed when the prediction unit for intra-prediction based on the coding unit is not the smallest coding unit, without having to divide it into multiple prediction units N×N.

[0068] Prediction units 120 and 125 may include an inter-frame prediction unit 120 that performs inter-frame prediction and an intra-frame prediction unit 125 that performs intra-frame prediction. It can be determined whether inter-frame prediction or intra-frame prediction is used for the prediction unit, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) is determined based on each prediction method. In this case, the processing unit performing the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode may be determined by the prediction unit, and the prediction may be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Furthermore, prediction mode information, motion vector information, etc., used for prediction, along with the residual value, can be encoded in the entropy coding unit 165 and transmitted to the decoder. When using a specific coding mode, the original block can also be directly encoded and transmitted to the decoder without generating a prediction block through the prediction units 120 and 125.

[0069] The inter-frame prediction unit 120 can predict prediction units based on information from at least one of the previous or next images of the current image. In some cases, it can also predict prediction units based on information from a portion of the encoded region within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0070] The reference image interpolation unit receives reference image information from memory 155 and can generate pixel information of integer pixels or fractional pixels from the reference image. For luminance pixels, in order to generate pixel information of fractional pixels in units of 1 / 4 pixels, an 8th-order DCT-based interpolation filter with different filter coefficients can be used. For chrominance signals, in order to generate pixel information of fractional pixels in units of 1 / 8 pixels, a 4th-order DCT-based interpolation filter with different filter coefficients can be used.

[0071] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as the Full Search-based Block Matching Algorithm (FBMA), the Three-Step Search (TSS), and the New Three-Step Search Algorithm (NTS). Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 pixel or 1 / 4 pixel. In the motion prediction unit, different motion prediction methods can be used to predict the current prediction unit. These methods include skipping, merging, Advanced Motion Vector Prediction (AMVP), and Intra Block Copying.

[0072] The intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block (the reference pixel information is pixel information within the current image). If the adjacent block of the current prediction unit is a block that has already undergone inter-frame prediction and the reference pixel is a pixel that has undergone inter-frame prediction, the reference pixels included in the block that has undergone inter-frame prediction can be used as the reference pixel information for the surrounding blocks that have undergone intra-frame prediction. That is, if a reference pixel is unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0073] In intra-frame prediction, the prediction mode can have an angular prediction mode that uses reference pixel information based on the prediction direction and a non-angular mode that does not use direction information when performing prediction. The mode used to predict luminance information and the mode used to predict chrominance information can be different. To predict chrominance information, either the intra-frame prediction mode information used to predict luminance information or the predicted luminance signal information can be used.

[0074] When performing intra-frame prediction, if the size of the prediction unit is the same as the size of the transform unit, intra-frame prediction can be performed based on the pixels to the left, upper left, and upper above the prediction unit. However, when performing intra-frame prediction, if the size of the prediction unit is different from the size of the transform unit, intra-frame prediction can be performed based on the reference pixels of the transform unit. Alternatively, intra-frame prediction using an N×N partition can be applied only to the smallest coding unit.

[0075] After applying an Adaptive IntraSmoothing (AIS) filter to a reference pixel based on the prediction mode, the intra-prediction method can generate a prediction block. The type of adaptive intra-smoothing filter applied to the reference pixel may differ. To perform the intra-prediction method, the intra-prediction mode of the current prediction unit can be predicted based on the intra-prediction modes of prediction units surrounding it. When using mode information predicted from surrounding prediction units to predict the prediction mode of the current prediction unit, if the intra-prediction modes of the current prediction unit and those of surrounding prediction units are the same, predetermined flag information can be used to convey information indicating that the prediction modes of the current prediction unit and those of surrounding prediction units are the same. If the prediction modes of the current prediction unit and those of surrounding prediction units are different, entropy coding can be performed to encode the prediction mode information of the current block.

[0076] Additionally, a residual block including residual information can be generated, the residual information being the difference between the prediction unit performing the prediction based on the prediction unit generated in the prediction units 120, 125 and the original block of the prediction unit. The generated residual block can be input to the transformation unit 130.

[0077] The transform unit 130 can transform the residual block using transform methods such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or transform skipping. The residual block includes residual information between the original block and the prediction units generated by the prediction units 120 and 125. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra-frame prediction mode information of the prediction units used to generate the residual block.

[0078] The quantization unit 135 can quantize the values ​​that have been transformed into the frequency domain in the transform unit 130. The quantization coefficients can be changed according to the importance of the block or image. The values ​​calculated in the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0079] The rearrangement unit 160 can rearrange the coefficient values ​​of the quantized residual values.

[0080] The rearrangement unit 160 can convert 2D block shape coefficients into 1D vector form using a coefficient scanning method. For example, the rearrangement unit 160 can use a zig-zag scan method to scan the DC coefficients and even the coefficients in the high-frequency domain, and convert them into 1D vector form. Depending on the size of the transform unit and the intra-frame prediction mode, instead of zig-zag scanning, vertical scanning along the column direction and horizontal scanning along the row direction can also be used to scan the 2D block shape coefficients. That is, the choice between zig-zag scanning, vertical scanning, and horizontal scanning can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0081] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. For example, entropy coding can use various coding methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0082] The entropy coding unit 165 can encode various information such as residual coefficient information and block type information, prediction mode information, partitioning unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding units originating from the rearrangement unit 160 and the prediction units 120 and 125.

[0083] The entropy coding unit 165 can entropy code the coefficient values ​​of the coding units input from the rearrangement unit 160.

[0084] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on multiple values ​​quantized by the quantization unit 135, and perform inverse transform on the values ​​transformed by the transform unit 130. A reconstructed block can be generated by merging the residual values ​​generated in the inverse quantization unit 140 and the inverse transform unit 145 with the prediction units predicted by the motion prediction unit, motion compensation unit, and intra-frame prediction unit included in the prediction units 120 and 125.

[0085] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0086] Deblocking filters remove block distortion generated in the reconstructed image due to the boundaries between blocks. To determine whether to perform deblocking, the number of pixels in the columns or rows included in the block can be used as a basis. When applying a deblocking filter to a block, a strong or weak filter can be applied depending on the desired deblocking intensity. Furthermore, when using a deblocking filter, horizontal and vertical filtering can be processed simultaneously.

[0087] The offset correction unit can correct the offset between the deblocked image and the original image on a pixel-by-pixel basis. Offset correction can be performed on a specified image in the following ways: after dividing the pixels included in the image into a predetermined number of regions, determine the regions to be offset and apply the offset to the corresponding regions, or apply the offset by taking into account the edge information of each pixel.

[0088] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered reconstructed image and the original image. After dividing the pixels in the image into predetermined groups, a filter can be determined for each group, and filtering can be performed differently for each group. Information regarding whether to apply adaptive loop filtering can be transmitted via a luminance signal for each coding unit (CU). The shape and filter coefficients of the applied adaptive loop filter can vary depending on the block. Alternatively, the same type (fixed type) of ALF can be applied regardless of the characteristics of the block to which it is applied.

[0089] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the prediction units 120 and 125 when performing inter-frame prediction.

[0090] Figure 2 This is a block diagram of a video decoder according to an embodiment of the present invention.

[0091] Reference Figure 2 The video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0092] When inputting a video bitstream from a video encoder, the input bitstream can be decoded by following the reverse steps of the video encoder.

[0093] The entropy decoding unit 210 can perform entropy decoding in the reverse order of the entropy encoding steps performed in the entropy encoding unit of the video encoder. For example, corresponding to the methods performed in the video encoder, various methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied.

[0094] The entropy decoding unit 210 can decode information related to intra-frame prediction and inter-frame prediction performed by the encoder.

[0095] The rearrangement unit 215 can perform rearrangement based on a method of rearranging the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. Multiple coefficients represented in 1D vector form can be reconstructed into 2D block-shaped coefficients for rearrangement. The rearrangement unit 215 can perform rearrangement by receiving information related to the coefficient scan performed by the encoding unit and performing a reverse scan based on the scan order performed by the corresponding encoding unit.

[0096] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged blocks.

[0097] Based on the quantization result performed by the video encoder, the inverse transform unit 225 can perform inverse transforms, namely inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit. The inverse transform can be performed based on the transmission unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, transform methods (e.g., DCT, DST, KLT) can be selectively performed based on multiple factors such as the prediction method, the size of the current block, and the prediction direction.

[0098] Prediction units 230 and 235 can generate prediction blocks based on information related to prediction block generation provided by entropy decoding unit 210 and previously decoded block or image information provided by memory 245.

[0099] As described above, when intra-prediction is performed in the same manner as in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra-prediction is performed on the prediction unit based on the pixels to its left, the pixels to its upper left, and the pixels above it. If the size of the prediction unit is different from the size of the transform unit, intra-prediction can be performed using reference pixels based on the transform unit. Alternatively, intra-prediction using an N×N partition can be applied only to the smallest coding unit.

[0100] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit receives various information input from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method. It classifies prediction units according to the current coding unit and determines whether the prediction unit is performing inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 can use information provided by the video encoder required to perform inter-frame prediction for the current prediction unit, and performs inter-frame prediction for the current prediction unit based on information included in at least one of the previous or next images of the current image to which the current prediction unit belongs. Alternatively, inter-frame prediction can also be performed based on information from a portion of the reconstructed region in the current image to which the current prediction unit belongs.

[0101] To perform inter-frame prediction, the motion prediction method of the prediction units included in the corresponding coding unit can be determined based on the coding unit, which of the following modes is used: Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), or Intra-Block Copy Mode.

[0102] The intra-prediction unit 235 can generate prediction blocks based on pixel information within the current image. When the prediction unit is one that has already performed intra-prediction, intra-prediction can be performed based on the intra-prediction mode information of the prediction unit provided by the video encoder. The intra-prediction unit 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra-smoothing filter is the part that performs filtering on the reference pixels of the current block, and whether to apply the filter can be determined based on the prediction mode of the current prediction unit. Adaptive intra-smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided by the video encoder and the adaptive intra-smoothing filter information. If the prediction mode of the current block is a mode that does not perform adaptive intra-smoothing filtering, then the adaptive intra-smoothing filter may not be applied.

[0103] If the prediction mode of the current prediction unit is a prediction unit that performs intra-frame prediction based on pixel values ​​interpolated from reference pixels, then the reference pixel interpolation unit can generate reference pixels with integer or fractional pixel units by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, then interpolation of reference pixels is not required. If the prediction mode of the current block is DC mode, then the DC filter can generate prediction blocks by filtering.

[0104] The reconstructed blocks or images can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0105] The video decoder can receive information from the video encoder regarding whether to apply a deblocking filter to a corresponding block or image, and information regarding whether to apply a strong or weak filter when applying the deblocking filter. The video decoder's deblocking filter can receive information from the video encoder regarding the deblocking filter, and the video decoder can perform deblocking filtering on the corresponding block.

[0106] The offset correction unit can perform offset correction on the reconstructed image based on the type and amount of offset correction applied to the image during encoding.

[0107] The ALF can be applied to the coding unit based on information provided by the encoder, such as whether the ALF is applied and ALF coefficient information. This ALF information can be provided by including it in a specific parameter set.

[0108] The memory 245 stores the reconstructed image or block, such that the image or block can be used as a reference image or reference block, and the reconstructed image can be provided to the output unit.

[0109] Figure 3 This is a diagram illustrating the basic coding tree unit of an embodiment of the present invention.

[0110] The largest coding block can be defined as the coding tree block. An image can be divided into multiple coding tree units (CTUs). The coding tree unit is the largest coding unit and can also be called the largest coding unit (LCU). Figure 3 An example of dividing an image into multiple coding tree units is shown.

[0111] The size of a coding tree unit can be defined at the image level or the sequence level. Therefore, information indicating the size of the coding tree unit can be sent via signals using either an image parameter set or a sequence parameter set.

[0112] For example, the size of the coding tree unit for the entire image within the sequence can be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image can be set to 128×128, and the size of the coding tree unit in the second image can be set to 256×256.

[0113] Coded blocks can be generated by dividing the coding tree into units. A coded block represents the basic unit used for encoding / decoding processing. For example, prediction or transformation can be performed on different coded blocks, or prediction coding modes can be determined on different coded blocks. The prediction coding mode represents the method for generating the predicted image. For example, prediction coding modes can include intra-prediction, inter-prediction, current picture referencing (CPR, or intra-block copy (IBC)), or combined prediction. For a coded block, at least one of the prediction coding modes—intra-prediction, inter-prediction, current picture referencing, or combined prediction—can be used to generate the prediction block associated with that coded block.

[0114] Information representing the predictive coding mode of the current block can be transmitted via a bitstream signal. For example, this information could be a 1-bit flag indicating whether the predictive coding mode is intra-frame or inter-frame. Current image reference or combined prediction can be used only if the predictive coding mode of the current block is determined to be inter-frame.

[0115] The current image reference is used to set the current image as the reference image and obtain the prediction block of the current block from the encoded / decoded regions within the current image. Here, the current image means the image that includes the current block. Information indicating whether the current image reference is applied to the current block can be sent via a bitstream signal. For example, this information could be a 1-bit flag. When the flag is true, the prediction coding mode of the current block can be determined as the current image reference; when the flag is false, the prediction mode of the current block can be determined as inter-frame prediction.

[0116] Alternatively, the predictive coding mode for the current block can be determined based on a reference image index. For example, when the reference image index points to the current image, the predictive coding mode for the current block can be determined as current image reference. When the reference image index points to another image instead of the current image, the predictive coding mode for the current block can be determined as inter-frame prediction. That is, current image reference is a prediction method that uses information from encoded / decoded regions in the current image, and inter-frame prediction is a prediction method that uses information from other encoded / decoded images.

[0117] Combinatorial prediction represents a coding mode composed of two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, when applying combinatorial prediction, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or the current image reference, and a second prediction block can be generated based on another. When generating the first and second prediction blocks, the final prediction block can be generated by averaging or weighted summing the first and second prediction blocks. Information indicating whether combinatorial prediction is applied can be transmitted via a bitstream signal. This information can be a 1-bit flag.

[0118] Figure 4 This is a diagram illustrating the various partitioning types of coded blocks.

[0119] A coded block can be divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. Furthermore, the divided coded blocks can be further divided into multiple coded blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0120] Quadtree partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block can be divided into four square partitions (see reference). Figure 4 (in 'SPLIT_QT').

[0121] Binary tree partitioning refers to a partitioning technique that divides the current block into two blocks. The process of partitioning the current block along a vertical direction (i.e., using a vertical line that crosses the current block) is called vertical binary tree partitioning, and the process of partitioning the current block along a horizontal direction (i.e., using a horizontal line that crosses the current block) is called horizontal binary tree partitioning. As a result of binary tree partitioning, the current block can be divided into two non-square partitions. Figure 4 In this context, "SPLIT_BT_VER" represents the result of the vertical binary tree partitioning, and Figure 4 In this context, "SPLIT_BT_HOR" represents the result of the horizontal binary tree partitioning.

[0122] Ternary tree partitioning refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along a vertical direction (i.e., using two vertical lines crossing the current block) is called vertical ternary tree partitioning, and the process of dividing the current block into three blocks along a horizontal direction (i.e., using two horizontal lines crossing the current block) is called horizontal ternary tree partitioning. As a result of ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. Figure 4 In this context, "SPLIT_TT_VER" represents the result of the vertical ternary tree partition, and Figure 4 The “SPLIT_TT_HOR” in the text represents the result of the horizontal ternary tree partitioning.

[0123] The number of partitions in a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or image level. Therefore, the maximum partitioning depth of a coding tree unit can vary depending on different sequences or images.

[0124] Alternatively, the maximum partition depth can be determined individually for each of the multiple partitioning techniques. For example, the maximum partition depth allowed for quadtree partitioning can be different from the maximum partition depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0125] The encoder can transmit information via signals from the bitstream indicating at least one of the partition type or partition depth of the current block. The decoder can determine the partition type and partition depth of the coded tree unit based on the information parsed from the bitstream.

[0126] Figure 5 This is a diagram illustrating an example of how coding tree units are divided.

[0127] The process of dividing coding blocks using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning is called multitree partitioning.

[0128] A coded block generated by applying a multi-way tree partitioning to a coded block can be called a downstream coded block. When the partitioning depth of a coded block is k, the partitioning depth of multiple downstream coded blocks is set to k+1.

[0129] On the other hand, for a coding block with a partitioning depth of k+1, the coding block with a partitioning depth of k can be called the upstream coding block.

[0130] The partition type of the current coding block can be determined based on at least one of the partition types of the upstream coding block or the partition types of adjacent coding blocks. Adjacent coding blocks are adjacent to the current coding block and can include at least one of the current coding block's upper adjacent block, left adjacent block, or upper-left adjacent block. The partition type can include at least one of whether to partition into a quadtree, whether to partition into a binary tree, the binary tree partition direction, whether to partition into a ternary tree, or the ternary tree partition direction.

[0131] To determine the type of code block partitioning, information indicating whether the code block has been partitioned can be sent via a bitstream using a signal. This information is a 1-bit flag "split_cu_flag," where a true flag indicates that the code block was partitioned using a multi-way tree partitioning technique.

[0132] When "split_cu_flag" is true, information indicating whether the coded block has been partitioned by a quadtree can be sent via a bitstream signal. This information is a 1-bit flag "split_qt_flag". When this flag is true, the coded block can be divided into 4 blocks.

[0133] For example, in Figure 5 The example shown illustrates the generation of four coded blocks with a partition depth of 1 when the coding tree unit is partitioned by a quadtree. Furthermore, the example illustrates applying quadtree partitioning again to the first and fourth coded blocks, which are the result of the quadtree partitioning. Ultimately, this generates four coded blocks with a partition depth of 2.

[0134] Alternatively, a code block with a partition depth of 3 can be generated by applying a quadtree partition to the code block with a partition depth of 2 again.

[0135] When a quadtree partition is not applied to the coded block, it can be determined whether to perform a binary tree partition or a ternary tree partition by considering at least one of the following: the size of the coded block, whether the coded block is located at an image boundary, the maximum partition depth, or the partition type of adjacent blocks. When it is determined whether to perform a binary tree partition or a ternary tree partition, information indicating the partition direction can be transmitted via a bitstream signal. This information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partition direction (vertical or horizontal) can be determined based on this flag. Alternatively, information indicating whether a binary tree partition or a ternary tree partition is applied to the coded block can be transmitted via a bitstream signal. This information can be a 1-bit flag "mtt_split_cu_binary_flag". The binary tree partition or ternary tree partition can be determined based on this flag.

[0136] For example, in Figure 5The example shown illustrates the application of a vertical binary tree partitioning to a coded block with a partitioning depth of 1, the application of a vertical ternary tree partitioning to the left coded block in the resulting coded block, and the application of a vertical binary tree partitioning to the right coded block.

[0137] Inter-frame prediction refers to using information from the previous image to predict the predictive coding mode of the current block. For example, a block in the previous image that is at the same position as the current block (hereinafter referred to as a collocated block) can be set as the prediction block for the current block. Hereinafter, the prediction block generated based on the block at the same position as the current block will be called a collocated prediction block.

[0138] On the other hand, if an object present in the previous image has moved to a different location in the current image, the object's motion can be used to effectively predict the current block. For example, if the direction and size of the object's movement can be known by comparing the previous and current images, the object's motion information can be considered to generate a predicted block (or predicted image) for the current block. Hereinafter, the predicted block generated using motion information can be referred to as a motion prediction block.

[0139] Residual blocks can be generated by subtracting prediction blocks from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using motion prediction blocks instead of co-position prediction blocks, thereby improving the compression performance of the residual block.

[0140] As mentioned above, the process of generating prediction blocks using motion information can be called motion-compensated prediction. In most inter-frame predictions, prediction blocks can be generated based on motion-compensated prediction.

[0141] Motion information may include at least one of motion vectors, reference image indices, prediction directions, or bidirectional weighted indexes. Motion vectors represent the direction and size of an object's movement. Reference image indices specify the reference image for the current block from among multiple reference images included in the reference image list. Prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 and L1 prediction). Motion information in either the L0 or L1 direction can be used based on the prediction direction of the current block. Bidirectional weighted indexes specify the weights applied to the L0 prediction block and the weights applied to the L1 prediction block.

[0142] Figure 6 This is a flowchart of the inter-frame prediction method according to an embodiment of the present invention.

[0143] refer to Figure 6The inter-frame prediction method includes the following steps: determining the inter-frame prediction mode of the current block (S601); obtaining motion information of the current block according to the determined inter-frame prediction mode (S602); and performing motion compensation prediction on the current block based on the obtained motion information (S603).

[0144] Inter-frame prediction modes represent various techniques used to determine the motion information of the current block, and may include inter-frame prediction modes using translational motion information and inter-frame prediction modes using affine motion information. For example, inter-frame prediction modes using translational motion information may include merging mode and advanced motion vector prediction mode, while inter-frame prediction modes using affine motion information may include affine merging mode and affine motion vector prediction mode. Depending on the inter-frame prediction mode, the motion information of the current block can be determined based on neighboring blocks adjacent to the current block or information parsed from the bitstream.

[0145] The following section details the inter-frame prediction method using affine motion information.

[0146] Figure 7 It is a diagram showing the nonlinear motion of an object.

[0147] The motion of objects within a video may be non-linear. For example, such as... Figure 7 The example shown may involve non-linear motion of the object, such as camera zoom-in, zoom-out, rotation, or affine transformation. When non-linear motion occurs, it is impossible to effectively represent the object's motion using translational motion vectors. Therefore, in parts where non-linear motion occurs, affine motion can be used instead of translational motion, thereby improving coding efficiency.

[0148] Figure 8 This is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0149] Whether to apply an affine motion-based inter-frame prediction technique to the current block can be determined based on information parsed from the bitstream. Specifically, whether to apply an affine motion-based inter-frame prediction technique to the current block can be determined based on at least one of a flag indicating whether to apply an affine merging mode or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0150] When an inter-frame prediction technique based on affine motion is applied to the current block, the affine motion model of the current block can be determined (S801). The affine motion model can be determined to be at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent the affine motion, and the 4-parameter affine motion model uses 4 parameters to represent the affine motion.

[0151] Equation 1 represents the case of affine motion using six parameters. Affine motion represents translational motion within a predetermined region defined by an affine seed vector.

[0152] Equation 1

[0153] v x =ax-by+e

[0154] v y =cx+dy+f

[0155] While using six parameters to represent affine motion allows for the representation of complex motions, the increased number of bits required to encode each parameter reduces encoding efficiency. Therefore, affine motion can also be represented using four parameters. Equation 2 illustrates the case of representing affine motion using four parameters.

[0156] Equation 2

[0157] v x =ax-by+e

[0158] v y =bx+ay+f

[0159] Information used to determine the affine motion model for the current block can be encoded and transmitted via a bitstream signal. For example, this information could be a 1-bit flag, "affine_type_flag". A value of 0 indicates the application of a 4-parameter affine motion model, and a value of 1 indicates the application of a 6-parameter affine motion model. The flag can be encoded at the slice, tile, or block level (e.g., coded block or coded tree unit). When the flag is transmitted at the slice level, the affine motion model determined at that slice level can be applied to all blocks belonging to that slice.

[0160] Alternatively, the affine motion model of the current block can be determined based on the affine inter-frame prediction mode of the current block. For example, when applying the affine merging mode, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when applying the affine motion vector prediction mode, the information used to determine the affine motion model of the current block can be encoded and transmitted as a signal via a bitstream. For example, when applying the affine motion vector prediction mode to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0161] Next, the affine seed vector of the current block can be exported (S802). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be exported. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block can be exported. The motion vectors at the control points can be called affine seed vectors. Control points can include at least one of the upper left, upper right, or lower left corners of the current block.

[0162] Figure 9 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0163] In a 4-parameter affine motion model, two related affine seed vectors can be derived from the top left, top right, or bottom left corners. For example, ... Figure 9 In the example shown in section (a), when the 4-parameter affine motion model is selected, the affine vectors can be derived by using the affine seed vector sv0 associated with the top-left corner of the current block (e.g., the top-left sample (x0, y0)) and the affine seed vector sv1 associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)). Alternatively, the affine seed vector associated with the bottom-left corner can be used instead of the affine seed vector associated with the top-left corner, or vice versa.

[0164] In a 6-parameter affine motion model, affine seed vectors related to the top-left, top-right, and bottom-left corners can be derived. For example, as... Figure 9 In the example shown in section (b), when the 6-parameter affine motion model is selected, the affine vectors can be derived by using the affine seed vector sv0 associated with the top left corner of the current block (e.g., the top left sample (x0, y0)), the affine seed vector sv1 associated with the top right corner of the current block (e.g., the top right sample (x1, y1)), and the affine seed vector sv2 associated with the top left corner of the current block (e.g., the top left sample (x2, y2)).

[0165] In the embodiments described later, under the 4-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments using the first and second affine seed vectors described later, at least one of the first and second affine seed vectors can be replaced by the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0166] Furthermore, in the 6-parameter affine motion model, the affine seed vectors of the upper left control point, upper right control point, and lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiments using the first, second, and third affine seed vectors described later, at least one of the first, second, and third affine seed vectors can be replaced by the affine seed vector of the lower right control point (the fourth affine seed vector).

[0167] An affine seed vector can be used to derive affine vectors for different sub-blocks (S803). Here, the affine vector represents the translational motion vector derived based on the affine seed vector. The affine vector of a sub-block can be called the affine sub-block motion vector or the sub-block motion vector.

[0168] Figure 10 This is a diagram showing an example of the affine vectors of a sub-block under a 4-parameter motion model.

[0169] The affine vector of a sub-block can be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine sub-block vector.

[0170] Equation 3

[0171]

[0172] In Equation 3, (x, y) represents the position of the sub-block. The position of the sub-block refers to the position of the reference sample included within it. The reference sample can be the sample located at the top left corner of the sub-block, or a sample whose x-axis or y-axis coordinate is at its center. (x0, y0) represents the position of the first control point, and (sv 0x sv 0y Let (x1, y1) represent the first affine seed vector. Additionally, (x1, y1) represents the position of the second control point, and (sv... 1x sv 1y ) represents the second affine seed vector.

[0173] When the first control point and the second control point correspond to the top left and top right corners of the current block, respectively, x1-x0 can be set to the same value as the width of the current block.

[0174] Then, motion compensation prediction can be performed on each sub-block using the affine vectors of each sub-block (S804). After performing motion compensation prediction, prediction blocks associated with each sub-block can be generated. The prediction blocks of the sub-blocks can be set as the prediction blocks of the current block.

[0175] The affine seed vector of the current block can be derived based on the affine seed vectors of its neighboring blocks. When the inter-frame prediction mode for the current block is affine merging mode, the affine seed vectors of the merge candidates included in the merge candidate list can be determined as the affine seed vector of the current block. Furthermore, when the inter-frame prediction mode for the current block is affine merging mode, motion information containing at least one of the following—the reference image index, the specific direction prediction flag, or the bidirectional weighting value for the current block—can be set to be the same as that of the merge candidates.

[0176] Merging candidates can be derived based on neighboring blocks of the current block. The neighboring blocks may include at least one of spatially adjacent blocks that are spatially adjacent to the current block and temporally adjacent blocks included in images different from the current image.

[0177] Figure 11 This is a diagram showing adjacent blocks that can be used to derive merge candidates.

[0178] The adjacent blocks of the current block can include at least one of the following: the adjacent block to the left of the current block (A), the adjacent block above the current block (B), the adjacent block to the upper right corner of the current block (C), the adjacent block to the lower left corner of the current block (D), or the adjacent block to the upper left corner of the current block. If the coordinates of the upper left sample of the current block are (x0, y0), then the left adjacent block A includes the sample at the position (x0-1, y0+H-1), and the upper adjacent block B includes the sample at the position (x0+W-1, y0-1). Here, W and H represent the width and height of the current block, respectively. The upper right adjacent block C includes the sample at the position (x0+W, y0-1), and the lower left adjacent block D includes the sample at the position (x0-1, y0+H). The upper left adjacent block E includes the sample at the position (x0-1, y0-1).

[0179] When neighboring blocks are encoded in affine inter-frame prediction mode, the affine seed vectors for merging candidates can be derived based on the affine seed vectors of the corresponding neighboring blocks. In the following text, neighboring blocks encoded in affine inter-frame prediction mode will be referred to as affine neighboring blocks.

[0180] Merging candidates for the current block can be generated by searching neighboring blocks in a predefined scan order. The scan order can be predefined in the encoder and decoder. For example, neighboring blocks can be searched in the order A, B, C, D, E. Alternatively, merging candidates can be derived sequentially from the searched affine neighboring blocks. Alternatively, the scan order can be adaptively determined based on at least one of the current block's size, shape, or affine motion model. That is, blocks with different sizes, shapes, or affine motion models will have different scan orders.

[0181] Alternatively, blocks above the current block are searched sequentially, and merge candidates are derived from the first found affine neighboring blocks. Similarly, blocks to the left of the current block are searched sequentially, and merge candidates are derived from the first found affine neighboring blocks. The multiple neighboring blocks above the current block include at least one of neighboring block E, neighboring block B, or neighboring block C, and the multiple blocks to the left of the current block include at least one of block A or block D. In this case, neighboring block E can also be classified as a block to the left of the current block.

[0182] Although not shown, merge candidates can be derived from temporally adjacent blocks of the current block. Temporally adjacent blocks can include blocks located at the same position as or adjacent to the current block in the co-located image. Specifically, when encoding temporally adjacent blocks of the current block using an affine inter-frame prediction mode, merge candidates can be derived based on the affine seed vector of the temporal merge candidates.

[0183] A merge candidate list including merge candidates can be generated, and the affine seed vector of one of the merge candidates included in the merge candidate list can be determined as the affine seed vector of the current block. To this end, index information identifying any one of the multiple merge candidates can be encoded and transmitted via a bitstream.

[0184] As another example, multiple neighboring blocks can be searched in scan order, and the affine seed vector of the current block can be derived from the affine seed vector of the first found affine neighboring block.

[0185] As mentioned above, in affine merge mode, the affine seed vector of the current block can be derived using the affine seed vector of the adjacent blocks.

[0186] When the inter-frame prediction mode for the current block is affine motion vector prediction mode, the affine seed vectors of the motion vector prediction candidates included in the motion vector prediction candidate list can be determined as the affine seed vector prediction value for the current block. The affine seed vector for the current block can be derived by adding the affine seed vector difference to the predicted affine seed vector value.

[0187] Affine seed vector prediction candidates can be derived based on the neighboring blocks of the current block. Specifically, multiple neighboring blocks above the current block are searched in a predetermined scan order, and a first affine seed vector prediction candidate can be derived from the first found affine neighboring block. Additionally, multiple neighboring blocks to the left of the current block are searched in a predetermined scan order, and a second affine seed vector prediction candidate can be derived from the first found affine neighboring block.

[0188] Information used to determine the affine seed vector difference can be encoded and transmitted via a bitstream. This information may include magnitude information representing the size of the affine seed vector difference and a sign signal representing the sign of the affine seed vector difference. The affine seed vector difference associated with each control point can be set to be the same. Alternatively, the affine seed vector difference can be set differently for each control point.

[0189] As described above, the affine seed vectors of merge candidates or affine seed vector prediction candidates can be derived from the affine seed vectors of affine neighboring blocks, and the affine seed vectors of the current block can be derived using the derived merge candidates or affine seed vector prediction candidates. Alternatively, after searching multiple affine neighboring blocks in a predetermined scan order, the affine seed vector of the current block can also be derived from the affine seed vectors of the first found affine neighboring blocks.

[0190] The following describes in detail a method for deriving the affine seed vector of the current block, a merge candidate, or an affine seed vector prediction candidate from the affine seed vectors of neighboring blocks. In embodiments described later, deriving the affine seed vector of the current block can be understood as deriving the affine seed vector of a merge candidate or deriving the affine seed vector of an affine seed vector prediction candidate.

[0191] Figure 12 This is a diagram illustrating how the affine seed vector of the current block is derived based on the affine seed vectors of neighboring affine blocks.

[0192] If the affine neighboring block stores a first affine seed vector nv0 associated with the top-left control point and a second affine seed vector nv1 associated with the top-right control point, then a third affine seed vector nv2 associated with the bottom-left control point of the affine neighboring block can be derived based on the first and second affine seed vectors. Equation 4 shows an example of deriving the third affine seed vector.

[0193] Equation 4

[0194]

[0195] In equation 4, (nv 0x , nv 0y ) represents the first affine seed vector nv0, (nv1x , nv 1y ) represents the second affine seed vector nv1, and (nv 2x , nv 2y () represents the third affine seed vector nv2. Additionally, (x n0 x n0 (x) represents the position of the first control point. n1 x n1 ) represents the location of the second control point, and (x n2 x n2 ) indicates the location of the third control point.

[0196] Then, the affine seed vector of the current block can be derived using the first affine seed vector, the second affine seed vector, and the third affine seed vector. Equation 5 shows an example of deriving the first affine seed vector v0 of the current block, and Equation 6 shows an example of deriving the second affine seed vector v1 of the current block.

[0197] Equation 5

[0198]

[0199] Equation 6

[0200]

[0201] In equations 5 and 6, (v 0x v 0y ) represents the first affine seed vector sv0 of the current block, and (v 1x v 1y Let (x0, y0) represent the second affine seed vector sv1 of the current block. Additionally, (x0, y0) represents the position of the first control point, and (x1, y1) represents the position of the second control point. For example, the first control point might represent the top-left corner of the current block, and the second control point might represent the top-right corner.

[0202] The example illustrates using three affine seed vectors associated with affine neighboring blocks to derive multiple affine seed vectors for the current block. As another example, it is also possible to derive the affine seed vector for the current block using only two of the multiple affine seed vectors from the affine neighboring blocks.

[0203] Alternatively, multiple affine seed vectors for the current block can be derived using the fourth affine seed vector associated with the bottom right corner, instead of using the first affine seed vector associated with the top left corner, the second affine seed vector associated with the top right corner, or the third affine seed vector associated with the bottom left corner of the affine neighboring block.

[0204] In particular, when the upper boundary of the current block touches the upper boundary of the coding tree unit and the affine seed vector of the upper control point (e.g., upper left or upper right corner) of the affine neighbor block (hereinafter referred to as the upper affine neighbor block) is to be used, these need to be pre-stored in memory, which may cause an increase in the number of line buffers. Therefore, when the upper boundary of the current block touches the upper boundary of the coding tree unit, it can be configured to use the affine seed vector of the lower control point (e.g., lower left or lower right corner) for the upper affine neighbor block instead of using the affine seed vector of the upper control point. For example, multiple affine seed vectors of the current block can be derived using the third affine seed vector associated with the lower left corner and the fourth affine seed vector associated with the lower right corner of the upper affine neighbor block. In this case, the affine seed vector associated with the lower corner can be derived by copying the affine seed vector associated with the upper corner, or it can be derived from multiple affine seed vectors associated with the upper corner. For example, the first, second, or third affine seed vector can be transformed / replaced with a fourth affine seed vector associated with the lower right corner.

[0205] Equations 7 and 8 represent examples of deriving the first and second affine seed vectors of the current block using the third affine seed vector associated with the lower left control point of the adjacent affine vector and the fourth affine seed vector associated with the lower right control point.

[0206] Equation 7

[0207]

[0208] Equation 8

[0209]

[0210] In equations 7 and 8, (x n2 y n2 ) represents the coordinates of the lower left control point of the affine neighboring block, and (x n3 y n3 (x0, y0) represents the coordinates of the bottom-right control point of the affine neighboring block, and (x1, y1) represents the coordinates of the top-left control point of the current block. (nv) 2x , nv 2y ) represents the affine seed vector (i.e., the third affine seed vector) that affines the lower left control point of the adjacent block, and (nv 3x , nv 3y ) represents the affine seed vector (i.e., the fourth affine seed vector) that maps the lower right control point of the adjacent block. (v 0x v 0y) represents the affine seed vector of the top-left control point of the current block (i.e., the first affine seed vector), and (v 1x v 1y ) represents the affine seed vector (i.e., the second affine seed vector) of the upper right control point of the current block.

[0211] The division operations included in Equations 7 and 8 can also be replaced with shift operations. Shift operations can be based on the width between the lower left and lower right control points (i.e., (x...)). n3 -x n2 The exported value is used for execution.

[0212] In the example above, multiple affine seed vectors of the current block can be derived from multiple affine seed vectors of the encoded / decoded affine neighboring blocks. This requires storing these multiple affine seed vectors in memory. However, storing multiple affine seed vectors of the affine neighboring blocks in memory, in addition to storing multiple translational motion vectors (i.e., multiple affine vectors) of the multiple sub-blocks included in the affine neighboring blocks, increases memory usage. To eliminate this problem, the affine seed vectors of the current block can be derived using the motion vectors of the sub-blocks adjacent to the control points of the affine neighboring blocks, thus replacing the affine seed vectors of the affine neighboring blocks. That is, the motion vectors of the sub-blocks adjacent to the control points of the affine neighboring blocks can be set as the affine seed vectors of the affine neighboring blocks. Here, a sub-block is a block with a predefined size / shape in the encoder and decoder, and can be a block with a basic size / shape for storing motion vectors. For example, a sub-block can be a 4×4 square block. Alternatively, the motion vector at a specified sample position can be set as the affine seed vector of the affine neighboring blocks.

[0213] Figure 13 This is a diagram illustrating an example of setting the motion vector of a sub-block to an affine seed vector that affines the adjacent block.

[0214] The motion vector of a sub-block adjacent to a control point can be set as the affine seed vector of the corresponding control point. For example, in Figure 13 In the example shown, the motion vector (nv) of the child block adjacent to the bottom left corner of the affine neighboring block (bottom left child block) can be... 4x , nv 4y ) set as the affine seed vector (nv) of the lower left control point 2x , nv 2y And it can also transfer the motion vector (nv) of the sub-block adjacent to the bottom right corner (bottom right sub-block). 5x , nv 5y Set the affine seed vector (nv) to the control point at the bottom right corner. 3x , nv 3yThe lower left sub-block refers to the block that includes the lower left control point (x) of the adjacent affine block. n2 y n2 Adjacent samples (e.g., (x) n2 y n2 -1) The sub-block of the sample at position), and the lower right sub-block refers to the sub-block that includes the lower right control point (x) of the adjacent affine block. n3 y n3 Adjacent samples (e.g., (x) n3 -1, y n3 -1)) block of sample). When deriving the affine seed vector of the current block based on Equations 7 and 8, the third affine seed vector of the affine neighboring block can be replaced by the motion vector of the lower left sub-block, and the fourth affine seed vector can be replaced by the motion vector of the lower right sub-block.

[0215] In the embodiments described later, the sub-blocks used as affine seed vectors for affine neighboring blocks will be referred to as affine sub-blocks.

[0216] According to one embodiment of the present invention, affine sub-blocks can be determined based on samples at specific locations. For example, sub-blocks including samples at specific locations can also be set as affine sub-blocks. Hereinafter, the samples at specific locations are referred to as affine reference samples. In addition, the reference sample used to determine the lower left control point of the affine sub-block is referred to as the lower left reference sample, and the reference sample used to determine the lower right control point of the affine sub-block is referred to as the lower right reference sample.

[0217] The lower-left and lower-right reference samples can be selected from multiple samples included in the affine neighboring blocks. For example, at least one of the upper-left, lower-left, upper-right, or lower-left samples of the lower-left sub-block can be set as the lower-left reference sample, and at least one of the upper-left, lower-left, upper-right, or lower-left samples of the lower-right sub-block can be set as the lower-right reference sample. Therefore, the motion vectors of the lower-left sub-block including the lower-left reference sample and the lower-right sub-block including the lower-right reference sample can be set as the affine seed vector associated with the lower-left control point and the affine seed vector associated with the lower-right control point, respectively.

[0218] As another example, at least one of the lower-left or lower-right reference samples can be set to a sample located outside the affine neighboring block. For this, the reference... Figures 14 to 16 Please provide a detailed explanation.

[0219] Figures 14 to 16 This is a diagram showing the location of the reference sample.

[0220] like Figure 14 In the example shown in section (a), for the lower left control point, the upper left sample of the lower left sub-block can be set as the reference sample (x). n4 yn4 Therefore, it is possible to include the reference sample (x). n4 y n4 The lower left sub-block of ) is set as an affine sub-block associated with the lower left control point.

[0221] For the bottom right control point, the sample to the right of the top right sample in the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it can be combined with the reference sample (x). n5 y n5 The rightmost adjacent sub-block of the bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0222] Alternatively, such as Figure 14 In the example shown in section (b), for the lower left control point, the sample to the left of the upper left sample in the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it can be combined with the reference sample (x). n4 y n4 The left-adjacent sub-block of the bottom-left sub-block is set as an affine sub-block associated with the bottom-left control point.

[0223] For the bottom right control point, the top right sample of the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it is possible to include the reference sample (x). n5 y n5 The bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0224] Alternatively, such as Figure 15 In the example shown in section (a), for the lower left control point, the lower left sample of the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it is possible to include the reference sample (x). n4 y n4 The lower left sub-block of ) is set as an affine sub-block associated with the lower left control point.

[0225] For the bottom right control point, the sample to the right of the bottom right sample in the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it can be combined with the reference sample (x). n5 y n5 The rightmost adjacent sub-block of the bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0226] Alternatively, such as Figure 15In the example shown in section (b), for the lower left control point, the sample to the left of the lower left sample in the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it can be combined with the reference sample (x). n4 y n4 The left-adjacent sub-block of the bottom-left sub-block is set as an affine sub-block associated with the bottom-left control point.

[0227] For the bottom right control point, the bottom right sample of the bottom right sub-block can be set as the reference sample (x). n5 y n5 Therefore, it is possible to include the reference sample (x). n5 y n5 The bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0228] Alternatively, such as Figure 16 In the example shown in section (a), for the lower left control point, the sample between the upper left and lower left samples located in the lower left sub-block (e.g., the middle sample on the left) can be set as the reference sample (x). n4 y n4 Therefore, it is possible to include the reference sample (x). n4 y n4 The lower left sub-block of ) is set as an affine sub-block associated with the lower left control point.

[0229] For the bottom right control point, the sample to the right of the sample located between the top right and bottom right samples in the bottom right sub-block (e.g., the middle sample on the right) can be set as the reference sample (x). n5 y n5 Therefore, it can be combined with the reference sample (x). n5 y n5 The rightmost adjacent sub-block of the bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0230] Alternatively, such as Figure 16 In the example shown in section (b), for the lower left control point, the sample to the left of the sample located between the upper left and lower left samples in the lower left sub-block can be set as the reference sample (x). n4 y n4 Therefore, it can be combined with the reference sample (x). n4 y n4 The left-adjacent sub-block of the bottom-left sub-block is set as an affine sub-block associated with the bottom-left control point.

[0231] For the bottom right control point, the sample between the top right sample and the bottom right sample located in the bottom right sub-block can be set as the reference sample (x). n5 y n5Therefore, it is possible to include the reference sample (x). n5 y n5 The bottom right sub-block is set as an affine sub-block associated with the bottom right control point.

[0232] When deriving multiple affine seed vectors for the current block based on Equations 7 and 8, the third affine seed vector of the affine neighboring block can be replaced by the motion vector of the affine sub-block associated with the lower left control point, and the fourth affine seed vector can be replaced by the motion vector of the affine sub-block associated with the lower right control point. Furthermore, the position of the lower left control point can be replaced by the position of the lower left reference sample, and the position of the lower right control point can be replaced by the position of the lower right reference sample.

[0233] and Figures 14 to 16 The content differs, and sub-blocks including samples adjacent to the reference sample can also be set as affine sub-blocks. Specifically, samples located outside of affine adjacent sub-blocks can be set as reference samples, and sub-blocks included in affine adjacent blocks can be set as affine sub-blocks. For example, in Figure 14 In the example shown in section (a), the sample to the right of the upper-right sample located in the lower-right sub-block can be set as the reference sample (x). n5 y n5 ), and the bottom-right sub-block can be set as an affine sub-block associated with the bottom-right corner. Alternatively, in Figure 14 In the example shown in section (b), the sample to the left of the top-left sample in the bottom-left sub-block can be set as the reference sample (x). n4 y n4 Furthermore, the bottom left sub-block can be set as an affine sub-block associated with the bottom left corner.

[0234] Can be applied equivalently Figure 15 and Figure 16 The embodiments described herein. That is, in Figure 15 (a) Part or Figure 16 In the example shown in section (a), the bottom right sample of the bottom right sub-block or the sample to the right of the middle sample on the right can be set as the reference sample (x). n5 y n5 ), and the bottom-right sub-block can be set as an affine sub-block associated with the bottom-right corner. Alternatively, in Figure 15 (b) Partial or Figure 16 In the example shown in section (b), the bottom left sample of the bottom left sub-block or the sample to the left of the middle left sample can be set as the reference sample (x). n4 y n4 Furthermore, the bottom left sub-block can be set as an affine sub-block associated with the bottom left corner.

[0235] In the example above, the affine seed vector of the affine neighboring block can be derived using the motion vector of the affine sub-block. Therefore, for an encoded / decoded block, the motion vector can be stored on a sub-block basis.

[0236] As another example, after storing a minimum number of affine seed vectors in the affine neighboring blocks, the motion vectors of the affine sub-blocks can be derived using the stored multiple affine seed vectors.

[0237] Equations 9 and 10 represent examples of deriving the motion vector of an affine sub-block using the affine seed vector of an affine neighboring block.

[0238] Equation 9

[0239]

[0240] Equation 10

[0241]

[0242] In equations 9 and 10, (nv 4x , nv 4y ) represents the motion vector of the affine sub-block associated with the lower left control point, and (nv 5x , nv 5y (nv) represents the motion vector of the affine sub-block associated with the lower right control point. Since the motion vector of the affine sub-block is set to the same as the affine seed vector of the control point, (nv) 4x , nv 4y It can be derived from the affine seed vector (nv) associated with the lower left control point. 2x , nv 2y ) instead, or (nv 5x , nv 5y It can be derived from the affine seed vector (nv) associated with the lower right control point. 3x , nv 3y )replace.

[0243] (x n4 y n4 () indicates the position of the reference sample in the bottom-left sub-block. Alternatively, the center position of the bottom-left sub-block or the position of the bottom-left control point can be used instead. (x n5 y n5 This indicates the position of the reference sample in the bottom right sub-block. Alternatively, the center position of the bottom right sub-block or the position of the bottom right control point can be used instead.

[0244] Equations 9 and 10 can be applied when the current block does not contact the boundary of the coding tree unit. When the current block contacts the upper boundary of the coding tree unit, instead of using Equations 9 and 10, the translational motion vector of the affine sub-block determined based on the lower left reference sample can be set as the third affine seed vector, and the translational motion vector of the affine sub-block determined based on the lower right reference sample can be set as the fourth affine seed vector.

[0245] In equations 7 and 8, (x n3 -x n2 The 'x' represents the width between the bottom left and bottom right control points. As mentioned above, x... n3 The position x of the reference sample in the lower right corner can be used as a reference. n5 Instead, and x n2 The position x of the lower left reference sample can be used as a reference. n4 Instead. Below, (x) n3 -x n2 ) or the value obtained by replacing the position of the aforementioned equation with the position of the reference sample (e.g., (x) n5 -x n4 Defined as variable W seed The variable mentioned is called the sub-seed vector width.

[0246] Depending on the location of the reference sample, the width of the sub-seed vector may not be a power of 2 (e.g., 2^32). n In cases where the bottom left sample of the bottom left sub-block is set as the bottom left reference sample and the bottom right sample of the bottom right sub-block is set as the bottom right reference sample, the width of the sub-seed vector is not a multiple of 2. As mentioned above, when the width of the sub-seed vector is not a power of 2, it can be transformed into a power of 2. This transformation can include adding / subtracting an offset to the sub-seed vector width, or using the position of a sample adjacent to the reference sample instead of the reference sample's position. For example, the transformed sub-seed vector width can be derived by adding 1 to the width between the bottom left and bottom right reference samples. Alternatively, the width between the adjacent reference sample to the right of the bottom right reference sample and the bottom left reference sample can be set as the transformed sub-seed vector width. Then, the affine seed vector of the current block can be derived by substituting the transformed sub-seed vector width into Equations 7 and 8.

[0247] The division operations included in Equations 7 and 8 can also be replaced with shift operations. Shift operations can be performed based on values ​​derived from the transformed sub-seed vector width (i.e., values ​​represented as powers of 2).

[0248] When the reference sample used to determine the affine sub-block does not belong to the affine neighboring block, the affine seed vector of the affine neighboring block can be derived based on the samples adjacent to the reference sample among the multiple samples included in the affine neighboring block. Specifically, the translational motion vector of the sub-block that includes samples adjacent to the reference sample in the affine neighboring block (hereinafter referred to as adjacent reference samples) can be set as the affine seed vector of the affine neighboring block. As described above, the method of deriving the affine seed vector using adjacent reference samples can be defined as a modified affine merge vector derivation method.

[0249] Figure 17 This is a diagram illustrating an example of applying a modified affine merging vector derivation method.

[0250] When affine the lower right reference sample (x) of adjacent block E n5 y n5 When a sample is not part of an affine neighbor block, it can be determined based on the sample (x) that is to the left of the lower-right reference sample within the samples included in the affine neighbor block. n5 -1, y n5 This is used to derive the affine seed vector. Specifically, it can include neighboring reference samples (x... n5 -1, y n5 The translational motion vector of the sub-block is set as the affine seed vector of the lower right control point.

[0251] exist Figure 17 In the example shown, the sample adjacent to the right side of the upper right sample of the lower right sub-block is shown as the lower right reference sample. When the sample adjacent to the right side of the lower right sample of the lower right sub-block or the sample adjacent to the right side of the middle right sample of the lower right sub-block is set as the lower right reference sample, the affine seed vector can be derived based on the sample adjacent to the left side of the adjacent reference sample.

[0252] Additionally, when the lower left reference sample does not belong to the affine neighboring block, the affine seed vector can also be derived based on the sample adjacent to the right side of the lower left reference sample, as described in the embodiment.

[0253] By setting the position of the reference sample and the sub-blocks used to derive the affine seed vector in different ways, the width of the sub-seed vector can be set to a power of 2.

[0254] Merging candidates, affine seed vector prediction candidates, or affine seed vectors for the current block can be derived using neighboring blocks around the current block that are not encoded in affine inter-frame mode. Specifically, blocks not encoded in affine inter-frame mode can be combined, and these combinations can be set as merging candidates or affine seed vector prediction candidates. For example, a combination of at least one of the motion vectors of any neighboring block adjacent to the top-left corner of the current block, any neighboring block adjacent to the top-right corner of the current block, and any neighboring block adjacent to the bottom-left corner of the current block can be set as a merging candidate or affine seed vector prediction candidate. In this case, the motion vectors of the neighboring blocks adjacent to the top-left corner, the top-right corner, and the bottom-left corner can be set as the first affine seed vector of the top-left control point, the second affine seed vector of the top-right control point, and the third affine seed vector of the bottom-left control point, respectively.

[0255] Alternatively, in the modified affine merge vector derivation method described above, neighboring blocks not encoded in affine inter-frame mode can be used to derive the merge candidate, affine seed vector prediction candidate, or affine seed vector of the current block. In the following text, neighboring blocks not encoded in affine inter-frame mode will be referred to as non-affine neighboring blocks.

[0256] Figure 18 This is a diagram illustrating an example of deriving the affine seed vector of the current block based on non-affine neighboring blocks.

[0257] exist Figure 18 In the example shown, it is assumed that all adjacent blocks to the current block are non-affine adjacent blocks.

[0258] To derive the affine seed vector of the current block from a non-affine neighboring block A in a neighboring block adjacent to the current block, the lower-left and lower-right reference samples of A can be set. For example, the sample to the left of the lower-left sample of block A can be set as the lower-left reference sample, and the lower-right sample of block A can be set as the lower-right reference sample. Since the lower-left reference sample is outside of block A, the motion vector of the sub-block including the sample to the right of the lower-left reference sample can be set as the third affine seed vector of block A. Additionally, the motion vector of the sub-block including the lower-right reference sample can be set as the fourth affine seed vector of block A. Thereafter, the first and second affine seed vectors of the current block can be derived from block A based on Equations 9 and 10.

[0259] The method of deriving affine seed vectors from non-affine neighboring blocks can only be used when performing motion compensation prediction for non-affine neighboring blocks on a sub-block basis. Here, the prediction techniques used to perform motion compensation prediction on a sub-block basis may include at least one of STMVP, ATMVP, bidirectional optical flow (BIO), overlapping block motion compensation (OBMC), and decoder-side motion vector correction (DMVR).

[0260] In the above embodiments, it has been described that when the upper boundary of the current block contacts the boundary of the coding tree unit, the third affine seed vector of the lower left control point and the fourth affine seed vector of the lower right control point of the affine neighboring block located above the current block are used to derive the merging candidate, affine seed vector prediction candidate or affine seed vector of the current block.

[0261] As another example, if the upper boundary of the current block touches the boundary of the coding tree unit, and the adjacent block above the current block belongs to a coding tree unit different from the coding tree unit of the current block, then instead of using the adjacent block, the closest adjacent block among the blocks included in the coding tree unit to which the current block belongs is used to derive the merge candidate, affine seed vector prediction candidate, or affine seed vector of the current block.

[0262] exist Figure 19 In the example shown, the current block is in contact with the upper boundary of the coding tree unit, and blocks B, C, and E above the current block belong to coding tree units different from the coding tree unit of the current block. Therefore, instead of using block E, the affine seed vector of the current block can be derived using block F, which is adjacent to block E among the blocks included in the coding tree unit to which the current block belongs.

[0263] For motion compensation prediction of the current block, affine seed vectors from multiple blocks can be used. For example, multiple merge candidates can be selected from a merge candidate list, and the affine seed vector of the current block or sub-block vector can be derived based on the affine seed vectors of the selected merge candidates. Encoding / decoding the current block using affine seed vectors from multiple blocks can be called a multi-affine merge coding method.

[0264] Information indicating whether a multi-affine merging encoding method has been applied to the current block can be encoded and transmitted via a bit stream. Alternatively, the application of a multi-affine merging encoding method to the current block can be determined based on at least one of the following: the number of affine neighboring blocks in the adjacent neighboring blocks, the number of merge candidates included in the merge candidate list, and the affine motion model of the current block.

[0265] Figure 20 and Figure 21 This is a flowchart illustrating a motion compensation prediction method using multiple merged candidates.

[0266] Figure 20 A diagram illustrating an example of deriving the affine seed vector of the current block using multiple merging candidate affine seed vectors is shown. Figure 21 This is a diagram illustrating an example of deriving the motion vector for each sub-block using affine seed vectors from multiple merging candidates.

[0267] The affine seed vector for the current block can be generated based on the sum, difference, average, or weighted sum of two candidate affine seed vectors.

[0268] Equations 11 and 12 below show an example of deriving the affine seed vector of the current block by adding the affine seed vectors of the merged candidates.

[0269] Equation 11

[0270] (sv 4x sv 4y )=(sv 0x sv 0y )+(sv 2x ,sv 2y )

[0271] Equation 12

[0272] (sv 5x sv 5y )=(sv 1x sv 1y )+(sv 3x sv 3y )

[0273] In Equations 11 and 12, sv4 represents the first affine seed vector of the current block, sv0 represents the first affine seed vector of the first merge candidate, and sv2 represents the first affine seed vector of the second merge candidate. Additionally, sv5 represents the second affine seed vector of the current block, sv1 represents the second affine seed vector of the first merge candidate, and sv3 represents the second affine seed vector of the second merge candidate.

[0274] Additionally, Equations 13 and 14 below show an example of deriving the affine seed vector of the current block by merging the weighted sum of the candidate affine seed vectors.

[0275] Equation 13

[0276] (sv 4x sv 4y )=(sv 0x ,sv 0y )((sv 2x ,sv 2y )>>1)

[0277] Equation 14

[0278] (sv 5x sv 5y )=(sv 1x sv 1y )((sv 3x sv 3y ))>>1)

[0279] As another example, a first sub-block motion vector and a second sub-block motion vector can be generated for each sub-block in the current block, based on the affine seed vectors of the first and second merge candidates. Subsequently, a final sub-block motion vector can be generated based on the sum, difference, average, or weighted sum of the first and second sub-block motion vectors.

[0280] Equation 15 below shows an example of obtaining the final sub-block motion vector by adding the first sub-block motion vector and the second sub-block motion vector.

[0281] Equation 15

[0282] (V 2x V 2y )=(V 0x V 0y )+(V 1x V 1y )

[0283] In Equation 15, V0 represents the motion vector of the first sub-block, V1 represents the motion vector of the second sub-block, and V2 represents the motion vector of the final sub-block.

[0284] Additionally, Equation 16 below shows an example of deriving the final sub-block motion vector by weighted summation of the first and second sub-block motion vectors.

[0285] Equation 16

[0286] (V 2x V 2y )=(V 0x V 0y )+((V 1x V 1y )>>1)

[0287] Intra-frame prediction uses reconstructed samples that have already been encoded / decoded from the surrounding blocks to predict the current block. In this case, intra-frame prediction of the current block can use reconstructed samples before the application of the in-loop filter.

[0288] Intra-prediction techniques include matrix-based intra-prediction and general intra-prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra-prediction technique for the current block can be signaled via a bitstream. This information can be a 1-bit flag. Alternatively, the intra-prediction technique for the current block can be determined based on at least one of the current block's position, size, shape, or the intra-prediction techniques of neighboring blocks. For example, when the current block crosses an image boundary, it can be set not to apply matrix-based intra-prediction.

[0289] Matrix-based intra-frame prediction is a method that obtains the predicted block for the current block by performing matrix multiplication between matrices stored in the encoder and decoder and reconstructed samples surrounding the current block. Information specifying any one of the stored matrices can be sent via a bitstream signal. The decoder can then determine the matrix for intra-frame prediction of the current block based on this information and the size of the current block.

[0290] General intra-frame prediction is a method for obtaining the prediction block of the current block based on non-angular intra-frame prediction mode or angular intra-frame prediction mode. The process of performing intra-frame prediction based on general intra-frame prediction is described in more detail below with reference to the accompanying drawings.

[0291] Figure 22 This is a flowchart of an intra-frame prediction method according to an embodiment of the present invention.

[0292] The reference sample line for the current block can be determined (S2201). The reference sample line refers to the set of reference samples included in the Kth line offset from above and / or to the left of the current block. The reference samples can be derived from the reconstructed samples that have been encoded / decoded around the current block.

[0293] Index information of reference sample lines identifying the current block among multiple reference sample lines can be transmitted via a bitstream using signals. The multiple reference sample lines may include at least one of a first, second, third, or fourth line located above and / or to the left of the current block. Table 1 shows the index assigned to each reference sample line. In Table 1, it is assumed that the first, second, and fourth lines are used as reference sample line candidates.

[0294] Table 1

[0295] 0 First reference sample line 1 Second reference sample line 2 Fourth reference sample line

[0296] The reference sample line for the current block can also be determined based on at least one of the following: the position, size, shape, or predicted coding patterns of adjacent blocks. For example, when the current block is adjacent to the boundary of an image, tile, strip, or coding tree unit, the first reference sample line can be determined as the reference sample line for the current block.

[0297] The reference sample line can include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper and left reference samples can be derived from the reconstructed samples surrounding the current block. The reconstructed samples can be in a state prior to the application of the in-loop filter.

[0298] Figure 23 This is a diagram showing the reference samples included in each reference sample line.

[0299] Based on the intra-prediction mode of the current block, a prediction sample can be obtained using at least one of the reference samples belonging to the reference sample line.

[0300] Next, the intra-prediction mode of the current block can be determined (S2202). For the intra-prediction mode of the current block, at least one of a non-angular intra-prediction mode or an angular intra-prediction mode can be determined as the intra-prediction mode of the current block. Non-angular intra-prediction modes include Planer and DC, and angular intra-prediction modes include 33 or 65 modes from the lower left diagonal to the upper right diagonal.

[0301] Figure 24 This is a diagram illustrating the intra-frame prediction mode.

[0302] Figure 24 (a) shows 35 intra-frame prediction modes, and Figure 24 (b) shows 67 intra-frame prediction modes.

[0303] It can also be defined as compared to Figure 24 The number of more or fewer intra-frame prediction modes shown.

[0304] The Most Probable Mode (MPM) can be set based on the intra-prediction modes of neighboring blocks adjacent to the current block. Neighboring blocks can include the left-side neighboring block to the left of the current block and the top-side neighboring block above the current block. When the coordinates of the top-left sample of the current block are (0, 0), the left-side neighboring block can include samples at positions (-1, 0), (-1, H-1), or (-1, (H-1) / 2), where H represents the height of the current block. The top-side neighboring block can include samples at positions (0, -1), (W-1, -1), or ((W-1) / 2, -1), where W represents the width of the current block.

[0305] When encoding adjacent blocks using general intra-prediction, the MPM can be derived based on the intra-prediction modes of the adjacent blocks. Specifically, the intra-prediction mode of the left adjacent block can be set to the variable candIntraPredModeA, and the intra-prediction mode of the upper adjacent block can be set to the variable candIntraPredModeB.

[0306] In this case, when a neighboring block is unavailable (e.g., when a neighboring block has not yet been encoded / decoded or when the neighboring block's position deviates from the image boundary), the variable `candIntraPredModeX` (where X is A or B), derived from the neighboring block's intra-prediction mode, can be set to the default mode if the neighboring block is encoded using matrix-based intra-prediction, inter-prediction, or is included in a different coding tree unit than the current block. The default mode can include at least one of planar mode, DC mode, vertical mode, or horizontal mode.

[0307] Alternatively, when encoding adjacent blocks using matrix-based intra-prediction, the intra-prediction mode corresponding to the index value used to specify any one of the matrices can be set to candIntraPredModeX. For this purpose, a lookup table indicating the mapping between the index values ​​used to specify the matrices and the intra-prediction modes can be pre-stored in the encoder and decoder.

[0308] The MPM can be derived based on the variables candIntraPredModeA and candIntraPredModeB. The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information representing the number of MPMs can be sent as a signal via a bitstream. Alternatively, the number of MPMs can be determined based on at least one of the predictive coding modes of neighboring blocks, the size of the current block, or its shape.

[0309] In the embodiments described later, it is assumed that there are 3 MPMs, which will be referred to as MPM[0], MPM[1], and MPM[2]. When there are more than 3 MPMs, the MPMs may include the 3 MPMs described in the embodiments described later.

[0310] When candIntraPredA and candIntraPredB are the same and candIntraPredA is in planar mode or DC mode, MPM[0] and MPM[1] can be set to planar mode and DC mode respectively. MPM[2] can be set to vertical intra-prediction mode, horizontal intra-prediction mode or diagonal intra-prediction mode. Diagonal intra-prediction mode can be lower left diagonal intra-prediction mode, upper left intra-prediction mode or upper right intra-prediction mode.

[0311] When candIntraPredA and candIntraPredB are the same and candIntraPredA is in intra-prediction mode, MPM[0] can be set to be the same as candIntraPredA. MPM[1] and MPM[2] can be set to intra-prediction modes similar to candIntraPredA. Intra-prediction modes similar to candIntraPredA can be intra-prediction modes with an index difference of ±1 or ±2 from candIntraPredA. Intra-prediction modes similar to candIntraPredA can be derived using modulo operations (%) and offsets.

[0312] When candIntraPredA and candIntraPredB are different, MPM[0] can be set to be the same as candIntraPredA, and MPM[1] can be the same as candIntraPredB. In this case, when both candIntraPredA and candIntraPredB are non-angular intra-prediction modes, MPM[2] can be set to a vertical intra-prediction mode, a horizontal intra-prediction mode, or a diagonal intra-prediction mode. Alternatively, when at least one of candIntraPredA and candIntraPredB is an angular intra-prediction mode, MPM[2] can be set to an intra-prediction mode derived by adding or subtracting an offset value from the larger of the plane, DC, or candIntraPredA or candIntraPredB values. The offset value can be 1 or 2.

[0313] An MPM list containing multiple MPMs can be generated, and information indicating whether an MPM with the same intra-prediction mode as the current block is included in the MPM list can be transmitted via a bitstream signal. This information is a 1-bit flag, referred to as the MPM flag. When the MPM flag indicates that an MPM with the same mode as the current block is included in the MPM list, index information identifying one of the MPMs can be transmitted via a bitstream signal. The MPM specified by the index information can be set as the intra-prediction mode of the current block. When the MPM flag indicates that an MPM with the same mode as the current block is not included in the MPM list, residual mode information indicating any of the residual intra-prediction modes other than the MPM can be transmitted via a bitstream signal. The residual mode information represents the index value corresponding to the intra-prediction mode of the current block when the index is reallocated to residual intra-prediction modes other than the MPM. The decoder can sort the MPMs in ascending order and determine the intra-prediction mode of the current block by comparing the residual mode information with the MPMs. For example, when the residual mode information is the same as or smaller than the MPM, the intra-prediction mode of the current block can be derived by adding 1 to the residual mode information.

[0314] Instead of setting the default mode to MPM, information indicating whether the intra-prediction mode of the current block is the default mode can be signaled via a bitstream. This information is a 1-bit flag, which may be called the default mode flag. The default mode flag can be signaled only if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As mentioned above, the default mode can include at least one of planar, DC, vertical, or horizontal modes. For example, when planar is set as the default mode, the default mode flag can indicate whether the intra-prediction mode of the current block is planar. When the default mode flag indicates that the intra-prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information can be set as the intra-prediction mode of the current block.

[0315] When multiple intra-prediction modes are set as the default mode, index information indicating any of the default modes can be further sent using a signal. The intra-prediction mode of the current block can be set to the default mode indicated by the index information.

[0316] When the index of the reference sample line of the current block is not 0, the default mode is set not to be used. Therefore, when the index of the reference sample line is not 0, the default mode flag is not sent by signal, and the value of the default mode flag can be set to a predefined value (i.e., false).

[0317] If the intra-prediction mode of the current block is determined, prediction samples related to the current block can be obtained based on the determined intra-prediction mode (S2203).

[0318] When DC mode is selected, predicted samples related to the current block can be generated based on the average of reference samples. Specifically, the values ​​of all samples in the predicted block can be generated based on the average of reference samples. The average can be derived using at least one of the upper reference sample located above the current block and the left reference sample located to the left of the current block.

[0319] The number or range of reference samples used to derive the average may vary depending on the shape of the current block. For example, when the current block is a non-square block with a width greater than its height, the average can be calculated using only the top reference sample. On the other hand, when the current block is a non-square block with a width less than its height, the average can be calculated using only the left reference sample. That is, when the width and height of the current block are different, the average can be calculated using only the reference sample adjacent to the longer length. Alternatively, it can be determined whether to use only the top reference sample or only the left reference sample to calculate the average based on the width-to-height ratio of the current block.

[0320] When the planar mode is selected, prediction samples can be obtained using horizontal and vertical prediction samples. Specifically, the horizontal prediction sample is obtained based on left and right reference samples located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on upper and lower reference samples located on the same vertical line as the prediction sample. The right reference sample can be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample can be generated by copying the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample can be obtained based on a weighted sum of the left and right reference samples, and the vertical prediction sample can be obtained based on a weighted sum of the upper and lower reference samples. In this case, the weighting value assigned to each reference sample can be determined based on the position of the prediction sample. Prediction samples can also be obtained based on the average or weighted sum of the horizontal and vertical prediction samples. When performing a weighted sum operation, the weighting value assigned to the horizontal and vertical prediction samples can be determined based on the position of the prediction sample.

[0321] When an angle prediction mode is selected, parameters representing the prediction direction (or prediction angle) of the selected angle prediction mode can be determined. Table 2 below shows the intrapredAng parameter for each intrapredangling prediction mode.

[0322] Table 2

[0323]

[0324] Table 2 shows the intra-direction parameters for each intra-prediction mode with an index of any one of 2 to 34 when 35 intra-prediction modes are defined. When more than 33 angular intra-prediction modes are defined, Table 2 is further subdivided into setting the intra-direction parameters for each angular intra-prediction mode.

[0325] After aligning the top and left reference samples of the current block into a single row, predicted samples can be obtained based on the values ​​of the intra-frame orientation parameters. In this case, when the value of the intra-frame orientation parameter is negative, the left and top reference samples can be aligned into a single row.

[0326] Figure 25 and Figure 26 This is a diagram illustrating an example of a one-dimensional arrangement of reference samples in a row.

[0327] Figure 25 An example of a vertically oriented one-dimensional array of reference samples is shown, and Figure 26 An example of a horizontally oriented one-dimensional array of reference samples is shown. This will be described under the assumption of defining 35 intra-frame prediction modes. Figure 25 and 26 Examples of implementations.

[0328] When the intra-prediction mode index is any one of 11 to 18, a one-dimensional horizontal arrangement of the upper reference sample can be applied, rotating counterclockwise. When the intra-prediction mode index is any one of 19 to 25, a one-dimensional vertical arrangement of the left reference sample can be applied, rotating clockwise. The intra-prediction mode angle can be considered when arranging the reference samples in a row.

[0329] Reference sample determination parameters can be determined based on intra-frame orientation parameters. These parameters may include a reference sample index for specifying the reference sample and weighting parameters for determining the weights applied to the reference sample.

[0330] The reference sample index iIdx and the weighted parameter ifact can be obtained through Equations 17 and 18, respectively.

[0331] Equation 17

[0332] iIdx=(y+1)*P ang / 32

[0333] Equation 18

[0334] i fact =[(y+1)*P ang ]&31

[0335] In equations 17 and 18, P ang This represents the intra-frame orientation parameter. The reference sample specified by the reference sample index iIdx is equivalent to an integer pixel (Integer pel).

[0336] To derive predicted samples, more than one reference sample can be specified. Specifically, the slope of the prediction pattern can be considered to specify the location of the reference sample used to derive the predicted samples. For example, the reference sample index iIdx can be used to specify the reference sample used to derive the predicted samples.

[0337] In this scenario, when the slope of the intra-prediction mode is not represented by a single reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, when the slope of the intra-prediction mode is the value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample can be obtained by interpolating the first and second reference samples. That is, when the angular line following the intra-prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample can be obtained by interpolating the reference samples that are adjacent to the left, right, or top and bottom of the position through which the angular line passes.

[0338] Equation 19 below represents an example of obtaining a predicted sample based on a reference sample.

[0339] Equation 19

[0340] P(x,y)=((32-i fact ) / 32)*Ref_1D(x+iIdx+1)+(i fact / 32)*Ref_1D(x+iIdx+2)

[0341] In Equation 19, P represents the predicted sample, and Ref_1D represents any one of the reference samples in a one-dimensional arrangement. In this case, the position of the reference sample can be determined based on the position (x, y) of the predicted sample and the index iIdx of the reference sample.

[0342] When the slope of the intra-prediction mode can be represented by a reference sample, the weighting parameter ifact can be set to 0. Therefore, Equation 19 can be simplified to Equation 20 below.

[0343] Equation 20

[0344] P(x,y)=Ref_1D(x+iIdx+1)

[0345] Intra-prediction can also be performed on the current block based on multiple intra-prediction modes. For example, intra-prediction modes can be derived for different prediction samples, and prediction samples can be derived based on the intra-prediction modes assigned to each prediction sample.

[0346] Alternatively, intra-prediction modes can be derived for different regions, and intra-prediction can be performed on each region based on the intra-prediction modes assigned to each region. Each region may include at least one sample. The size or shape of the region can be adaptively determined based on at least one of the size, shape, or intra-prediction mode of the current block. Alternatively, at least one of the size or shape of the region can be predefined in the encoder and decoder, independent of the size or shape of the current block.

[0347] Alternatively, intra-prediction can be performed based on multiple intra-prediction methods, and the final prediction sample can be derived based on the average or weighted sum of multiple prediction samples obtained through multiple intra-prediction operations. For example, a first prediction sample can be obtained by performing intra-prediction based on a first intra-prediction mode, and a second prediction sample can be obtained by performing intra-prediction based on a second intra-prediction mode. Then, the final prediction sample can be obtained based on the average or weighted sum of the first and second prediction samples. In this case, the weighting values ​​assigned to the first and second prediction samples can be determined by considering at least one of whether the first intra-prediction mode is a non-angle / angle prediction mode, whether the second intra-prediction mode is a non-angle / angle prediction mode, or the intra-prediction modes of adjacent blocks.

[0348] Multiple intra-frame prediction modes can be a combination of non-angle intra-frame prediction modes and angle prediction modes, a combination of angle prediction modes, or a combination of non-angle prediction modes.

[0349] Figure 27 This is a diagram showing the angle formed between the prediction pattern within the angular frame and a straight line parallel to the x-axis.

[0350] like Figure 27 In the example shown, the angle prediction pattern can exist between the lower left diagonal direction and the upper right diagonal direction. When described as the angle formed by the x-axis and the angle prediction pattern, the angle prediction pattern can exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0351] If the current block is not square, the prediction sample is derived by using a reference sample that is farther away from the prediction sample from the reference samples located on the corner that follows the intra-prediction angle, rather than a reference sample that is closer to the prediction sample, depending on the intra-prediction mode of the current block.

[0352] Figure 28 This is a diagram showing an example of obtaining a predicted sample when the current block is not a square.

[0353] For example, as in Figure 28 In the example shown in (a), it is assumed that the current block is a non-square shape with a width greater than its height, and the intra-frame prediction mode of the current block is an angular intra-frame prediction mode with an angle between 0 and 45 degrees. In this case, when deriving the prediction sample A near the right column of the current block, a left reference sample L, which is far from the prediction sample in the angular mode located at the angle, is used instead of the upper reference sample T, which is close to the prediction sample.

[0354] As another example, such as in Figure 28In the example shown in (b), it is assumed that the current block is a non-square shape with a height greater than its width, and the intra-frame prediction mode of the current block is an angular intra-frame prediction mode with an angle between -90 degrees and -135 degrees. In the above case, when deriving the prediction sample A near the lower row of the current block, a situation occurs where an upper reference sample T, which is far from the prediction sample in the angular mode located at the angle, is used instead of a left reference sample L that is close to the prediction sample.

[0355] To address the above issue, when the current block is not square, the intra-prediction mode of the current block can be replaced with an intra-prediction mode in the opposite direction. Therefore, for non-square blocks, a mode with a higher frequency of prediction can be used. Figure 24 The angle prediction modes shown are for angles larger or smaller than the indicated angles. This type of intra-frame prediction mode can be defined as a wide-angle intra-frame prediction mode. A wide-angle intra-frame prediction mode refers to an intra-frame prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0356] Figure 29 This is a diagram illustrating the wide-angle intra-frame prediction mode.

[0357] exist Figure 29 In the example shown, the intra-prediction modes with indices -1 to -14 and the intra-prediction modes with indices 67 to 80 represent wide-angle intra-prediction modes.

[0358] Despite Figure 29 The diagram shows 14 wide-angle intra-prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra-prediction modes (67 to 80) with angles less than -135 degrees, but more or fewer wide-angle intra-prediction modes can be defined.

[0359] When using the wide-angle intra-frame prediction mode, the length of the upper reference sample is set to 2W+1, and the length of the left reference sample is set to 2H+1.

[0360] When using the wide-angle intra-frame prediction mode, a reference sample T can be used for prediction. Figure 28 (a) shows sample A, and can be predicted using reference sample L. Figure 28 Sample A is shown in (b).

[0361] By adding the existing intra-prediction modes to N wide-angle intra-prediction modes, a total of 67+N intra-prediction modes can be used. For example, Table 3 shows the intra-direction parameters of the intra-prediction modes when 20 wide-angle intra-prediction modes are defined.

[0362] Table 3

[0363] intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -1 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 11 12 13 14 15 16 17 18 intraPredAngle 13 11 9 7 5 3 2 1 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -1 -2 -3 -5 -7 -9 -11 -13 -15 PredModeIntra 28 29 30 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -11 -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -1 0 1 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 11 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

[0364] When the current block is not square and the intra-prediction mode of the current block obtained in step S2202 falls within the transformation range, the intra-prediction mode of the current block can be transformed into a wide-angle intra-prediction mode. The transformation range can be determined based on at least one of the size, shape, or ratio of the current block. The ratio can represent the ratio between the width and height of the current block.

[0365] When the current block is a non-square with a width greater than its height, the transformation range can be set from the index of the intra-prediction mode in the upper right diagonal direction (e.g., 66) to (the index of the intra-prediction mode in the upper right diagonal direction - N). Here, N can be determined based on the ratio of the current block. When the intra-prediction mode of the current block falls within the transformation range, the intra-prediction mode can be transformed into a wide-angle intra-prediction mode. The transformation can be performed by subtracting a predefined value from the intra-prediction mode; the predefined value can be the total number of intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 67).

[0366] According to the embodiment, the intra-frame prediction modes between the 66th and 53rd frames can be transformed into wide-angle intra-frame prediction modes between the -1st and -14th frames, respectively.

[0367] When the current block is a non-square with a height greater than its width, the transformation range can be set from the intra-prediction mode index in the lower left diagonal direction (e.g., 2) to (the index of the intra-prediction mode in the lower left diagonal direction + M). Here, M can be determined based on the ratio of the current block. When the intra-prediction mode of the current block falls within the transformation range, the intra-prediction mode can be transformed into a wide-angle intra-prediction mode. The transformation can be performed by adding a predefined value to the intra-prediction mode; the predefined value can be the total number of angular intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 65).

[0368] According to the embodiment, the intra-frame prediction modes between the 2nd and 15th frames can be transformed into wide-angle intra-frame prediction modes between the 67th and 80th frames, respectively.

[0369] Hereinafter, the intra-frame prediction modes that fall within the transform range will be referred to as wide-angle intra-frame replacement prediction modes.

[0370] The transform range can be determined based on the ratio of the current block. For example, Tables 4 and 5 show the transform range when 35 intra-prediction modes and 67 intra-prediction modes, excluding the wide-angle intra-prediction mode, are defined, respectively.

[0371] Table 4

[0372]

[0373]

[0374] Table 5

[0375] W / H = 2 Patterns 2, 3, 4, 5, 6, 7 W / H>2 Patterns 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H = 1 none H / W = 1 / 2 Patterns 61, 62, 63, 64, 65, 66 H / W<1 / 2 Patterns 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0376] As shown in the examples in Tables 4 and 5, the number of wide-angle intra-frame replacement prediction modes falling within the transform range can vary depending on the ratio of the current block.

[0377] With the use of a wide-angle intra prediction mode in addition to the existing intra prediction modes, the resources required for encoding the wide-angle intra prediction mode increase, potentially reducing coding efficiency. Therefore, instead of directly encoding the wide-angle intra prediction mode, it is proposed to encode an alternative intra prediction mode associated with the wide-angle intra prediction mode, thereby improving coding efficiency.

[0378] For example, when encoding the current block using the 67th wide-angle intra-prediction mode, the 67th wide-angle replacement intra-prediction mode (number 2) can be encoded as the intra-prediction mode for the current block. Conversely, when encoding the current block using the -1st wide-angle intra-prediction mode, the -1st wide-angle replacement intra-prediction mode (number 66) can be encoded as the intra-prediction mode for the current block.

[0379] The decoder can decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode is included in the transform range. When the decoded intra-prediction mode is a wide-angle replacement intra-prediction mode, it can transform the intra-prediction mode into a wide-angle intra-prediction mode.

[0380] Alternatively, when encoding the current block in wide-angle intra-prediction mode, wide-angle intra-prediction mode can also be encoded directly.

[0381] Encoding of the intra-prediction mode can be implemented based on the MPM list. The method for configuring the MPM list will be described in detail below. In the embodiments described below, it is assumed that 10 wide-angle intra-prediction modes (-1 to -10) with angles greater than 45 degrees and 10 wide-angle intra-prediction modes (67 to 76) with angles less than -135 degrees are defined.

[0382] When encoding adjacent blocks in wide-angle intra-prediction mode, the MPM can be set based on the wide-angle alternative intra-prediction mode corresponding to the wide-angle intra-prediction mode. For example, when adjacent blocks are encoded in wide-angle intra-prediction mode, the variable candIntraPredX (where X is A or B) can be set to the wide-angle alternative intra-prediction mode.

[0383] Alternatively, the method for deriving the MPM can be determined based on the shape of the current block. For example, when the current block is a square shape with the same width and height, candIntraPredX can be set to wide-angle replacement intra-prediction mode. On the other hand, when the current block is not a square shape, candIntraPredX can be set to wide-angle intra-prediction mode.

[0384] Alternatively, the decision to set `candIntraPredX` to a wide-angle intra-prediction mode can be determined based on whether the wide-angle intra-prediction modes of adjacent blocks are applicable to the current block. For example, if the current block is a non-square shape with a width greater than its height, the wide-angle intra-prediction mode with an index greater than the index of the intra-prediction mode in the upper right diagonal direction is set to `candIntraPredX` as is. However, for wide-angle intra-prediction modes with an index less than the index of the intra-prediction mode in the lower left diagonal direction, the corresponding wide-angle replacement intra-prediction mode is set to `candIntraPredX`. On the other hand, if the current block is a non-square shape with a height greater than its width, the wide-angle intra-prediction mode with an index less than the index of the intra-prediction mode in the lower left diagonal direction is set to `candIntraPredX` as is. However, for wide-angle intra-prediction modes with an index greater than the index of the intra-prediction mode in the upper right diagonal direction, the corresponding wide-angle replacement intra-prediction mode is set to `candIntraPredX`.

[0385] That is, based on whether the shape of the adjacent block encoded in the wide-angle intra-prediction mode is the same as or similar to the shape of the current block, it can be determined whether to use the wide-angle intra-prediction mode to derive the MPM or whether to use the wide-angle replacement intra-prediction mode to derive the MPM.

[0386] Alternatively, the wide-angle intra-prediction mode of adjacent blocks can be set to candIntraPredX, regardless of the shape of the current block.

[0387] In summary, candIntraPredX can be set to either the wide-angle intra-prediction mode for adjacent blocks or the wide-angle replacement intra-prediction mode.

[0388] MPMs can be derived based on candIntraPredA and candIntraPredB. In this case, MPMs can be derived in an intra-prediction mode similar to candIntraPredA or candIntraPredB. Intra-prediction modes similar to candIntraPredA or candIntraPredB can also be derived based on modulo operations and offsets. In this case, the constants and offsets used for modulo operations can be determined differently depending on the shape of the current block.

[0389] Table 6 shows an example of deriving the MPM based on the shape of the current block.

[0390] Table 6

[0391]

[0392]

[0393] Assume that `candIntraPredA` and `candIntraPredB` are identical, and `candIntraPredA` is an angular intra-prediction mode. When the current block is square, an intra-prediction mode similar to `candIntraPredA` can be obtained by modulo operation based on the value obtained by subtracting 1 from the total number of angular intra-prediction modes excluding the wide-angle intra-prediction mode. For example, when the number of angular intra-prediction modes excluding the wide-angle intra-prediction mode is 65, the MPM can be derived from the value obtained by modulo operation based on `candIntraPredA` and 64. On the other hand, when the current block is non-square, an intra-prediction mode similar to `candIntraPredA` can be obtained by modulo operation based on the value obtained by subtracting 1 from the total number of angular intra-prediction modes including the wide-angle intra-prediction mode. For example, when the number of wide-angle intra-prediction modes is 20, the MPM can be derived from the value obtained by modulo operation based on `candIntraPredA` and 84.

[0394] Because the constants used for modulo operations are set differently depending on the shape of the current block, it can be determined whether the wide-angle intra-prediction mode can be set to an angular intra-prediction mode similar to candIntraPredA. For example, in modulo operations using 64, the wide-angle intra-prediction mode may not be set to an angular intra-prediction mode similar to candIntraPredA, while in modulo operations using 84, the wide-angle intra-prediction mode can be set to an angular intra-prediction mode similar to candIntraPredA.

[0395] Alternatively, when candIntraPredA and candIntraPredB are the same, the shape of the current block and whether candIntraPredA is a wide-angle intra-prediction mode can be considered to derive the MPM.

[0396] Table 7 shows an example of deriving the MPM based on the shape of the current block.

[0397] Table 7

[0398]

[0399] Assume that candIntraPredA and candIntraPredB are the same.

[0400] When the current block is square and candIntraPredA is in wide-angle intra-prediction mode, MPM can be set to the default mode. For example, MPM[0], MPM[1] and MPM[2] can be set to planar mode, DC mode and vertical intra-prediction mode, respectively.

[0401] When the current block is square and candIntraPredA is an angular intra-prediction mode other than the wide-angle intra-prediction mode, the MPM can be set to an angular intra-prediction mode similar to candIntraPredA. For example, MPM[0] can be set to candIntraPredA, and MPM[1] and MPM[2] can be set to angular intra-prediction modes similar to candIntraPredA.

[0402] When the current block is not square and candIntraPredA is an angular intra-prediction mode, MPM can be set to an angular intra-prediction mode similar to candIntraPredA. For example, MPM[0] can be set to candIntraPredA, and MPM[1] and MPM[2] can be set to angular intra-prediction modes similar to candIntraPredA.

[0403] A modulo operation and offset can be used to derive an angular intra-prediction pattern similar to candIntraPredA. In this case, the constants used for the modulo operation may vary depending on the shape of the current block. Additionally, the offset used to derive the angular intra-prediction pattern similar to candIntraPredA can be set differently depending on the shape of the current block. For example, when the current block is a non-square shape with a width greater than its height, an offset of 2 can be used to derive an angular intra-prediction pattern similar to candIntraPredA. On the other hand, when the current block is a non-square shape with a height greater than its width, an offset of 2 and -8 can be used to derive an angular intra-prediction pattern similar to candIntraPredA.

[0404] Alternatively, the MPM can be derived by considering whether candIntraPredX is a wide-angle intra-prediction mode with the maximum or minimum index.

[0405] Table 8 shows an example of deriving the MPM by taking into account the wide-angle intra-frame prediction mode index.

[0406] Table 8

[0407]

[0408] Assume that candIntraPredA and candIntraPredB are the same. For ease of explanation, the wide-angle intra-prediction mode with an index value less than that of the intra-prediction mode in the lower left diagonal direction is called the downward wide-angle intra-prediction mode, and the wide-angle intra-prediction mode with an index value greater than that of the intra-prediction mode in the upper right diagonal direction is called the rightward wide-angle intra-prediction mode.

[0409] When candIntraPredA is a downward wide-angle intra-prediction mode, MPM can be set to an angle intra-prediction mode similar to candIntraPredA. In this case, when candIntraPredA is a downward wide-angle intra-prediction mode with a minimum value, MPM can be set to a downward wide-angle intra-prediction mode with a predefined index value. Here, the predefined index can be the index with the maximum value among the indices of the downward wide-angle intra-prediction mode. For example, when candIntraPredA is -10, MPM[0], MPM[1], and MPM[2] can be set to -10, -1, and -9, respectively.

[0410] When candIntraPredA is a right-facing wide-angle intra-prediction mode, MPM can be set to an angle intra-prediction mode similar to candIntraPredA. In this case, when candIntraPredA is a right-facing wide-angle intra-prediction mode with a maximum value, MPM can be set to a right-facing wide-angle intra-prediction mode with a predefined index value. Here, the predefined index can be the index with the minimum value among the indices of the right-facing wide-angle intra-prediction mode. For example, when candIntraPredA is 77, MPM[0], MPM[1], and MPM[2] can be set to 77, 76, and 67, respectively.

[0411] Alternatively, the MPM can be set to the default mode when the index obtained by subtracting 1 from the index of candIntraPredA is less than the minimum value among the indices of the intra-prediction modes, or when the index obtained by adding 1 is greater than the maximum value. Here, the default mode may include at least one of the following: planar mode, DC mode, vertical intra-prediction mode, horizontal intra-prediction mode, and diagonal intra-prediction mode.

[0412] Alternatively, when the index obtained by subtracting 1 from the index of candIntraPredA is less than the minimum value among the indices of the intra-prediction mode, or when the index obtained by adding 1 is greater than the maximum value, the MPM can be set to an intra-prediction mode opposite to candIntraPredA or an intra-prediction mode similar to the intra-prediction mode opposite to candIntraPredA.

[0413] Alternatively, the shape of the current block and the shapes of adjacent blocks can be considered to derive MPM candidates. For example, the method for deriving MPM when both the current block and adjacent blocks are non-square shapes may differ from the method for deriving MPM when the current block is square but adjacent blocks are non-square shapes.

[0414] The MPMs in the MPM list can be rearranged (or reordered) by considering at least one of the following: the size of the current block, the shape of the current block, the size of neighboring blocks, and the shape of neighboring blocks. Here, rearrangement means reassigning the index assigned to each MPM. For example, a smaller index can be assigned to an MPM with the same intra-prediction mode as a neighboring block that has the same size or shape as the current block.

[0415] Suppose that MPM[0] and MPM[1] are set to the intra-prediction mode candIntraPredA of the left adjacent block and the intra-prediction mode candIntraPredB of the upper adjacent block, respectively.

[0416] When the current block and the upper neighboring block are non-square shapes with a width greater than their height, the MPM can be rearranged so that the intra-prediction mode candIntraPredB of the upper neighboring block has a smaller index. That is, candIntraPredB can be rearranged as MPM[0], and candIntraPredA can be rearranged as MPM[1].

[0417] Alternatively, when the current block and the upper neighboring block are non-square shapes with a height greater than their width, the MPM can be rearranged such that the intra-prediction mode candIntraPredB of the upper neighboring block has a smaller index. That is, candIntraPredB can be rearranged to MPM[0], and candIntraPredA can be rearranged to MPM[1].

[0418] Alternatively, when the current block and the upper neighboring block are square-shaped, the MPM can be rearranged such that the intra-prediction mode candIntraPredB of the upper neighboring block has a smaller index. That is, candIntraPredB can be rearranged to MPM[0], and candIntraPredA can be rearranged to MPM[1].

[0419] Instead of rearranging the MPM, when candIntraPredX is initially assigned to the MPM, at least one of the following can be considered: the size of the current block, the shape of the current block, the size of the adjacent blocks, and the shape of the adjacent blocks.

[0420] The MPM can be rearranged based on the size or shape of the current block. For example, if the current block is a non-square shape with a width greater than its height, the MPM can be rearranged in descending order. On the other hand, if the current block is a non-square shape with a height greater than its width, the MPM can be rearranged in ascending order.

[0421] The resulting residual image can be derived by subtracting the predicted image from the original image. In this case, transforming the residual image into the frequency domain does not significantly degrade the subjective image quality of the video, even if high-frequency components are removed. Therefore, converting the values ​​of high-frequency components to smaller values ​​or setting them to 0 can improve compression efficiency without causing significant visual distortion. Reflecting these characteristics, the current block can be transformed to decompose the residual image into 2D frequency components. This transformation can be performed using techniques such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST).

[0422] DCT uses a cosine transform to decompose (or transform) the residual image into 2D frequency components, while DST uses a sine transform to decompose (or transform) the residual image into 2D frequency components. As the transformed result of the residual image, the frequency components can be represented as the basic image. For example, when performing a DCT transform on a block of size N×N, N can be obtained. 2 There are several basic pattern components. The sizes of each basic pattern component in an N×N block can be obtained through transformation. Depending on the transformation technique used, the sizes of the basic pattern components can be called DCT coefficients or DST coefficients.

[0423] The Direct Transformation Technique (DCT) is primarily used to transform images with a high proportion of low-frequency non-zero components. The Direct Transformation Technique (DST) is primarily used for images with a high proportion of high-frequency components.

[0424] Transformation techniques other than DCT or DST can also be used to transform residual images.

[0425] The process of transforming the residual image into two-dimensional frequency components is referred to as a two-dimensional image transformation. Furthermore, the magnitudes of the fundamental pattern components obtained through the transformation are called transformation coefficients. For example, transformation coefficients can refer to DCT coefficients or DST coefficients. When the primary and secondary transformations, which will be described later, are applied simultaneously, the transformation coefficients can represent the magnitudes of the fundamental pattern components generated by the result of the secondary transformation.

[0426] Transform techniques can be determined on a block-by-block basis. A transform technique can be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, when the current block is coded in intra-predictive mode and the size of the current block is less than N×N, the transform technique DST can be used to perform the transform. On the other hand, when the aforementioned conditions cannot be met, the transform technique DCT can be used to perform the transform.

[0427] In the residual image, a portion of the image may not undergo 2D image transformation. This omission of 2D image transformation is called transform skipping. When transform skipping is applied, quantization can be applied to the residual values ​​for which no transformation was performed.

[0428] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the DCT- or DST-based transformation can be defined as the primary transformation, and the process of transforming the block again using the primary transformation is called the secondary transformation.

[0429] The main transform can be performed using any of a number of transform kernel candidates. For example, the main transform can be performed using any of DCT2, DCT8, or DCT7.

[0430] Different transform cores can be used for the horizontal and vertical directions. Information representing combinations of horizontal and vertical transform cores can also be transmitted as signals via bitstreams.

[0431] The execution units for the primary and secondary transformations will differ. For example, a primary transformation can be performed on an 8×8 block, and a secondary transformation can be performed on 4×4 sub-blocks within the transformed 8×8 block. In this case, the transformation coefficients of the remaining regions where the secondary transformation is not performed can also be set to 0.

[0432] Alternatively, a primary transformation can be performed on a 4×4 block, and a secondary transformation can be performed on an 8×8 region of the 4×4 block that includes the transformation.

[0433] Information indicating whether a second transformation should be performed can be sent via a bitstream signal.

[0434] The inverse of the second inverse transform (second inverse transform) can be performed in the decoder, and the inverse of the main transform (first inverse transform) can be performed on its result. The residual signal of the current block can be obtained as the result of the second inverse transform and the first inverse transform.

[0435] Quantization is used to reduce the energy of the block, and the quantization process involves dividing the transformation coefficients by a specific constant. This constant can be derived from quantization parameters, which can be defined as values ​​between 1 and 63.

[0436] If transform and quantization are performed in the encoder, the decoder can obtain the residual block through inverse quantization and inverse transform. The decoder can obtain the reconstructed block of the current block by adding the predicted block and the residual block.

[0437] If a reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss during quantization and encoding. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter will be referred to as the first reconstructed block, and the reconstructed block after applying the in-loop filter will be referred to as the second reconstructed block.

[0438] A second reconstructed block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block. In this case, SAO or ALF can be applied after the deblocking filter.

[0439] Deblocking filters are used to mitigate the image quality degradation (blocking artifact) that occurs at block boundaries when quantization is performed on a block-by-block basis. To apply a deblocking filter, the block strength (BS) between the first reconstructed block and its adjacent reconstructed blocks can be determined.

[0440] Figure 30 This is a flowchart illustrating the process of determining block strength.

[0441] like Figure 30 In the example shown, P represents the first reconstructed block, and Q represents the adjacent reconstructed block. The adjacent reconstructed block can be adjacent to the left or top of the current block.

[0442] exist Figure 30The example shown illustrates how to determine block strength by considering the predictive coding patterns of P and Q, whether non-zero transform coefficients are included, whether the same reference image is used to perform inter-frame prediction, and whether the difference in motion vectors is greater than or equal to a threshold.

[0443] Based on the block strength, it can be determined whether a deblocking filter has been applied. For example, when the block strength is 0, filtering may not be performed.

[0444] SAO (Sound Analysis and Offset) is used to mitigate the ringing artifact that occurs when performing quantization in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. Methods for determining the offset include edge offset (EO) or band offset. EO indicates a method of determining the offset of the current sample based on the pattern of surrounding pixels. BO indicates a method of applying a common offset to a set of pixels with similar brightness values ​​within a region. Specifically, pixel brightness can be divided into 32 equal intervals, and pixels with similar brightness values ​​can be grouped together. For example, four adjacent bands out of the 32 bands can be grouped together, and the same offset can be applied to samples belonging to the four bands.

[0445] ALF is a method for generating a second reconstructed image by applying a predefined filter of size / shape to a first reconstructed image or a reconstructed image with a deblocking filter applied. Equation 21 below shows an example of ALF application.

[0446] Equation 21

[0447]

[0448] You can select any of the predefined filter candidates at the image, coding tree unit, coding block, prediction block, or transform block level. Each filter candidate can have a different size or shape.

[0449] Figure 31 This is a graph showing predefined filter candidates.

[0450] As in Figure 31 In the example shown, at least one of the following rhombuses can be selected: 5×5, 7×7, and 9×9.

[0451] Only 5×5 rhombuses can be used for chromaticity components.

[0452] To encode high-resolution video, such as panoramic video, 360-degree video, or 4K / 8K UHD (ultra-high definition), an image can be divided into multiple regions and encoded / decoded in parallel. For this purpose, the image can be divided into tiles (i.e., the basic unit of parallel encoding / decoding), and these tiles can be processed in parallel.

[0453] Tiles can be constrained to have a rectangular shape. Data from other tiles is not used when encoding / decoding tiles. A probability table for the context adaptive binary arithmetic coding (CABAC) context can be initialized on a tile-by-tile basis and can be configured not to apply loop filters at tile boundaries.

[0454] Figure 32 An example of dividing an image into multiple tiles is shown.

[0455] A tile includes at least one coding tree unit, and the boundary of the tile coincides with the boundary of the coding tree unit.

[0456] As in Figure 32 In the example shown, the image can be divided into multiple tile sets. Information for dividing the image into multiple tile sets can be transmitted via a bitstream using signals.

[0457] Depending on the image's partitioning type, tiles can have the same size in all regions except for the image boundaries.

[0458] Alternatively, the image can be divided into horizontally adjacent tiles with the same height, or the image can be divided into vertically adjacent tiles with the same width.

[0459] When an image is divided using at least one of a vertical or horizontal line that intersects the image, each tile belongs to a different column and / or row. In the exemplary embodiments described below, the column to which a tile belongs is referred to as a tile column, and the row to which a tile belongs is referred to as a tile row.

[0460] Information used to determine the shape of an image divided into tiles can be transmitted as a bitstream using signals. This information can be encoded using an image parameter set or a sequence parameter set and transmitted as signals. This information is used to determine the number of tiles in the image and can include information indicating the number of tile rows and information indicating the number of tile columns. For example, the syntax element `num_tile_columns_minus1` indicates the value obtained by subtracting 1 from the number of tile columns, and the syntax element `num_tile_rows_minus1` indicates the value obtained by subtracting 1 from the number of tile rows.

[0461] exist Figure 32In the example shown, since the number of tile columns is 4 and the number of tile rows is 3, num_tile_columns_minus1 can be 3 and num_tile_rows_minus1 can be 2.

[0462] When an image is divided into multiple tiles, information indicating the tile size can be sent via a bitstream using signals. For example, when an image is divided into multiple tile columns, information indicating the width of each tile column can be sent via a bitstream using signals, and when an image is divided into multiple tile rows, information indicating the height of each tile row can be sent via a bitstream using signals. For example, for each tile column, the syntax element `column_width_minus1` indicating the width of the tile column can be encoded and sent via signals, and for each tile row, the syntax element `row_height_minus1` indicating the height of the tile row can be encoded and sent via signals.

[0463] column_width_minus1 can indicate the value obtained by subtracting 1 from the width of the tile column, and row_height_minus1 can indicate the value obtained by subtracting 1 from the height of the tile row.

[0464] For the last tile column, the encoding of column_width_minus1 can be omitted, and for the last tile row, the encoding of row_height_minus1 can be omitted. The image size can be considered to derive the width of the last tile column and the height of the last row.

[0465] The decoder can determine the size of the tile based on column_width_minus1 and row_height_minus1.

[0466] Table 9 shows the syntax table for dividing an image into tiles.

[0467] Table 9

[0468]

[0469]

[0470] Referring to Table 9, the syntax element num_tile_columns_minus1, which indicates the number of tile columns, and the syntax element num_tile_rows_minus1, which indicates the number of tile rows, can be sent using signals.

[0471] Next, the syntax element `uniform_spacing_flag` can be used to signal whether the image is divided into tiles of equal size. When `uniform_spacing_flag` is true, tiles in the remaining regions except the image boundaries can be divided into tiles of equal size.

[0472] When uniform_spacing_flag is false, the syntax element column_width_minus1, which indicates the width of each tile column, and the syntax element row_height_minus1, which indicates the height of each tile row, can be signaled.

[0473] The syntax element loop_filter_across_tiles_enabled_flag indicates whether loop filters are allowed at tile boundaries.

[0474] A tile column with the minimum width in a tile column can be called a minimum-width tile, and a tile row with the minimum height in a tile row can be called a minimum-height tile. Information indicating the width of the minimum-width tile and information indicating the height of the minimum-height tile can be sent via bitstream signals. For example, the syntax element `min_column_width_minus1` indicates the value obtained by subtracting 1 from the width of the minimum-width tile, and the syntax element `min_row_height_minus1` indicates the value obtained by subtracting 1 from the height of the minimum-height tile.

[0475] For each tile column, information indicating the difference from the minimum tile width can be sent using signals. For example, the syntax element `diff_column_width` indicates the width difference between the current tile column and the minimum tile column. The width difference can be represented as the difference in the number of coding tree unit columns. The decoder can derive the width of the current tile by adding the width of the minimum width tile derived from `min_column_width_minus1` to the width difference derived from `diff_column_width`.

[0476] Additionally, for each tile row, information indicating the difference in height from the minimum tile height can be sent using signals. For example, the syntax element `diff_row_height` indicates the height difference between the current tile row and the minimum tile row. The height difference can be represented as the difference in the number of coding tree unit rows. The decoder can derive the height of the current tile by adding the height of the minimum-height tile derived from `min_row_height_minus1` to the height difference derived from `diff_row_height`.

[0477] Table 10 shows a syntax table that includes information about size differences.

[0478] Table 10

[0479]

[0480] An image can be divided such that horizontally adjacent tiles have different heights, or it can be divided such that vertically adjacent tiles have different widths. These image division methods can be called flexible tile division methods, and tiles divided using flexible tile division methods can be called flexible tiles.

[0481] Figure 33 This is a diagram illustrating the image partitioning pattern based on the flexible tile technique.

[0482] The search order of tiles generated by dividing an image can follow a predetermined scan order. Additionally, an index can be assigned to each tile according to the predetermined scan order.

[0483] The scanning order of the tiles can be any of raster scanning, diagonal scanning, vertical scanning, or horizontal scanning. Figure 33 (a) to Figure 33 (d) shows an example of assigning an index to each tile based on raster scan, diagonal scan, vertical scan, and horizontal scan, respectively.

[0484] The next scan order can be determined based on the size or position of the current tile. For example, when the height of the current tile is different from the height of the tile adjacent to the right of the current tile (e.g., when the height of the right adjacent tile is greater than the height of the current tile), the leftmost tile on the same vertical line as the tile adjacent below the current tile can be identified as the scan target after the current tile.

[0485] The scanning order of tiles can be determined on an image or sequence basis.

[0486] Alternatively, the size of the first tile in the image can be considered to determine the scanning order of the tiles. For example, when the width of the first tile is greater than its height, the scanning order can be set to horizontal scanning. When the height of the first tile is greater than its width, the scanning order can be set to vertical scanning. When the width and height of the first tile are the same, the scanning order can be set to raster scanning or diagonal scanning.

[0487] Information indicating the total number of tiles can be sent via a bitstream signal. For example, when applying flexible tile technology, the syntax element `number_of_tiles_in_picture_minus2`, derived by subtracting 2 from the total number of tiles in the image, can be sent via signal. The decoder can then identify the number of tiles included in the current image based on `number_of_tiles_in_picture_minus2`.

[0488] Table 11 shows a syntax table that includes information about the number of tiles.

[0489] Table 11

[0490]

[0491] To reduce the number of bits required to encode the size of a tile, information indicating the size of a sub-tile can be encoded and transmitted via signals. A sub-tile is the basic unit that makes up a tile, and each tile can be configured to include at least one sub-tile. A sub-tile can include one or more coding tree units.

[0492] For example, the syntax element `subtile_width_minus1` indicates the value obtained by subtracting 1 from the width of the subtile. The syntax element `subtile_height_minus1` indicates the value obtained by subtracting 1 from the height of the subtile.

[0493] Information indicating whether tiles other than the first tile have the same size as the previous tile can be encoded and sent as a signal. For example, the syntax element `use_previous_tile_size_flag` indicates whether the current tile's size is the same as the previous tile's size. When `use_previous_tile_size_flag` is true, it indicates that the current tile's size is the same as the previous tile's size. When `use_previous_tile_size_flag` is false, information indicating the current tile's size can be encoded and sent as a signal. For the first tile, the encoding of `use_previous_tile_size_flag` can be omitted, and the flag's value can be set to false.

[0494] Information indicating the size of a tile may include the syntax element tile_width_minus1[i] indicating the width of the i-th tile and the syntax element tile_height_minus1[i] indicating the height of the i-th tile.

[0495] Information indicating tile size can indicate the difference between the size of the tiles and the size of the sub-tiles. When using sub-tile size information, encoding / decoding efficiency can be improved by reducing the number of bits required to encode the size of each tile. For example, the width of the i-th tile, `tileWidth`, can be derived based on Equation 22, and the height of the i-th tile, `tileHeight`, can be derived based on Equation 23.

[0496] Equation 22

[0497] tileWidth=(subtile_width_minus1+1)*(tile_width_minus1[i]+1)

[0498] Equation 23

[0499] tileHeight=(subtile_height_minus1+1)*(tile_height_minus1[]+1)

[0500] Alternatively, the encoding of sub-tile size information can be omitted, and the size of the i-th tile can be encoded as is. Encoding of sub-tile size information is optional. Information indicating whether sub-tile size information has been encoded can be transmitted via signals using a video parameter set, sequence parameter set, or image parameter set.

[0501] Information related to tile size can be encoded as an indication of the number of coding tree units and transmitted via signals. For example, column_width_minus1, min_column_width_minus1, subtile_width_minus1, and tile_width_minus1 can indicate the number of columns of coding tree units included in a tile. Additionally, diff_column_width can indicate the difference between the number of columns of coding tree units included in the minimum width tile and the number of columns of coding tree units included in the current tile.

[0502] Additionally, `row_height_minus1`, `min_row_height_minus1`, `subtile_height_minus1`, and `tile_height_minus1` can indicate the number of coding tree cell rows included in a tile. Furthermore, `diff_row_height` can indicate the difference between the number of coding tree cell rows included in the minimum height tile and the number of coding tree cell rows included in the current tile.

[0503] The decoder can determine the tile size based on the number of columns and / or rows of coding tree units derived from the syntax elements, as well as the size of the coding tree units. For example, the width of the i-th tile can be set to (tile_width_minus1[i]+1)*(width of the coding tree unit), and the height of the i-th tile can be set to (tile_height_minus1[i]+1)*(height of the coding tree unit).

[0504] At the same time, information indicating the size of the coding tree unit can be sent by signaling through sequence parameter sets or image parameter sets.

[0505] Table 11 describes the use of the syntax element `use_previous_tile_size_flag` to indicate whether the size of the current tile is the same as the size of the previous tile. As another example, information indicating whether the width of the current tile is the same as the width of the previous tile or information indicating whether the height of the current tile is the same as the height of the previous tile can be encoded and sent as a signal.

[0506] Table 12 shows a syntax table that includes information indicating whether the width of the current tile is the same as the width of the previous tile.

[0507] Table 12

[0508]

[0509] The syntax element `use_previous_tile_width_flag` indicates whether the width of the current tile is the same as the width of the previous tile. When `use_previous_tile_width_flag` is true, the width of the current tile can be set to be equal to the width of the previous tile. In this case, the encoding of the information indicating the width of the current tile can be omitted, and the width of the current tile can be derived from the width of the previous tile.

[0510] When use_previous_tile_width_flag is false, information indicating the width of the current tile can be sent using a signal. For example, tile_width_minus1[i] can indicate the value obtained by subtracting 1 from the width of the i-th tile.

[0511] The syntax element use_previous_tile_width_flag may be encoded and signaled only when it is determined that the size of the current tile is different from the size of the previous tile (for example, when the value of use_previous_tile_size_flag is 0).

[0512] `tile_width_minus1[i]` can have a value obtained by subtracting 1 from the number of columns of the coding tree units included in the i-th tile. The decoder can derive the number of columns of the coding tree units belonging to the i-th tile by adding 1 to `tile_width_minus1[i]`, and multiply the derived value by the width of the coding tree units to calculate the tile width.

[0513] Table 13 shows a syntax table that further includes information indicating whether the height of the current tile is the same as the height of the previous tile.

[0514] Table 13

[0515]

[0516]

[0517] The syntax element `use_previous_tile_height_flag` indicates whether the height of the current tile is the same as the height of the previous tile. When `use_previous_tile_height_flag` is true, the height of the current tile can be set to be equal to the height of the previous tile. In this case, the encoding of the information indicating the height of the current tile can be omitted, and the height of the current tile can be derived from the height of the previous tile.

[0518] When use_previous_tile_height_flag is false, information indicating the height of the current tile can be sent using a signal. For example, tile_height_minus1[i] can indicate the value obtained by subtracting 1 from the height of the i-th tile.

[0519] The syntax element `use_previous_tile_height_flag` may only be encoded and signaled if it is determined that the size of the current tile is different from the size of the previous tile (e.g., when the value of `use_previous_tile_size_flag` is 0). Furthermore, the syntax element `use_previous_tile_height_flag` is only signaled when `use_previous_tile_width_flag` is false.

[0520] Table 12 shows an example of using use_previous_tile_width_flag, and Table 13 shows an example of using both use_previous_tile_width_flag and use_previous_tile_height_flag. Although not shown in the tables above, the encoding of use_previous_tile_width_flag can be omitted, and only use_previous_tile_height_flag can be used.

[0521] The choice between `use_previous_tile_height_flag` and `use_previous_tile_size_flag` can be determined based on at least one of the tile scan order, the width and height of the first tile, and the width and height of the preceding tile. For example, `use_previous_tile_height_flag` can be used when the tile scan order is vertical, while `use_previous_tile_width_flag` can be used when the tile scan order is horizontal. Alternatively, `use_previous_tile_width_flag` can be used when the first or preceding tile is a non-square shape with a width greater than its height, and `use_previous_tile_height_flag` can be used when the first or preceding tile is a non-square shape with a height greater than its width.

[0522] When transmitting the number of tiles included in an image using signals, the encoding of information related to the tile size can be omitted for the last tile.

[0523] Table 14 shows an example of omitting the encoding of tile size information for the last tile.

[0524] Table 14

[0525]

[0526]

[0527] When the tile size is specified except for the last tile, the remaining area in the image can be set as the last tile.

[0528] For each coding tree unit, an identifier (hereinafter referred to as TileID) can be assigned to identify the tile to which the coding tree unit belongs.

[0529] Figure 34This is a diagram illustrating an example of assigning tile IDs to each coding tree unit.

[0530] The same tile ID can be assigned to coding tree units belonging to the same tile. Specifically, the Nth TileID can be assigned to a coding tree unit belonging to tile N.

[0531] To determine the tile ID assigned to each coding tree unit, variables x and y indicating the position of the coding tree unit in the image can be determined. Here, x represents the value obtained by dividing the x-axis coordinate of the position (x0, y0) of the top-left sample of the coding tree unit by the width of the coding tree unit, and y represents the value obtained by dividing the y-axis coordinate of the position (x0, y0) of the top-left sample of the coding tree unit by the height of the coding tree unit. Specifically, x and y can be derived using the following equations 24 and 25.

[0532] Equation 24

[0533] x = (x0 / (CTU width))

[0534] Equation 25

[0535] Y = (y0 / (CTU height))

[0536] The operation of assigning tile IDs to each coding tree unit can be performed through the process described below.

[0537] i) Initialize tile ID

[0538] The tile ID of each coding tree unit can be initialized to a value obtained by subtracting 1 from the number of tiles in the image.

[0539] Table 15

[0540]

[0541] ii) Export tile ID

[0542] Table 16

[0543]

[0544] In the above embodiments, a flag indicating whether a loop filter is allowed to be applied at tile boundaries has been described by signaling via an image parameter set. However, if the setting is configured to not use a loop filter at any tile boundaries, subjective image quality degradation and coding efficiency may occur.

[0545] Therefore, information indicating whether a loop filter is allowed to be applied to each tile can be encoded and sent as a signal.

[0546] Figure 35 This is a diagram illustrating an example of selectively determining whether to apply a loop filter to each tile.

[0547] As in Figure 35 In the example shown, it can be determined for each tile whether a loop filter (e.g., deblocking filter, SAO, and / or ALF) is allowed to be applied at the horizontal or vertical boundaries.

[0548] Table 17 shows an example of how information indicating whether a loop filter is allowed to be applied is encoded for each tile.

[0549] Table 17

[0550]

[0551] In the examples in Table 17, the syntax element `loop_filter_across_tiles_flag[i]` indicates whether a loop filter is allowed to be applied to the i-th tile. A value of 1 for `loop_filter_across_tile_flag[i]` indicates that a loop filter can be used at the horizontal and vertical boundaries of the tile with tile ID i. A value of 0 for `loop_filter_across_tile_flag[i]` indicates that a loop filter is not used at the horizontal and vertical boundaries of the tile with tile ID i.

[0552] Information indicating whether a loop filter is allowed to be applied in each of the horizontal and vertical directions can be encoded.

[0553] Table 18 shows examples of encoding information indicating whether a loop filter is allowed in the horizontal and vertical directions, respectively.

[0554] Table 18

[0555]

[0556] In the examples in Table 18, the syntax element `loop_filter_hor_across_tiles_flag[i]` indicates whether a loop filter is allowed at the location where it intersects with the i-th tile in the horizontal direction. The syntax element `loop_filter_ver_across_tiles_flag[i]` indicates whether a loop filter is allowed at the location where it intersects with the i-th tile in the vertical direction.

[0557] A value of 1 for loop_filter_hor_across_tile_flag[i] indicates that a loop filter can be used at the horizontal boundary of the tile with tile ID i. A value of 0 for loop_filter_hor_across_tile_flag[i] indicates that a loop filter is not used at the vertical boundary of the tile with tile ID i.

[0558] A value of 1 for loop_filter_ver_across_tile_flag[i] indicates that a loop filter can be used at the vertical boundary of the tile with tile ID i, and a value of 0 for loop_filter_ver_across_tile_flag[i] indicates that a loop filter is not used at the vertical boundary of the tile with tile ID i.

[0559] Alternatively, information indicating whether a group of tiles comprising multiple tiles is permitted to have a loop filter applied can be encoded and transmitted as a signal. This information can then be used to determine whether the multiple tiles included in the group are permitted to have a loop filter applied.

[0560] To determine a tile group, at least one of the following can be signaled via a bitstream: the number of tiles belonging to the tile group, the size of the tile group, and image segmentation information. Alternatively, predefined sized regions in the encoder and decoder can be set as tile groups.

[0561] The encoding of information indicating whether a loop filter is allowed can be omitted, and the permission to apply a loop filter can be determined based on at least one of the number of coding tree units included in the tile, the width of the tile, and the height of the tile. For example, when the tile width is less than a reference value, a loop filter can be allowed to be applied in the horizontal direction, while when the tile height is less than a reference value, a loop filter can be applied in the vertical direction.

[0562] When using a loop filter at tile boundaries, reconstructed data outside the tile can be generated based on data included within the tile. In this case, the reconstructed video outside the tile can be obtained by padding or interpolating the data included within the tile. The loop filter can then be applied using this reconstructed data from outside the tile.

[0563] Examples described focusing on the decoding or encoding process are also included within the scope of this invention. Variations of multiple embodiments described in a predetermined order, in a different order than those described, are also included within the scope of this invention.

[0564] The embodiments of the present invention also disclose at least the following solutions:

[0565] Solution 1. A video decoding method, comprising the following steps:

[0566] Generate a list of merge candidates for the current block; specify any one of the multiple merge candidates included in the list;

[0567] Based on the first and second affine seed vectors of the specified merge candidates, derive the first and second affine seed vectors of the current block;

[0568] Affine vectors of sub-blocks within the current block are derived using the first and second affine seed vectors of the current block, wherein the sub-blocks are regions smaller than the size of the current block; and

[0569] Motion compensation prediction is performed on the sub-block based on the affine vector, wherein,

[0570] The first and second affine seed vectors of the merging candidate are derived based on the motion information of the neighboring blocks adjacent to the current block.

[0571] Option 2. The video decoding method according to Option 1, wherein,

[0572] When the adjacent block is included in a coding tree unit that is different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merging candidate are derived based on the motion vectors of the lower left and lower right sub-blocks of the adjacent block.

[0573] Option 3. The video decoding method according to Option 2, wherein,

[0574] The lower left sub-block includes a lower left reference sample located at the lower left corner of the adjacent block, and the lower right sub-block includes a lower right reference sample located at the lower right corner of the adjacent block.

[0575] Option 4. The video decoding method according to Option 3, wherein,

[0576] The first and second affine seed vectors of the merging candidates are derived using a scaling factor based on a shift operation of the difference between the motion vectors of the lower left sub-block and the lower right sub-block.

[0577] Furthermore, the scaling factor is derived based on the value obtained by adding the horizontal distance between the lower left reference sample and the lower right reference sample to the offset.

[0578] Option 5. The video decoding method according to Option 3, wherein,

[0579] The first and second affine seed vectors of the merged candidate are derived using a scaling factor based on a shift operation of the difference between the motion vectors of the lower left sub-block and the lower right sub-block, and the scaling factor is derived based on the distance between the adjacent sample to the right of the lower right reference sample and the lower left reference sample.

[0580] Solution 6. The video decoding method according to Solution 1, wherein,

[0581] The merge candidate list includes a first merge candidate and a second merge candidate. The first merge candidate is derived from the upper neighbor of the first available block in the upper neighbor block determined to be above the current block, and the second merge candidate is derived from the left neighbor of the first available block in the left neighbor block determined to be to the left of the current block.

[0582] Solution 7. The video decoding method according to Solution 1, wherein,

[0583] When the adjacent block is included in the same coding tree unit as the current block,

[0584] The first and second affine seed vectors of the merged candidate are derived based on the first and second affine seed vectors of the adjacent blocks.

[0585] Option 8. A video encoding method, comprising the following steps:

[0586] Generate a list of merge candidates for the current block; specify any one of the multiple merge candidates included in the list;

[0587] Based on the first and second affine seed vectors of the specified merge candidates, derive the first and second affine seed vectors of the current block;

[0588] Affine vectors of sub-blocks within the current block are derived using the first and second affine seed vectors of the current block, wherein the sub-blocks are regions smaller than the size of the current block; and

[0589] Motion compensation prediction is performed on the sub-block based on the affine vector, wherein,

[0590] The first and second affine seed vectors of the merging candidate are derived based on the motion information of the neighboring blocks adjacent to the current block.

[0591] Option 9. The video coding method according to Option 8, wherein,

[0592] When the adjacent block is included in a coding tree unit that is different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merging candidate are derived based on the motion vectors of the lower left and lower right sub-blocks of the adjacent block.

[0593] Option 10. The video coding method according to Option 9, wherein,

[0594] The lower left sub-block includes a lower left reference sample located at the lower left corner of the adjacent block, and the lower right sub-block includes a lower right reference sample located at the lower right corner of the adjacent block.

[0595] Solution 11. The video coding method according to Solution 10, wherein,

[0596] The first and second affine seed vectors of the merging candidates are derived using a scaling factor based on a shift operation of the difference between the motion vectors of the lower left sub-block and the lower right sub-block.

[0597] Furthermore, the scaling factor is derived based on the value obtained by adding the horizontal distance between the lower left reference sample and the lower right reference sample to the offset.

[0598] Option 12. The video coding method according to Option 10, wherein,

[0599] The first and second affine seed vectors of the merging candidates are derived using a scaling factor based on a shift operation of the difference between the motion vectors of the lower left sub-block and the lower right sub-block.

[0600] Furthermore, the scaling factor is derived based on the distance between the adjacent sample to the right of the lower right reference sample and the lower left reference sample.

[0601] Option 13. The video coding method according to Option 8, wherein,

[0602] The merge candidate list includes a first merge candidate and a second merge candidate. The first merge candidate is derived from the upper neighbor of the first available block in the upper neighbor block determined to be above the current block, and the second merge candidate is derived from the left neighbor of the first available block in the left neighbor block determined to be to the left of the current block.

[0603] Option 14. The video coding method according to Option 8, wherein,

[0604] When the adjacent block is included in the same coding tree unit as the current block,

[0605] The first and second affine seed vectors of the merged candidate are derived based on the first and second affine seed vectors of the adjacent blocks.

[0606] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention, and they can be performed simultaneously or in a different order as needed. Furthermore, in the above embodiments, the structural elements constituting the block diagrams (e.g., units, modules, etc.) can also be implemented as hardware devices or software, and multiple structural elements can be combined to implement a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc., individually or in combination. Examples of computer-readable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. The hardware device can be configured to operate as one or more software modules to perform the processing according to the invention, and vice versa.

[0607] [Industrial Applicability]

[0608] This invention can be applied to electronic devices that encode / decode video.

Claims

1. A video decoding method, comprising: Generate a list of merge candidates for the current block; Specify one merge candidate from the plurality of merge candidates included in the merge candidate list; Based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate, derive the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block. Using the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block, derive the affine vectors of the sub-blocks in the current block, wherein the sub-blocks are regions smaller than the current block; as well as Based on the affine vector, motion compensation prediction is performed on the sub-block, wherein... The first affine seed vector (nv0) and the second affine seed vector (nv1) of a merge candidate are derived based on the motion information of neighboring blocks adjacent to the current block. The adjacent blocks are included in coding tree units different from those of the current block, and the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate are derived based on the motion vectors of the lower left and lower right sub-blocks of the adjacent blocks. The lower left sub-block includes the lower left affine reference sample control point (x) located at the lower left corner of the adjacent block. n4 , y n4 The lower right sub-block and the lower right affine reference sample control point (x) located to the right of the lower right sample of the lower right sub-block. n5 , y n5 (Adjacent) 2. The method according to claim 1, wherein, The first and second affine seed vectors of a merge candidate are derived from values ​​obtained by applying a shift operation to the width between the lower left and lower right sub-blocks, wherein the shift operation shifts the width using a scaling factor.

3. The method according to claim 1, wherein, The width between the lower left sub-block and the lower right sub-block is defined as a variable, which is the width of the sub-seed vector set to a power of 2.

4. A video encoding method, comprising: Generate a list of merge candidates for the current block; Specify one merge candidate from the plurality of merge candidates included in the merge candidate list; Based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate, derive the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block. Using the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block, derive the affine vectors of the sub-blocks in the current block, wherein the sub-blocks are regions smaller than the current block; as well as Based on the affine vector, motion compensation prediction is performed on the sub-block, wherein... The first affine seed vector (nv0) and the second affine seed vector (nv1) of a merge candidate are derived based on the motion information of neighboring blocks adjacent to the current block. The adjacent blocks are included in coding tree units different from those of the current block, and the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate are derived based on the motion vectors of the lower left and lower right sub-blocks of the adjacent blocks. The lower left sub-block includes the lower left affine reference sample control point (x) located at the lower left corner of the adjacent block. n4 , y n4 The lower right sub-block and the lower right affine reference sample control point (x) located to the right of the lower right sample of the lower right sub-block. n5 , y n5 (Adjacent) 5. The method according to claim 4, wherein, The first and second affine seed vectors of a merge candidate are derived from values ​​obtained by applying a shift operation to the width between the lower left and lower right sub-blocks, wherein the shift operation shifts the width using a scaling factor.

6. The method according to claim 4, wherein, The width between the lower left sub-block and the lower right sub-block is defined as a variable, which is the width of the sub-seed vector set to a power of 2.

7. A video decoder, including an inter-frame prediction unit, the inter-frame prediction unit being configured to perform the following operations: Generate a list of merge candidates for the current block; Specify one merge candidate from the plurality of merge candidates included in the merge candidate list; Based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate, derive the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block. Using the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block, derive the affine vectors of the sub-blocks in the current block, wherein the sub-blocks are regions smaller than the current block; as well as Based on the affine vector, motion compensation prediction is performed on the sub-block, wherein... The first affine seed vector (nv0) and the second affine seed vector (nv1) of a merge candidate are derived based on the motion information of neighboring blocks adjacent to the current block. The adjacent blocks are included in coding tree units different from those of the current block, and the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate are derived based on the motion vectors of the lower left and lower right sub-blocks of the adjacent blocks. The lower left sub-block includes the lower left affine reference sample control point (x) located at the lower left corner of the adjacent block. n4 , y n4 The lower right sub-block and the lower right affine reference sample control point (x) located to the right of the lower right sample of the lower right sub-block. n5 , y n5 (Adjacent) 8. The video decoder according to claim 7, wherein, The first and second affine seed vectors of a merge candidate are derived from values ​​obtained by applying a shift operation to the width between the lower left and lower right sub-blocks, wherein the shift operation shifts the width using a scaling factor.

9. The video decoder according to claim 7, wherein, The width between the lower left sub-block and the lower right sub-block is defined as a variable, which is the width of the sub-seed vector set to a power of 2.

10. A video encoder including an inter-frame prediction unit, the inter-frame prediction unit being configured to perform the following operations: Generate a list of merge candidates for the current block; Specify one merge candidate from the plurality of merge candidates included in the merge candidate list; Based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate, derive the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block. Using the first affine seed vector (sv0) and the second affine seed vector (sv1) of the current block, derive the affine vectors of the sub-blocks in the current block, wherein the sub-blocks are regions smaller than the current block; as well as Based on the affine vector, motion compensation prediction is performed on the sub-block, wherein... The first affine seed vector (nv0) and the second affine seed vector (nv1) of a merge candidate are derived based on the motion information of neighboring blocks adjacent to the current block. The adjacent blocks are included in coding tree units different from those of the current block, and the first affine seed vector (nv0) and the second affine seed vector (nv1) of the merge candidate are derived based on the motion vectors of the lower left and lower right sub-blocks of the adjacent blocks. The lower left sub-block includes the lower left affine reference sample control point (x) located at the lower left corner of the adjacent block. n4 , y n4 The lower right sub-block and the lower right affine reference sample control point (x) located to the right of the lower right sample of the lower right sub-block. n5 , y n5 (Adjacent) 11. The video encoder according to claim 10, wherein, The first and second affine seed vectors of a merge candidate are derived from values ​​obtained by applying a shift operation to the width between the lower left and lower right sub-blocks, wherein the shift operation shifts the width using a scaling factor.

12. The video encoder according to claim 10, wherein, The width between the lower left sub-block and the lower right sub-block is defined as a variable, which is the width of the sub-seed vector set to a power of 2.

13. A bit stream transmission method, characterized in that, Perform the encoding method according to any one of claims 4-6 to generate a bit stream and transmit the bit stream.

Images