Video signal encoding / decoding method and equipment therefor

Abstract

This invention provides a video encoding / decoding method and equipment for performing interpretation using an affine model on video signals. [Solution] The inter prediction method includes the steps of: generating a list of merge candidates for the current block; specifying one of several merge candidates included in the merge candidate list; deriving the first and second affine seed vectors for the current block based on the first and second affine seed vectors of the specified merge candidate; deriving the affine vectors for the subblocks within the current block using the first and second affine seed vectors for the current block; and predicting motion compensation for the subblocks based on the affine vectors.

Description

[Technical Field]

[0001] The present invention relates to a video signal encoding / decoding method and equipment therefor. [Background technology]

[0002] As display panels continue to grow larger, higher-resolution video services are becoming increasingly necessary. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, research into improving video compression ratios is being actively pursued. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013. Its compression performance is approximately twice that of H.264 / AVC. However, with the rapid growth of high-definition video services, the performance limitations of HEVC are gradually becoming apparent. [Overview of the Initiative] [Problems that the invention aims to solve]

[0003] The object of the present invention is to provide an interpretation method that uses an affine model when encoding / decoding video signals, and equipment used in the interpretation method.

[0004] The object of the present invention is to provide a method for deriving an affine seed vector using the translational motion vector of a subblock when encoding / decoding a video signal, and an apparatus for performing the said method.

[0005] Another object of the present invention is to provide a method for deriving an affine seed vector by converting the distance between adjacent blocks and the current block into a power series of 2 when encoding / decoding a video signal, and an apparatus for performing the said method.

[0006] The technical problems that this invention aims to solve are not limited to those mentioned above, and a person with ordinary skill in the art to which this invention belongs will clearly understand other technical problems not mentioned in the following description. [Means for solving the problem]

[0007] The video signal decoding / encoding method according to the present invention includes the steps of: generating a merge candidate list for the current block; specifying one of a plurality of merge candidates included in the merge candidate list; deriving the first affine seed vector and the second affine seed vector of the current block based on the first affine seed vector and the second affine seed vector of the specified merge candidate; deriving the affine vector of a subblock within the current block using the first affine seed vector and the second affine seed vector of the current block; and performing motion compensation prediction for the subblock based on the affine vector. In this case, the subblock is a region smaller in size than the current block. The first affine seed vector and the second affine seed vector of the merge candidate can be derived based on motion information of an adjacent block adjacent to the current block.

[0008] In the video signal decoding / encoding method according to the present invention, if the adjacent block is included in a different encoding tree unit than the encoding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on the motion vectors of the lower left subblock and the lower right subblock of the adjacent block.

[0009] In the video signal decoding / encoding method according to the present invention, the lower left subblock may include a lower left reference sample located in the lower left corner of the adjacent block, and the lower right subblock may include a lower right reference sample located in the lower right corner of the adjacent block.

[0010] In the video signal decoding / encoding method according to the present invention, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on a value obtained by performing a shift operation on the difference value of the motion vector between the lower left subblock and the lower right subblock using a scale factor, and the scale factor can be derived based on a value obtained by adding the horizontal distance and offset between the lower left reference sample and the lower right reference sample.

[0011] In the video signal decoding / encoding method according to the present invention, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on a value obtained by performing a shift operation on the difference value of the motion vector between the lower left subblock and the lower right subblock using a scale factor, and the scale factor can be derived based on the distance between the adjacent sample adjacent to the right of the lower right reference sample and the lower left reference sample.

[0012] In the video signal decoding / encoding method according to the present invention, the merge candidate list includes a first merge candidate and a second merge candidate, wherein the first merge candidate is derived based on an upper adjacent block determined to be the first available block among the upper adjacent blocks located above the current block, and the second merge candidate is derived based on a left adjacent block determined to be the first available block among the left adjacent blocks located to the left of the current block.

[0013] In the video signal decoding / encoding method according to the present invention, if the adjacent block is included in an encoding tree unit that is the same as the encoding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on the first affine seed vector and the second affine seed vector of the adjacent block.

[0014] The above-mentioned brief summary of the features of the present invention is merely an illustrative embodiment of the invention as described in the detailed description below, and does not limit the scope of the invention. [Effects of the Invention]

[0015] According to the present invention, the interpretation method using an affine model has the effect of improving prediction efficiency.

[0016] According to the present invention, deriving the affine seed vector using the translational motion vector of the subblock has the effect of improving coding efficiency.

[0017] According to the present invention, the coding efficiency is improved by deriving an affine seed vector by converting the distance between adjacent blocks and the current block into a power series of 2.

[0018] The effects that can be obtained with the present invention are not limited to those described above, and a person with ordinary skill in the art to which the present invention belongs will be able to clearly understand other effects not mentioned below from the following description. [Brief explanation of the drawing]

[0019] [Figure 1] This is a block diagram showing a video encoder according to an embodiment of the present invention. [Figure 2] This is a block diagram showing a video decoder according to an embodiment of the present invention. [Figure 3] This figure shows a basic coding tree unit according to an embodiment of the present invention. [Figure 4] This diagram shows multiple types of coding block partitioning. [Figure 5] This figure shows an example of a coding tree unit partition. [Figure 6] The flowchart shows an inter prediction method according to an embodiment of the present invention. [Figure 7] This figure shows the nonlinear motion of the object. [Figure 8] This flowchart shows an inter-prediction method based on affine motion according to an embodiment of the present invention. [Figure 9] This figure shows examples of affine seed vectors for each affine motion model. [Figure 10] This figure shows an example of an affine vector for a subblock in a 4-parameter motion model. [Figure 11] This figure shows adjacent blocks that can be used to derive merge candidates. [Figure 12] This diagram shows how the affine seed vector of the current block is derived based on the affine seed vectors of the affine adjacent blocks. [Figure 13] This figure shows an example where the motion vector of a subblock is used as the affine seed vector of an affine adjacent block. [Figure 14] This diagram shows the location of the reference sample. [Figure 15] This diagram shows the location of the reference sample. [Figure 16] This diagram shows the location of the reference sample. [Figure 17]This figure shows an example of applying the modified affine merge vector derivation method. [Figure 18] This figure shows an example of deriving the affine seed vector of the current block based on non-affine adjacent blocks. [Figure 19] This diagram shows an example of replacing an adjacent block with another adjacent block. [Figure 20] This is a flowchart showing a motion compensation prediction method using multiple merge candidates. [Figure 21] This is a flowchart showing a motion compensation prediction method using multiple merge candidates. [Figure 22] This flowchart shows an intra-prediction method according to an embodiment of the present invention. [Figure 23] This figure shows the reference samples included in each reference sample line. [Figure 24] This figure shows the intra-prediction mode. [Figure 25] This figure shows an example of a one-dimensional array in which reference samples are arranged in a single row. [Figure 26] This figure shows an example of a one-dimensional array in which reference samples are arranged in a single row. [Figure 27] This figure shows the angle formed between the angle intra-prediction mode and a line parallel to the x-axis. [Figure 28] This figure shows an example of obtaining a prediction sample when the block is currently not square. [Figure 29] This figure shows the wide-angle intra-predictive mode. [Figure 30] This is a flowchart showing the process for determining block strength. [Figure 31] This is a diagram showing predefined filter candidates. [Figure 32] This figure shows an example of dividing an image into multiple tiles. [Figure 33] This figure shows an image segmentation mode based on flexible tiling technology. [Figure 34] This figure shows an example of assigning a tile ID to each coding tree unit. [Figure 35] This diagram illustrates an example of selectively deciding whether or not to apply a loop filter to each tile. [Modes for carrying out the invention]

[0020] The embodiments of the present invention will be described in detail below with reference to the drawings.

[0021] Video encoding and decoding are performed in blocks. For example, encoding / decoding operations such as transformation, quantization, prediction, loop filtering, or reconstruction can be performed on an encoding block, a transform block, or a prediction block.

[0022] Hereafter, the block to be encoded / decoded will be referred to as the "current block." For example, according to the current encoding / decoding process step, the current block can represent an encoded block, a transformed block, or a predicted block.

[0023] In this specification, the term "unit" may be understood to refer to a basic unit for performing a particular encoding / decoding process, and "block" may be understood to refer to a sample array of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described later, encoding blocks and encoding units may be understood to have the same meaning.

[0024] Figure 1 is a block diagram showing a video encoder according to an embodiment of the present invention.

[0025] Referring to Figure 1, the video encoding device 100 may include an image splitting unit 110, prediction units 120, 125, a conversion unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse conversion unit 145, a filter unit 150, and a memory 155.

[0026] Each component shown in Figure 1 is shown individually and represents a distinct characteristic function in a video encoding device, but does not indicate that each component consists of separate hardware or a single software assembly. In other words, for ease of explanation, each component is shown as a representative component and may include at least two components combined into one, or one component may be divided into multiple components to perform its function. Without departing from the essence of the present invention, embodiments in which such components are combined and embodiments in which components are separated are also within the scope of the present invention.

[0027] Furthermore, some components are not essential for performing the essential functions of the present invention, but are merely optional components for improving performance. The present invention may be implemented by including only the components necessary for realizing the essence of the present invention (excluding components for improving performance), and a structure including only the necessary components (excluding components for improving performance) also falls within the scope of the rights of the present invention.

[0028] The image splitting unit 110 can split an input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image splitting unit 110 splits a single image into a combination of multiple coding units, prediction units, and transform units. Based on a predetermined criterion (e.g., a cost function), a combination of coding units, prediction units, and transform units can be selected to encode the image.

[0029] For example, a single image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used, with one video or the largest coding unit as the root, and the coding unit being divided into other coding units. The coding unit may have the same number of child nodes as the number of coding units it has been divided into. Coding units that are not divided due to certain restrictions become leaf nodes. In other words, assuming that one coding unit can only achieve square partitioning, one coding unit can be divided into up to four other coding units.

[0030] In the embodiments of the present invention described below, the encoding unit may refer to a unit that performs encoding, or it may refer to a unit that performs decoding.

[0031] A prediction unit in one coding unit can be divided into at least one square or rectangle of the same size, and a prediction unit in one coding unit can also be divided into a shape and / or size different from another prediction unit.

[0032] If the prediction unit that performs intraprediction based on the coding unit is not the minimum coding unit, it is not necessary to divide it into multiple prediction units N×N, and intraprediction can be performed.

[0033] The prediction units 120 and 125 may include an inter-prediction unit 120 that performs inter-prediction and an intra-prediction unit 125 that performs intra-prediction. The prediction unit can decide whether to use inter-prediction or intra-prediction, and can also determine specific information (e.g., intra-prediction mode, motion vector, reference image, etc.) based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction unit can determine the prediction method and prediction mode, and the conversion unit can perform the prediction. The residual value (residual block) between the generated prediction block and the original block can be input to the conversion unit 130. Prediction mode information, motion vector information, etc. for prediction can be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When using a specific encoding mode, the original block can be directly encoded and transmitted to the decoder without generating prediction blocks by the prediction units 120 and 125.

[0034] The interpretation unit 120 can predict prediction units based on information from at least one image in the image immediately preceding or following the current image. In some cases, it can also predict prediction units based on information from a specific encoded region in the current image. The interpretation unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0035] The reference image interpolation unit receives reference image information from memory 155 and can generate pixel information for integer pixels or fractional pixels from the reference image. For luminance pixels, an 8-tap interpolation filter based on a DCT with different filter coefficients can be used to generate pixel information for fractional pixels in units of 1 / 4 pixels. For chromaticity signals, a 4-tap interpolation filter based on a DCT with different filter coefficients can be used to generate pixel information for fractional pixels in units of 1 / 8 pixels.

[0036] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Multiple methods can be used to calculate the motion vector, including the Full search-based Block Matching Algorithm (FBMA), the Three Step Search (TSS), and the New Three-Step Search Algorithm (NTS). Depending on the interpolated pixels, the motion vector may have motion vector values ​​in units of 1 / 2 pixels or 1 / 4 pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Multiple motion prediction methods can be used, including the Skip method, Merge method, Advanced Motion Vector Prediction (AMVP), and Intra Block Copy method.

[0037] The intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block (the reference pixel information is pixel information in the current image). If the adjacent block of the current prediction unit is a block for which inter-prediction has been performed, and the reference pixel is a pixel for which inter-prediction has been performed, the reference pixel included in the block for which inter-prediction has been performed can be used as the reference pixel information for the surrounding blocks for which intra-prediction has been performed. In other words, if a reference pixel is unavailable, at least one of the available reference pixels can be used instead of the unavailable reference pixel information.

[0038] In intra-prediction, the prediction mode may include an angle prediction mode that uses reference pixel information based on the prediction direction and a non-angle mode that does not use direction information when performing prediction. The mode for predicting luminance information may be different from the mode for predicting chromaticity information. To predict chromaticity information, intra-prediction mode information for predicting luminance information or predicted luminance signal information can be used.

[0039] When performing intraprediction, if the size of the prediction unit is the same as the size of the transformation unit, intraprediction can be performed on the prediction unit based on the pixels located to the left, the pixels located to the upper left, and the pixels located above the prediction unit. However, when performing intraprediction, if the size of the prediction unit is different from the size of the transformation unit, intraprediction can be performed based on the reference pixels of the transformation unit. Furthermore, intraprediction using N×N partitioning can be applied only to the minimum coding unit.

[0040] After applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel based on the prediction mode, a prediction block can be generated using an intra-prediction method. The type of Adaptive Intra Smoothing filter applied to the reference pixel may vary. To perform the intra-prediction method, the intra-prediction mode of the current prediction unit can be predicted based on the intra-prediction modes of prediction units located around the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted from surrounding prediction units, if the intra-prediction mode of the current prediction unit is the same as that of the surrounding prediction units, information indicating that the prediction mode of the current prediction unit is the same as that of the surrounding prediction units can be transmitted using predetermined flag information. If the prediction mode of the current prediction unit is different from that of the surrounding prediction units, the prediction mode information of the current block can be encoded by performing entropy coding.

[0041] Furthermore, residual blocks containing residual information can be generated. The residual information is the difference between the prediction unit that performs the prediction based on the prediction units generated by the prediction units 120 and 125 and the original block of the prediction unit. The generated residual blocks can be input to the conversion unit 130.

[0042] The transformation unit 130 can transform the residual block using a transformation method such as discrete cosine transform (DCT), discrete sine transform (DST), or transformation skip. The residual block includes residual information between the original block and the prediction units generated by the prediction units 120 and 125. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra-prediction mode information of the prediction unit used to generate the residual block.

[0043] The quantization unit 135 can quantize the values ​​converted to the frequency domain by the conversion unit 130. The quantization coefficients may vary depending on the importance of the blocks or images. The values ​​calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0044] The rearrangement unit 160 can perform a rearrangement of coefficient values ​​for the quantized residual values.

[0045] The rearrangement unit 160 can convert two-dimensional block shape coefficients into one-dimensional vector form using a coefficient scanning method. For example, the rearrangement unit 160 can scan DC coefficients or coefficients in the high-frequency region using a zig-zag scan method and convert them into one-dimensional vector form. Depending on the size of the conversion unit and the intra-prediction mode, vertical scanning, which scans two-dimensional block shape coefficients along the column direction, and horizontal scanning, which scans two-dimensional block shape coefficients along the row direction, can also be used instead of zigzag scanning. In other words, it is possible to decide whether to use zigzag scanning, vertical scanning, or horizontal scanning based on the size of the conversion unit and the intra-prediction mode.

[0046] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. For example, entropy coding can use multiple coding methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0047] The entropy coding unit 165 can encode multiple pieces of information from the rearrangement unit 160 and the prediction units 120 and 125, including residual coefficient information and block type information of the coding units, prediction mode information, divided unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0048] The entropy coding unit 165 can perform entropy coding on the coefficient values ​​of the coding units input from the rearrangement unit 160.

[0049] The inverse quantization unit 140 and the inverse transformation unit 145 perform inverse quantization on multiple values ​​quantized by the quantization unit 135, and perform inverse transformation on the values ​​transformed by the transformation unit 130. By merging the residual values ​​generated by the inverse quantization unit 140 and the inverse transformation unit 145 with the prediction units predicted by the motion prediction unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125, a reconstructed block can be generated.

[0050] The filter unit 150 may include at least one of the following: a deblocking filter, an offset correction unit, or an adaptive loop filter (ALF).

[0051] A deblocking filter can remove block distortion generated in the reconstructed image due to the boundaries between blocks. To determine whether to perform deblocking, it is possible to determine whether to apply a deblocking filter to the current block based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied based on the required deblocking filtering strength. Furthermore, when performing vertical or horizontal filtering during the process of using a deblocking filter, horizontal and vertical filtering can be performed synchronously.

[0052] The offset correction unit can correct the offset between the deblocking image and the original image on a pixel-by-pixel basis. Offset correction can be performed on a specified image using the following method: After dividing the pixels contained in the image into a predetermined number of regions, the regions requiring offset correction are determined, and offset correction is applied to the corresponding regions, or offset correction is applied considering the edge information of each pixel.

[0053] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered reconstructed image and the original image. After dividing the pixels in the image into predetermined groups, a single filter can be determined for each corresponding group, and filtering can be performed differentially for each group. Information regarding whether or not to apply adaptive loop filtering can be transmitted to each coding unit (CU) via a luminance signal. The shape and filter coefficients of the applied adaptive loop filter may differ for each block. In addition, the same type (constant type) of ALF can be applied regardless of the characteristics of the block to which it is applied.

[0054] The memory 155 can store the reconstructed blocks or blocks calculated by the filter unit 150, and when performing interpretation, it can provide the stored reconstructed blocks or images to the prediction units 120 and 125.

[0055] Figure 2 is a block diagram showing a video decoder according to an embodiment of the present invention.

[0056] Referring to Figure 2, the video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0057] When a video bitstream is input from a video encoder, the input bitstream can be decoded in the reverse step of the video encoder's process.

[0058] The entropy decoding unit 210 can perform entropy decoding in steps that are the reverse of the steps performed by the entropy coding unit of the video encoder. For example, several methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied to correspond to the methods performed by the video encoder.

[0059] The entropy decoding unit 210 can decode information related to intra-prediction and inter-prediction performed by the encoder.

[0060] The rearrangement unit 215 can perform rearrangement by rearranging the bitstream that has been entropically decoded by the entropy decoding unit 210 in the encoding unit. Rearrangement can be performed by reconstructing multiple coefficients represented in one-dimensional vector form into two-dimensional block-shaped coefficients. The rearrangement unit 215 can perform rearrangement in the following manner: It receives information related to the coefficient scan performed by the encoding unit and performs a reverse scan according to the scan order performed by the corresponding encoding unit.

[0061] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged blocks.

[0062] With respect to the quantization results performed by the video encoder, the inverse transformer 225 can perform inverse transforms on the DCT, DST, and KLT transforms performed by the transformer. That is, it performs inverse DCT, inverse DST, and inverse KLT. The inverse transform may also be performed by a transmission unit determined by the video encoder. In the inverse transformer 225 of the video decoder, a transform method (e.g., DCT, DST, KLT) can be selectively performed based on multiple pieces of information such as the prediction method, the size of the current block, and the prediction direction.

[0063] The prediction units 230 and 235 can generate prediction blocks based on information related to the generation of prediction blocks provided by the entropy decoding unit 210 and previously decoded blocks or image information provided by the memory 245.

[0064] As described above, when performing intraprediction using the same method as in a video encoder, if the size of the prediction unit is the same as the size of the conversion unit, intraprediction can be performed on the prediction unit based on the pixels located to the left, the pixels located to the upper left, and the pixels located above the prediction unit. When performing intraprediction, if the size of the prediction unit is different from the size of the conversion unit, intraprediction can be performed based on the reference pixels of the conversion unit. Furthermore, intraprediction using N×N partitioning can be applied only to the minimum coding unit.

[0065] The prediction units 230 and 235 may include a prediction unit determination unit, an inter-prediction unit, and an intra-prediction unit. The prediction unit determination unit receives multiple pieces of information from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-prediction method, and motion prediction-related information of the inter-prediction method, classifies the prediction units based on the currently encoded unit, and determines whether the prediction unit is performing inter-prediction or intra-prediction. The inter-prediction unit 230 can perform inter-prediction on the current prediction unit using the information provided by the video encoder that is necessary for the current prediction unit to perform inter-prediction, based on information contained in at least one image that is either the image immediately preceding or following the current image to which the current prediction unit belongs. Alternatively, it can perform inter-prediction based on information from a reconstructed portion of the current image to which the current prediction unit belongs.

[0066] To perform interpretation, based on the encoded unit, it is possible to determine whether the motion prediction method of the prediction unit contained within the corresponding encoded unit is Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), or Intrablock Duplication Mode.

[0067] The intra-prediction unit 235 can generate prediction blocks based on pixel information in the current image. If the prediction unit is the same prediction unit that performed intra-prediction, it can perform intra-prediction based on the intra-prediction mode information of the prediction unit provided by the video encoder. The intra-prediction unit 235 may also include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra-smoothing filter is the part that performs filtering on the reference pixels of the current block, and can determine whether to apply the filter based on the prediction mode of the current prediction unit. Adaptive intra-smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit and the adaptive intra-smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode that does not perform adaptive intra-smoothing filtering, the adaptive intra-smoothing filter does not need to be applied.

[0068] If the prediction mode of the prediction unit is a prediction unit that performs intra-prediction based on pixel values ​​to be interpolated for reference pixels, the reference pixel interpolation unit can generate reference pixels in units of integer or decimal values ​​by interpolating for the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks in a way that does not interpolate for reference pixels, interpolation for reference pixels is not required. If the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks by filtering.

[0069] The reconstructed block or image can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0070] The video decoder can receive information from the video encoder regarding whether to apply a deblocking filter to the corresponding block or image, and if so, whether to use a strong or weak filter. The video decoder's deblocking filter can receive the information provided by the video encoder regarding the deblocking filter, and the video decoder can perform deblocking filtering on the corresponding block.

[0071] The offset correction unit can perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and the offset information.

[0072] Based on information provided by the encoder regarding whether or not to apply ALF, ALF coefficient information, etc., ALF can be applied to the encoding unit. Such ALF information may be provided by being included in a specific set of parameters.

[0073] Memory 245 stores the reconstructed image or block, making the image or block available as a reference image or reference block, and can also provide the reconstructed image to the output unit.

[0074] Figure 3 shows a basic coding tree unit according to an embodiment of the present invention.

[0075] The largest coding block can be defined as a coding tree block. A single image may be divided into multiple coding tree units (CTUs). A coding tree unit is the largest coding unit and may be called a Largest Coding Unit (LCU). Figure 3 shows an example of dividing a single image into multiple coding tree units.

[0076] The size of the coded tree unit may be defined at the image level or the sequence level. Therefore, information indicating the size of the coded tree unit can be transmitted using a signal based on an image parameter set or a sequence parameter set.

[0077] For example, the size of the encoding tree unit for the entire sequence of images can be set to 128×128. Alternatively, the size of the encoding tree unit can be determined to be either 128×128 or 256×256 at the image level. For example, the size of the encoding tree unit for the first image can be set to 128×128, and the size of the encoding tree unit for the second image can be set to 256×256.

[0078] Encoded blocks can be generated by dividing the encoding tree units. Encoded blocks represent basic units for encoding / decoding processing. For example, prediction or transformation can be performed depending on different encoding blocks, or a predictive encoding mode can be determined depending on different encoding blocks. Here, the predictive encoding mode represents a method for generating a predicted image. For example, the predictive encoding mode may include intra-prediction, inter-prediction, current picture referencing (CPR), intra-block copy (IBC), or combined prediction. For each encoding block, a predictive block related to the encoding block can be generated using at least one predictive encoding mode from intra-prediction, inter-prediction, current picture referencing, or combined prediction.

[0079] Information representing the predictive coding mode of the current block can be transmitted via a bitstream signal. For example, this information may be a 1-bit flag indicating whether the predictive coding mode is intra-mode or inter-mode. Only when it is determined that the predictive coding mode of the current block is inter-mode can current image referencing or combined prediction be used.

[0080] The current image reference is used to obtain the predicted block of the current block from the encoded / decoded region of the current image, using the current image as the reference image. Here, the current image means the image containing the current block. Information indicating whether to apply the current image reference to the current block can be transmitted via a bitstream signal. For example, the information may be a 1-bit flag. If the flag is true, the predicted encoding mode of the current block can be determined as current image reference. If the flag is false, the prediction mode of the current block can be determined as inter-prediction.

[0081] Alternatively, the predictive coding mode for the current block can be determined based on a reference image index. For example, if the reference image index points to the current image, the predictive coding mode for the current block can be determined as current image reference. If the reference image index points to an image other than the current image, the predictive coding mode for the current block can be determined as inter-prediction. In other words, current image reference is a prediction method that uses information from the coded / decoded region in the current image, while inter-prediction is a prediction method that uses information from other coded / decoded images.

[0082] Combinatorial prediction is an encoding mode formed by combining two or more of the following: intra-prediction, inter-prediction, and current image reference. For example, when applying combinational prediction, a first prediction block can be generated based on one of the intra-prediction, inter-prediction, or current image reference, and a second prediction block can be generated based on another. When generating the first and second prediction blocks, a final prediction block can be generated by averaging and weighting the first and second prediction blocks. Information indicating whether or not to apply combinational prediction can be transmitted via a bitstream in a signal. This information may be a 1-bit flag.

[0083] Figure 4 shows multiple partitioning types for encoded blocks.

[0084] Based on quadtree, binary, or ternary tree partitioning, an encoded block can be divided into multiple encoded blocks. Furthermore, based on quadtree, binary, or ternary tree partitioning, the divided encoded block can be further divided into multiple encoded blocks.

[0085] Quadrarrow partitioning is a partitioning technique that divides the current block into four blocks. As a result of quadrarrow partitioning, the current block can be divided into four square partitions (see "SPLIT_QT" in Figure 4(a)).

[0086] Binary tree partitioning is a partitioning technique that divides the current block into two blocks. The process of dividing the current block into two blocks along the vertical direction (i.e., using vertical lines that cross the current block) is called vertical binary tree partitioning, and the process of dividing the current block into two blocks along the horizontal direction (i.e., using horizontal lines that cross the current block) can be called horizontal binary tree partitioning. As a result of binary tree partitioning, the current block can be divided into two non-square partitions. In Figure 4(b), "SPLIT_BT_VER" represents the result of vertical binary tree partitioning, and in Figure 4(c), "SPLIT_BT_HOR" represents the result of horizontal binary tree partitioning.

[0087] Ternary tree partitioning is a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines that cross the current block) is called vertical ternary tree partitioning, and the process of dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines that cross the current block) can be called horizontal ternary tree partitioning. As a result of ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located in the center of the current block may be twice the width / height of the other partitions. In Figure 4(d), "SPLIT_TT_VER" represents the result of vertical ternary tree partitioning, and in Figure 4(e), "SPLIT_TT_HOR" represents the result of horizontal ternary tree partitioning.

[0088] The number of divisions in an encoded tree unit can be defined as the partitioning depth. The maximum partitioning depth of an encoded tree unit can be determined at the sequence or image level. Therefore, the maximum partitioning depth of an encoded tree unit may vary depending on the sequence or image.

[0089] Alternatively, the maximum partitioning depth can be determined independently for each of the multiple partitioning techniques. For example, the maximum partitioning depth that allows quadtree partitioning may be different from the maximum partitioning depth that allows binary tree partitioning and / or ternary tree partitioning.

[0090] The encoder can transmit information via the bitstream in a signal that represents at least one of the division type or division depth of the current block. The decoder can determine the division type and division depth of the coded tree unit based on the information analyzed from the bitstream.

[0091] Figure 5 shows an example of a coding tree unit partitioning.

[0092] The process of dividing an encoded block using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning can be called multi-tree partitioning.

[0093] The coded blocks generated by applying multi-tree partitioning to a coded block can be called downstream coded blocks. If the partitioning depth of a coded block is k, the partitioning depth of multiple downstream coded blocks is k+1.

[0094] On the other hand, for a coding block with a division depth of k+1, a coding block with a division depth of k can be called an upstream coding block.

[0095] The partition type of the currently encoded block can be determined based on at least one of the partition types of the upstream encoded block or the partition types of the adjacent encoded blocks. Here, the adjacent encoded block is adjacent to the currently encoded block and may include at least one of the above adjacent block, left adjacent block, or adjacent block adjacent to the upper left corner of the currently encoded block. Here, the partition type may include at least one of whether to perform a quadtree partition, whether to perform a binary tree partition, the direction of the binary tree partition, whether to perform a ternary tree partition, or the direction of the ternary tree partition.

[0096] To determine the type of partitioning of an encoded block, information indicating whether the encoded block has been partitioned can be transmitted via a bitstream signal. This information is a 1-bit flag "split_cu_flag", and if the flag is true, it indicates that the encoded block is partitioned using a multi-tree partitioning technique.

[0097] If "split_cu_flag" is true, a signal can be transmitted via the bitstream indicating whether the encoded block has been quadtree-split. This information is a 1-bit flag "split_qt_flag", and if this flag is true, the encoded block may be split into four blocks.

[0098] For example, in the example shown in Figure 5, it is illustrated that when the coding tree unit is quadrulated, four coding blocks with a division depth of 1 are generated. Furthermore, the quadrulation is then applied again to the first and fourth coding blocks among the four coding blocks generated as a result of the quadrulation. Ultimately, four coding blocks with a division depth of 2 can be generated.

[0099] Furthermore, by applying quadtree partitioning again to an encoded block with a partitioning depth of 2, it is possible to generate an encoded block with a partitioning depth of 3.

[0100] If a quadtree split is not applied to a coded block, it is possible to determine whether to perform a binary tree split or a ternary tree split on the coded block by considering at least one of the following: the size of the coded block, whether the coded block is located at the boundary of the image, the maximum splitting depth, or the splitting type of adjacent blocks. If it is determined that a binary tree split or a ternary tree split should be performed on the coded block, information indicating the splitting direction can be transmitted via a bitstream as a signal. This information may be a 1-bit flag "mtt_split_cu_vertical_flag". Based on this flag, it is possible to determine whether the splitting direction is vertical or horizontal. Furthermore, it is possible to transmit information via a bitstream as a signal indicating whether to apply a binary tree split or a ternary tree split to the coded block. This information may be a 1-bit flag "mtt_split_cu_binary_flag". Based on this flag, it is possible to determine whether to apply a binary tree split or a ternary tree split to the coded block.

[0101] For example, in the example shown in Figure 5, a vertical binary tree partition is applied to an encoded block with a partitioning depth of 1. A vertical ternary tree partition is applied to the left encoded block of the encoded block generated as a result of the partitioning, and a vertical binary tree partition is applied to the right encoded block.

[0102] Interpretation is a predictive coding mode that predicts the current block using information from the previous image. For example, a block in the previous image at the same position as the current block (hereinafter referred to as a collocated block) can be used as the predicted block for the current block. Hereafter, a predicted block generated based on a block at the same position as the current block can be called a collocated prediction block.

[0103] On the other hand, if an object present in the previous image moves to a different position in the current image, the current block can be effectively predicted due to symmetrical movement. For example, by comparing the previous image with the current image, the direction and size of the object's movement can be determined, and a predicted block (or predicted image) of the current block can be generated considering the object's movement information. Hereafter, a predicted block generated using motion information can be called a motion prediction block.

[0104] A residual block can be generated by subtracting a predicted block from the current block. In this case, if the target motion exists, using a collated prediction block instead of a motion prediction block can reduce the energy of the residual block and improve the compression performance of high-residual blocks.

[0105] As mentioned above, the process of generating prediction blocks using motion information can be called motion-compensated prediction. In most interpretations, prediction blocks can be generated based on motion-compensated prediction.

[0106] Motion information may include at least one of the following: motion vector, reference image index, prediction direction, or bidirectional weighted index. The motion vector represents the direction and size of movement of the object. The reference image index specifies the reference image of the current block among multiple reference images included in the reference image list. The prediction direction refers to one of the following: unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). Based on the prediction direction of the current block, at least one of the motion information in the L0 direction or the motion information in the L1 direction can be used. The bidirectional weighted index specifies the weights applied to the L0 prediction block and the weights applied to the L1 prediction block.

[0107] Figure 6 is a flowchart showing an inter-prediction method according to an embodiment of the present invention.

[0108] Referring to Figure 6, the inter-prediction method includes the steps of determining the inter-prediction mode of the current block (S601), obtaining motion information of the current block based on the determined inter-prediction mode (S602), and performing motion compensation prediction for the current block based on the obtained motion information (S603).

[0109] Here, the interpretation mode represents multiple techniques for determining the motion information of the current block, and may include an interpretation mode using translation motion information and an interpretation mode using affine motion information. For example, the interpretation mode using translation motion information may include a merge mode and an advanced motion vector prediction mode. The interpretation mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. According to the interpretation mode, the motion information of the current block can be determined based on information analyzed from adjacent blocks or bitstreams adjacent to the current block.

[0110] The following explains in detail the inter-prediction method using affine motion information.

[0111] Figure 7 shows the nonlinear motion of the object.

[0112] Symmetrical motion in video may be nonlinear. For example, as shown in Figure 7, symmetrical nonlinear motion such as camera zoom-in, zoom-out, rotation, or affine transformation can occur. When nonlinear motion occurs in an object, the translational motion vector cannot effectively represent the object's motion. Therefore, encoding efficiency can be improved by using affine motion instead of translational motion in parts where nonlinear motion occurs in the object.

[0113] Figure 8 is a flowchart showing an inter-prediction method based on affine motion according to an embodiment of the present invention.

[0114] Based on the information analyzed from the bitstream, it is possible to determine whether to apply affine motion-based interpretation techniques to the current block. Specifically, it is possible to determine whether to apply affine motion-based interpretation techniques to the current block based on at least one of the following flags: a flag indicating whether to apply affine merge mode to the current block, or a flag indicating whether to apply affine motion vector prediction mode to the current block.

[0115] When applying interpretation techniques based on affine motion to a current block, the affine motion model of the current block can be determined (S801). The affine motion model may be determined to be at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model represents affine motion with six parameters, and the 4-parameter affine motion model represents affine motion with four parameters.

[0116] Equation 1 represents the case where affine motion is expressed using six parameters. Affine motion represents translational motion within a predetermined region determined by the affine seed vector.

number

[0117] While representing affine motion with six parameters allows for the representation of complex motion, it increases the number of bits required for encoding each parameter, thus reducing encoding efficiency. Therefore, affine motion can also be represented with four parameters. Equation 2 shows the case where affine motion is represented with four parameters.

number

[0118] Information for determining the affine motion model of a block can be encoded and transmitted as a signal via a bitstream. For example, the information may be a 1-bit flag "affine_type_flag". A value of 0 for the flag indicates that a 4-parameter affine motion model is applied. A value of 1 for the flag indicates that a 6-parameter affine motion model is applied. The flag can be encoded on a slice, tile, or block basis (e.g., encoded block or encoded tree unit). When the flag is transmitted as a signal at the slice level, the affine motion model determined at the slice level can be applied to all blocks belonging to that slice.

[0119] Alternatively, the affine motion model of the current block can be determined based on the affine interpretation mode of the current block. For example, when applying the affine merge mode, the affine motion model of the current block can be determined as a four-parameter motion model. On the other hand, when applying the affine motion vector prediction mode, the information for determining the affine motion model of the current block can be encoded and transmitted as a signal via a bitstream. For example, when applying the affine motion vector prediction mode to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0120] Next, the affine seed vector of the current block can be derived (S802). If a 4-parameter affine motion model is selected, motion vectors can be derived for the two control points of the current block. If a 6-parameter affine motion model is selected, motion vectors can be derived for the three control points of the current block. Motion vectors at control points can be called affine seed vectors. The control points may include at least one of the upper left corner, upper right corner, or lower left corner of the current block.

[0121] Figure 9 shows examples of affine seed vectors for each affine motion model.

[0122] In a four-parameter affine motion model, it is possible to derive affine seed vectors related to two of the following corners: the upper left corner, the upper right corner, or the lower left corner. For example, as shown in Figure 9(a), if a four-parameter affine motion model is selected, the affine vector can be derived using the affine seed vector sv0 related to the upper left corner of the current block (e.g., the upper left sample (x0, y0)) and the affine seed vector sv1 related to the upper right corner of the current block (e.g., the upper right sample (x1, y1)). Alternatively, the affine seed vector related to the lower left corner can be used instead of the affine seed vector related to the upper left corner. Or, the affine seed vector related to the lower left corner can be used instead of the affine seed vector related to the upper right corner.

[0123] In a 6-parameter affine motion model, affine seed vectors related to the upper left corner, upper right corner, and lower left corner can be derived. For example, as shown in the example in Figure 9(b), if a 6-parameter affine motion model is selected, affine vectors can be derived using the affine seed vector sv0 related to the upper left corner of the current block (e.g., upper left sample (x0,y0)), the affine seed vector sv1 related to the upper right corner of the current block (e.g., upper right sample (x1,y1)), and the affine seed vector sv2 related to the upper left corner of the current block (e.g., upper left sample (x2,y2)).

[0124] In the embodiments described later, in the 4-parameter affine motion model, the affine seed vectors of the upper-left control point and the upper-right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments using the first and second affine seed vectors described later, at least one of the first and second affine seed vectors can be replaced with the affine seed vector of the lower-left control point (third affine seed vector) or the affine seed vector of the lower-right control point (fourth affine seed vector).

[0125] In the 6-parameter affine motion model, the affine seed vectors for the upper-left, upper-right, and lower-left control points are referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector, respectively. In the embodiment using the first, second, and third affine seed vectors, described later, at least one of the first, second, and third affine seed vectors can be replaced with the affine seed vector for the lower-right control point (the fourth affine seed vector).

[0126] Using an affine seed vector, affine vectors can be derived for different subblocks (S803). Here, the affine vector represents the translational motion vector derived based on the affine seed vector. The affine vector of a subblock can be called the affine subblock motion vector or subblock motion vector.

[0127] Figure 10 shows an example of an affine vector for a subblock in a four-parameter motion model.

[0128] The affine vector of a subblock can be derived based on the position of the control points, the position of the subblock, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine subblock vector.

number

[0129] In Equation 3, (x,y) represents the position of the subblock. Here, the position of the subblock represents the position of the reference sample contained within the subblock. The reference sample may be a sample located in the upper left corner of the subblock, or it may be at least one sample located at the center in its x-axis or y-axis coordinates. (x0,y0) represents the position of the first control point, and (sv 0x sv 0y ) represents the first affine seed vector. Note that (x1, y1) represents the position of the second control point, and (sv 1x sv 1y ) represents the second affine seed vector.

[0130] If the first and second control points correspond to the top-left and top-right corners of the current block, respectively, then x1-x0 can be set to a value equal to the width of the current block.

[0131] Next, motion compensation prediction can be performed for each subblock using the affine vector of each subblock (S804). After performing motion compensation prediction, prediction blocks related to each subblock can be generated. The prediction block of a subblock can be set as the prediction block of the current block.

[0132] The affine seed vector of the current block can be derived based on the affine seed vectors of the adjacent blocks. If the interpretation mode of the current block is affine merge mode, the affine seed vector of the merge candidate included in the merge candidate list can be determined as the affine seed vector of the current block. Furthermore, if the interpretation mode of the current block is affine merge mode, the motion information of the current block, including at least one of the following: reference image index, specific direction prediction flag, or bidirectional weighting value, can be set to be the same as that of the merge candidate.

[0133] Merging candidates can be derived based on the adjacent blocks of the current block. The adjacent blocks may include at least one of the spatially adjacent blocks that are spatially adjacent to the current block, and the temporally adjacent blocks that are included in an image different from the current image.

[0134] Figure 11 shows adjacent blocks that can be used to derive merge candidates.

[0135] The adjacent blocks of the current block may include at least one of the following: (A) adjacent to the left of the current block, (B) adjacent to the information of the current block, (C) adjacent to the upper right corner of the current block, (D) adjacent to the lower left corner of the current block, or adjacent to the upper left corner of the current block. If the coordinates of the upper left sample of the current block are (x0, y0), then the left adjacent block A includes a sample at position (x0-1, y0+H-1), and the upper adjacent block B includes a sample at position (x0+W-1, y0-1), where W and H represent the width and height of the current block, respectively. The upper right adjacent block C includes a sample at position (x0+W, y0-1), and the lower left adjacent block D includes a sample at position (x0-1, y0+H). The upper left adjacent block E includes a sample at position (x0-1, y0-1).

[0136] In affine interpretation mode, when encoding adjacent blocks, the affine seed vector of the merge candidate can be derived based on the affine seed vector of the corresponding adjacent block. In the following section, adjacent blocks encoded in affine interpretation mode will be referred to as affine adjacent blocks.

[0137] Merging candidates for the current block can be generated by searching for adjacent blocks according to a predefined scan order. The scan order can be predefined in the encoder and decoder. For example, adjacent blocks can be searched in the order A, B, C, D, E. Merging candidates can then be sequentially derived from the searched affine adjacent blocks. Alternatively, the scan order can be adaptively determined based on at least one of the current block's size, shape, or affine motion model. In other words, blocks with at least one of different sizes, shapes, or affine motion models will have different scan orders.

[0138] Alternatively, the blocks located above the current block are searched sequentially, and a merge candidate is derived from the first affine adjacent block found. Similarly, the blocks located to the left of the current block are searched sequentially, and a merge candidate is derived from the first affine adjacent block found. Here, the multiple adjacent blocks located above the current block include at least one of adjacent block E, adjacent block B, or adjacent block C, and the multiple blocks located to the left of the current block may include at least one of block A or block D. In this case, adjacent block E may be classified as a block located to the left of the current block.

[0139] Although not shown in the diagram, merge candidates can be derived from the temporally adjacent blocks of the current block. Here, the temporally adjacent blocks may include blocks located at the same position as the current block in the collated image or blocks adjacent to it. Specifically, when encoding is performed on the temporally adjacent blocks of the current block in affine interpretation mode, merge candidates can be derived based on the affine seed vector of the temporal merge candidate.

[0140] A merge candidate list containing merge candidates can be generated, and one of the merge candidates in the merge candidate list can be determined as the affine seed vector for the current block. To achieve this, index information identifying any one of the multiple merge candidates can be encoded and transmitted via a bitstream.

[0141] As another example, multiple adjacent blocks can be explored depending on the scanning order, and the affine seed vector of the current block can be derived from the affine seed vector of the first discovered affine adjacent block.

[0142] As mentioned above, in affine merge mode, the affine seed vector of the current block can be derived using the affine seed vectors of adjacent blocks.

[0143] If the current block's interpretation mode is affine motion vector prediction mode, the affine seed vector of the motion vector prediction candidate included in the motion vector prediction candidate list can be determined as the affine seed vector prediction value for the current block. By adding the affine seed vector difference value to the affine seed vector prediction value, the affine seed vector for the current block can be derived.

[0144] Based on the adjacent blocks of the current block, affine seed vector prediction candidates can be derived. Specifically, according to a predetermined scanning order, multiple adjacent blocks located above the current block can be searched, and a first affine seed vector prediction candidate can be derived from the first affine adjacent block found. Furthermore, according to a predetermined scanning order, multiple adjacent blocks located to the left of the current block can be searched, and a second affine seed vector prediction candidate can be derived from the first affine adjacent block found.

[0145] Information for determining the affine seed vector difference value can be encoded and transmitted via a bitstream. This information may include size information representing the size of the affine seed vector difference value and symbol information representing the symbol of the affine seed vector difference value. The affine seed vector difference value for each control point can be set to be the same. Alternatively, the affine seed vector difference value can be set to be different for each control point.

[0146] As described above, the affine seed vectors of merge candidates or affine seed vector prediction candidates can be derived from the affine seed vectors of affine adjacent blocks, and the affine seed vector of the current block can be derived using the derived affine seed vectors of the merge candidates or affine seed vector prediction candidates. Alternatively, after searching for multiple affine adjacent blocks according to a predetermined scan order, the affine seed vector of the current block can be derived from the affine seed vector of the first affine adjacent block found.

[0147] Later, a method for deriving the affine seed vector of the current block, merge candidates, or affine seed vector prediction candidates from the affine seed vectors of affine adjacent blocks will be described in detail. In the embodiments described later, the derivation of the affine seed vector of the current block may be understood as the derivation of the affine seed vector of the merge candidates or the derivation of the affine seed vector of the affine seed vector prediction candidates.

[0148] Figure 12 shows how the affine seed vector of the current block is derived based on the affine seed vectors of the affine adjacent blocks.

[0149] When the first affine seed vector nv0 related to the upper left control point and the second affine seed vector nv1 related to the upper right control point are stored in the affine adjacent block, the third affine seed vector nv2 related to the lower left control point of the affine adjacent block can be derived based on the first affine seed vector and the second affine seed vector. Equation 4 shows an example of the derivation of the third affine seed vector.

Equation

[0150] In Equation 4, (nv 0x , nv 0y ) represents the first affine seed vector nv0, (nv 1x , nv 1y ) represents the second affine seed vector nv1, and (nv 2x , nv 2y ) represents the third affine seed vector nv2. Here, (x n0 , x n0 ) represents the position of the first control point, (x n1 , x n1 ) represents the position of the second control point, and (x n2 , x n2 ) represents the position of the third control point.

[0151] Subsequently, the affine seed vector of the current block can be derived using the first affine seed vector, the second affine seed vector, and the third affine seed vector. Equation 5 shows an example of the derivation of the first affine seed vector v0 of the current block, and Equation 6 shows an example of the derivation of the second affine seed vector v1 of the current block.

Equation

Equation

[0152] In Equation 5 and Equation 6, (v 0x , v0y ) represents the first affine seed vector sv0 of the current block, and also, (v 1x ,v 1y ) represents the second affine seed vector sv1 of the current block. Note that (x0, y0) represents the position of the first control point, and (x1, y1) represents the position of the second control point. For example, the first control point represents the upper left corner of the current block, and the second control point represents the upper right corner of the current block.

[0153] In the previous example, we explained how to derive multiple affine seed vectors for the current block using three affine seed vectors related to affine adjacent blocks. As another example, it is also possible to derive the affine seed vector for the current block using only two of the multiple affine seed vectors of the affine adjacent blocks.

[0154] Alternatively, without using the first affine seed vector at the top-left corner, the second affine seed vector at the top-right corner, or the third affine seed vector at the bottom-left corner, which are related to the affine adjacent blocks, the multiple affine seed vectors of the current block can be derived using the fourth affine seed vector related to the bottom-right corner.

[0155] In particular, when the upper boundary of the current block is in contact with the upper boundary of the coding tree unit, and we intend to use the affine seed vector of the upper control point (e.g., upper left corner or upper right corner) of an affine adjacent block adjacent to the upper part of the current block (hereinafter referred to as the upper affine adjacent block), these must be pre-stored in memory. This can lead to the problem of increasing the number of line buffers. Therefore, when the upper boundary of the current block is in contact with the upper boundary of the coding tree unit, it is possible to set the system to use the affine seed vector of the lower control point (e.g., lower left corner or lower right corner) for the upper affine adjacent block without using the affine seed vector of the upper control point. For example, multiple affine seed vectors for the current block can be derived using the third affine seed vector related to the lower left corner and the fourth affine seed vector related to the lower right corner of the upper affine adjacent block. In this case, the affine seed vector related to the lower corner can be derived by duplicating the affine seed vector related to the upper corner, or by deriving it from multiple affine seed vectors related to the upper corners. For example, the first affine seed vector, the second affine seed vector, or the third affine seed vector can be transformed / replaced with the fourth affine seed vector related to the lower right corner.

[0156] Equations 7 and 8 show an example of deriving the first and second affine seed vectors of the current block using the third affine seed vector related to the lower-left control point and the fourth affine seed vector related to the lower-right control point of the adjacent affine vector.

number

number

[0157] In equations 7 and 8, (x n2 ,y n2 ) represents the coordinates of the lower-left control point of the affine adjacent block, and also, (x n3 ,y n3(x0, y0) represents the coordinates of the bottom-right control point of the affine adjacent block. (x0, y0) represents the coordinates of the top-left control point of the current block, and (x1, y1) represents the coordinates of the top-right control point of the current block. (nv 2x ,nv 2y ) represents the affine seed vector (i.e., the third affine seed vector) of the lower left control point of the affine adjacent block, and also, (nv 3x ,nv 3y ) represents the affine seed vector (i.e., the fourth affine seed vector) of the lower-right control point of the affine adjacent block. (v 0x ,v 0y ) represents the affine seed vector of the top-left control point of the current block (i.e., the first affine seed vector), and also, (v 1x ,v 1y ) represents the affine seed vector of the current block's upper right control point (i.e., the second affine seed vector).

[0158] The divisions in equations 7 and 8 can also be changed to shift operations. The shift operation is the width between the lower left control point and the lower right control point (i.e., (x n3 -x n2 It can be executed based on the value derived from )).

[0159] In the above example, multiple affine seed vectors for the current block can be derived based on multiple affine seed vectors of the encoded / decoded affine neighboring blocks. For this purpose, multiple affine seed vectors of the encoded / decoded affine neighboring blocks can be stored in memory. However, since multiple affine seed vectors of the affine neighboring block are stored in memory in addition to multiple translational motion vectors (i.e., multiple affine vectors) of multiple subblocks contained in the affine neighboring block, this leads to the problem of increased memory usage. To resolve this problem, the affine seed vector of the current block can be derived using the motion vector of a subblock adjacent to the control point of the affine neighboring block. This replaces the affine seed vector of the affine neighboring block. In other words, the motion vector of a subblock adjacent to the control point of the affine neighboring block can be set as the affine seed vector of the affine neighboring block. Here, a subblock is a block with a size / shape predefined in the encoder and decoder, or it may be a block with a basic size / shape for storing motion vectors. For example, a subblock may be a square block with a size of 4x4. Alternatively, the motion vector specifying the sample position can be set to the affine seed vector of the affine adjacent block.

[0160] Figure 13 shows an example where the motion vector of a subblock is the affine seed vector of an affine adjacent block.

[0161] The motion vector of a subblock adjacent to a control point can be used as the affine seed vector of the corresponding control point. For example, in the example shown in Figure 13, the motion vector (nv) of the subblock adjacent to the lower left corner of the affine adjacent block (lower left subblock) 4x ,nv 4y ) is the affine seed vector (nv 2x ,nv 2y ) can be set as, and the motion vector (nv) of the subblock adjacent to the bottom right corner (bottom right subblock)5x ,nv 5y ) is the affine seed vector (nv) of the control point in the lower right corner. 3x ,nv 3y ) can be set as follows. Here, the lower left subblock is the lower left control point (x) of the adjacent affine block. n2 ,y n2 ) adjacent to a sample (for example, (x n2 ,y n2-1 This is a subblock containing a sample at the position (x), and the lower right subblock is the lower right control point (x) of the adjacent affine block. n3 ,y n3 ) adjacent to a sample (for example, (x n3-1 ,y n3-1 This is a block containing a sample at position ). When deriving the affine seed vector of the current block based on Equations 7 and 8, the motion vector of the lower-left subblock can be used instead of the third affine seed vector of the affine adjacent block, and the motion vector of the lower-right subblock can be used instead of the fourth affine seed vector.

[0162] In the following examples, a subblock used as the affine seed vector for an affine adjacent block is referred to as an affine subblock.

[0163] According to one embodiment of the present invention, an affine subblock can be determined based on a sample at a specific location. For example, a subblock containing a sample at a specific location can be defined as an affine subblock. Hereinafter, a sample at a specific location will be referred to as an affine reference sample. A reference sample used to determine the affine subblock at the lower left control point will be referred to as the lower left reference sample, and a reference sample used to determine the affine subblock at the lower right control point will be referred to as the lower right reference sample.

[0164] The lower-left reference sample and the lower-right reference sample may be selected from multiple samples contained in the affine adjacent block. For example, at least one of the upper-left sample, lower-left sample, upper-right sample, or lower-left sample of the lower-left subblock may be used as the lower-left reference sample, and at least one of the upper-left sample, lower-left sample, upper-right sample, or lower-left sample of the lower-right subblock may be used as the lower-right reference sample. Therefore, the motion vectors of the lower-left subblock containing the lower-left reference sample and the lower-right subblock containing the lower-right reference sample can be used as the affine seed vector related to the lower-left control point and the affine seed vector related to the lower-right control point, respectively.

[0165] As another example, at least one of the lower-left or lower-right reference samples can be a sample located outside the affine adjacent block. This will be explained in detail with reference to Figures 14 to 16.

[0166] Figures 14 to 16 show the locations of the reference samples.

[0167] For example, as shown in the example in Figure 14(a), the upper left sample of the lower left subblock is referenced to the lower left control point (x n4 ,y n4 ) can be used as the reference sample (x n4 ,y n4 The lower left subblock containing ) can be made into an affine subblock related to the lower left control point.

[0168] For the lower right control point, the sample located to the right of the upper right sample in the lower right subblock is the reference sample (x n5 ,y n5 ) can be used as the reference sample (x n5 ,y n5 A subblock adjacent to the right of the lower right subblock containing ) can be made an affine subblock related to the lower right control point.

[0169] Alternatively, as shown in the example in Figure 14(b), a reference sample (x) is defined as the sample located to the left of the upper left sample of the lower left subblock relative to the lower left control point. n4 ,y n4 ) can be used as the reference sample (x n4 ,y n4 A subblock adjacent to the left of the lower left subblock containing ) can be designated as an affine subblock related to the lower left control point.

[0170] For the lower right control point, the upper right sample of the lower right subblock is referenced (x n5 ,y n5 ) can be used as the reference sample (x n5 ,y n5 The lower right subblock containing ) can be made into an affine subblock related to the lower right control point.

[0171] Alternatively, as shown in the example in Figure 15(a), the lower left sample of the lower left subblock is used as a reference sample (x n4 ,y n4 ) can be used as the reference sample (x n4 ,y n4 The lower left subblock containing ) can be made into an affine subblock related to the lower left control point.

[0172] For the lower right control point, the sample located to the right of the lower right sample in the lower right subblock is the reference sample (x n5 ,y n5 ) can be used as the reference sample (x n5 ,y n5 A subblock adjacent to the right of the lower right subblock containing ) can be made an affine subblock related to the lower right control point.

[0173] Alternatively, as shown in the example in Figure 15(b), a reference sample (x) is defined as the sample located to the left of the lower left sample in the lower left subblock relative to the lower left control point. n4 ,y n4 ) can be used as the reference sample (xn4 , y n4 The sub-block adjacent to the left side of the lower-left sub-block including ) can be set as an affine sub-block related to the lower-left control point.

[0174] For the lower-right control point, the lower-right sample of the lower-right sub-block can be set as the reference sample (x n5 , y n5 ). Therefore, the lower-right sub-block including the reference sample (x n5 , y n5 ) can be set as an affine sub-block related to the lower-right control point.

[0175] Alternatively, as in the example shown in (a) of FIG. 16, for the lower-left control point, the sample located between the upper-left sample and the lower-left sample of the lower-left sub-block (for example, the left intermediate sample) can be set as the reference sample (x n4 , y n4 ). Therefore, the lower-left sub-block including the reference sample (x n4 , y n4 ) can be set as an affine sub-block related to the lower-left control point.

[0176] For the lower-right control point, the sample located on the right side of the sample between the upper-right sample and the lower-right sample of the lower-right sub-block (for example, the right intermediate sample) can be set as the reference sample (x n5 , y n5 ). Therefore, the sub-block adjacent to the right side of the lower-right sub-block including the reference sample (x n5 , y n5 ) can be set as an affine sub-block related to the lower-right control point.

[0177] Alternatively, as in the example shown in (b) of FIG. 16, for the lower-left control point, the sample located on the left side of the sample between the upper-left sample and the lower-left sample of the lower-left sub-block can be set as the reference sample (x n4 , y n4 ). Therefore, the reference sample (x n4 , y n4The sub-block adjacent to the left side of the lower-left sub-block including can be set as the affine sub-block related to the lower-left control point.

[0178] For the lower-right control point, the sample located between the upper-right sample and the lower-right sample of the lower-right sub-block can be set as the reference sample (x n5 , y n5 ). Therefore, the lower-right sub-block including the reference sample (x n5 , y n5 ) can be set as the affine sub-block related to the lower-right control point.

[0179] When deriving a plurality of affine seed vectors of the current block based on Equation 7 and Equation 8, instead of the third affine seed vector of the affine adjacent block, the motion vector of the affine sub-block related to the lower-left control point can be used, and instead of the fourth affine seed vector, the motion vector of the affine sub-block related to the lower-right control point can be used. Note that instead of the position of the lower-left control point, the position of the lower-left reference sample can be used, and instead of the position of the lower-right control point, the position of the lower-right reference sample can be used.

[0180] Different from the descriptions in FIGS. 14 to 16, the sub-block including the sample adjacent to the reference sample can also be set as the affine sub-block. Specifically, the sample located outside the affine adjacent sub-block can be set as the reference sample, and the sub-block included in the affine adjacent block can be set as the affine sub-block. For example, in the example shown in (a) of FIG. 14, the sample located on the right side of the upper-right sample of the lower-right sub-block can be set as the reference sample (x n5 , y n5 ), and the lower-right sub-block can be set as the affine sub-block related to the lower-right corner. Alternatively, in the example shown in (b) of FIG. 14, the sample located on the left side of the upper-left sample of the lower-left sub-block can be set as the reference sample (x n4 , y n4) can be made into an affine subblock related to the lower left corner.

[0181] The embodiments shown in Figures 15 and 16 can be applied similarly. That is, in the example shown in Figure 15(a) or Figure 16(a), the sample located to the right of the lower right sample of the lower right subblock or the sample located to the right of the middle right sample is used as the reference sample (x n5 ,y n5 ) can be made into a reference sample (x n4 ,y n4 ) can be made into an affine subblock related to the lower left corner.

[0182] In the example described above, the affine seed vector of an affine adjacent block can be derived by using the motion vector of an affine subblock. For this reason, with respect to encoded / decoded blocks, the motion vector can be stored on a subblock basis.

[0183] As another example, after storing a minimum number of affine seed vectors in an affine adjacent block, the motion vectors of affine subblocks can be derived using the stored affine seed vectors.

[0184] Equations 9 and 10 illustrate an example of deriving the motion vector of an affine subblock using the affine seed vector of an affine adjacent block.

number

number

[0185] In equations 9 and 10, (nv 4x ,nv 4y ) represents the motion vector of the affine subblock related to the lower left control point, and also, (nv 5x ,nv 5y ) represents the motion vector of the affine subblock related to the lower right control point. Since the motion vector of the affine subblock and the affine seed vector of the control point are set to be the same, (nv 4x ,nv 4y Instead of ), use the affine seed vector (nv) related to the lower left control point. 2x ,nv 2y ) can be used, or (nv 5x ,nv 5y Instead of ), use the affine seed vector (nv) related to the lower right control point. 3x ,nv 3y ) can be used.

[0186] (x n4 ,y n4 ) represents the position of the reference sample in the lower left subblock. Alternatively, the center position of the lower left subblock or the position of the lower left control point can be used instead. (x n5 ,y n5 ) represents the position of the reference sample in the lower right subblock. Alternatively, the center position of the lower right subblock or the position of the lower right control point can be used instead.

[0187] Equations 9 and 10 can be applied if the current block does not touch the boundary of the coding tree unit. If the current block touches the upper boundary of the coding tree unit, instead of applying equations 9 and 10, the translational motion vector of the affine subblock determined based on the lower-left reference sample can be used as the third affine seed vector, and the translational motion vector of the affine subblock determined based on the lower-right reference sample can be used as the fourth affine seed vector.

[0188] In equations 7 and 8, (x n3 -x n2) represents the width between the lower left control point and the lower right control point. As mentioned above, x n3 Instead, the position x of the reference sample in the lower right n5 You can use x n2 Instead, the position x of the reference sample in the lower left. n4 You can use the following, (x n3 -x n2 ) or a value obtained by using the position of the reference sample instead of the above equation (for example, (x n5 -x n4 )) to variable W seed This is defined as follows, and the aforementioned variable is called the subseed vector width.

[0189] According to the reference sample's position, the subseed vector width is a power series of 2 (for example, 2 n ) may occur. For example, if the bottom-left sample of the bottom-left subblock is used as the bottom-left reference sample and the bottom-right sample of the bottom-right subblock is used as the bottom-right reference sample, the sub-seed vector width is not a multiple of 2. As mentioned above, if the sub-seed vector width is not a power of 2, it can be converted to a power of 2. The conversion may include adding / subtracting an offset to the sub-seed vector width, or using the position of a sample adjacent to the reference sample instead of the position of the reference sample. For example, the converted sub-seed vector width can be derived by adding 1 to the width between the bottom-left reference sample and the bottom-right reference sample. Alternatively, the width between the adjacent reference sample to the right of the bottom-right reference sample and the bottom-left reference sample can be used as the converted sub-seed vector width. Subsequently, the affine seed vector of the current block can be derived by substituting the converted sub-seed vector width into equations 7 and 8.

[0190] The divisions in equations 7 and 8 can also be changed to shift operations. These shift operations can be performed based on values ​​derived from the transformed sub-seed vector width (i.e., values ​​expressed as a power series of 2).

[0191] If the reference sample used to determine an affine subblock does not belong to an affine adjacent block, the affine seed vector of the affine adjacent block can be derived based on the samples adjacent to the reference sample among the multiple samples contained in the affine adjacent block. Specifically, the translational motion vector of the subblock containing the sample adjacent to the reference sample in the affine adjacent block (hereinafter referred to as the adjacent reference sample) can be used as the affine seed vector of the affine adjacent block. As described above, the method of deriving the affine seed vector using adjacent reference samples can be defined as the modified affine merge vector derivation method.

[0192] Figure 17 shows an example of applying the modified affine merge vector derivation method.

[0193] Affine adjacent block E lower right reference sample (x n5 ,y n5 If ) does not belong to an affine adjacent block, then among the samples included in the affine adjacent block, the sample adjacent to the left of the lower right reference sample (x n5-1 ,y n5 Based on this, the affine seed vector can be derived. Specifically, the adjacent reference sample (x n5-1 ,y n5 The translational motion vector of the subblock containing ) can be used as the affine seed vector of the lower-right control point.

[0194] In the example shown in Figure 17, the sample adjacent to the right of the upper right sample in the lower right subblock is shown as the lower right reference sample. If the sample adjacent to the right of the lower right sample in the lower right subblock, or the sample adjacent to the right of the right middle sample in the lower right subblock, is designated as the lower right reference sample, then the affine seed vector can be derived based on the sample adjacent to the left of the adjacent reference sample.

[0195] Furthermore, if the lower-left reference sample does not belong to an affine adjacent block, the affine seed vector can also be derived based on the sample adjacent to the right of the lower-left reference sample, depending on the described embodiment.

[0196] By setting the reference sample position and the subblock for affine seed vector derivation using different methods, the subseed vector width can be made a power series of 2.

[0197] By using adjacent blocks around the current block that are not encoded in affine intermode, merge candidates, affine seed vector prediction candidates, or affine seed vectors for the current block can be derived. Specifically, blocks that are not encoded in affine intermode can be combined, and this combination can be used as a merge candidate or an affine seed vector prediction candidate. For example, at least one combination of the motion vector of any one adjacent block adjacent to the upper left corner of the current block, the motion vector of any one adjacent block adjacent to the upper right corner of the current block, and the motion vector of any one adjacent block adjacent to the lower left corner of the current block can be used as a merge candidate or an affine seed vector prediction candidate. In this case, the motion vector of the adjacent block adjacent to the upper left corner, the motion vector of the adjacent block adjacent to the upper right corner, and the motion vector of the adjacent block adjacent to the lower left corner can be used as the first affine seed vector of the upper left control point, the second affine seed vector of the upper right control point, and the third affine seed vector of the lower left control point, respectively.

[0198] Alternatively, in the modified affine merge vector derivation method described above, merge candidates for the current block, affine seed vector prediction candidates, or affine seed vectors can be derived using adjacent blocks that are not encoded in affine intermode. In the following, adjacent blocks that are not encoded in affine intermode are referred to as non-affine adjacent blocks.

[0199] Figure 18 shows an example of deriving the affine seed vector of the current block based on non-affine adjacent blocks.

[0200] In the example shown in Figure 18, we assume that all adjacent blocks to the current block are non-affine adjacent blocks.

[0201] When attempting to derive the affine seed vector of the current block from a non-affine adjacent block A that is adjacent to the current block, the lower-left and lower-right reference samples of A can be set. For example, the sample adjacent to the left of the lower-left sample of block A can be set as the lower-left reference sample, and the lower-right sample of block A can be set as the lower-right reference sample. Since the lower-left reference sample is outside block A, the motion vector of the subblock containing the sample adjacent to the right of the lower-left reference sample can be set as the third affine seed vector of block A. Furthermore, the motion vector of the subblock containing the lower-right reference sample can be set as the fourth affine seed vector of block A. Subsequently, based on equations 9 and 10, the first and second affine seed vectors of the current block can be derived from block A.

[0202] The method for deriving the affine seed vector from the non-affine adjacent block can be used only when performing motion compensation prediction for non-affine adjacent blocks on a subblock basis. Here, the prediction technique for performing motion compensation prediction on a subblock may include at least one of the following: STMVP, ATMVP, Bidirectional Optical Flow (BIO), Overlap Block Motion Compensation (OBMC), and Decoder-Side Motion Vector Modification (DMVR).

[0203] In the above embodiment, when the upper boundary of the current block contacts the boundary of the coding tree unit, the third affine seed vector of the lower left control point and the fourth affine seed vector of the lower right control point of the affine adjacent block located above the current block are used to derive the merge candidate, affine seed vector prediction candidate, or affine seed vector of the current block.

[0204] As another example, if the upper boundary of the current block is in contact with the boundary of a coding tree unit, and an adjacent block located above the current block belongs to a different coding tree unit than the current block, then the merge candidate for the current block, the affine seed vector prediction candidate, or the affine seed vector is derived using the adjacent block that is closest to the aforementioned adjacent block among these blocks included in the coding tree unit to which the current block belongs, without using the aforementioned adjacent block.

[0205] In the example shown in Figure 19, the current block is in contact with the upper boundary of the coding tree unit, and blocks B, C, and E, located above the current block, belong to different coding tree units than the current block. Therefore, instead of block E, we use block F, which is adjacent to block E and is included in the coding tree unit to which the current block belongs, to derive the affine seed vector of the current block.

[0206] For motion compensation prediction of the current block, affine seed vectors of multiple blocks can be used. For example, multiple merge candidates can be selected from a list of merge candidates, and the affine seed vector or subblock vector of the current block can be derived based on the affine seed vectors of the selected merge candidates. Performing encoding / decoding on the current block using affine seed vectors of multiple blocks may be called a multi-affine merge coding method.

[0207] Encoding can be performed on information indicating whether the multi-affine merge encoding method is applied to the current block, and it can be transmitted via a bitstream. Alternatively, based on at least one of the number of affine adjacent blocks among adjacent blocks adjacent to the current block, the number of merge candidates included in the merge candidate list, and the affine motion model of the current block, it is possible to determine whether to apply the multi-affine merge encoding method to the current block.

[0208] FIG. 20 and FIG. 21 are flowcharts showing a motion compensation prediction method using a plurality of merge candidates.

[0209] FIG. 20 is a diagram showing an example of deriving the affine seed vector of the current block by using the affine seed vectors of a plurality of merge candidates. FIG. 21 is a diagram showing an example of deriving the motion vectors of each sub-block by using the affine seed vectors of a plurality of merge candidates.

[0210] The affine seed vector of the current block can be generated based on the sum, difference, average value, or weighted addition of the affine seed vectors of two merge candidates.

[0211] The following equations 11 and 12 show examples of deriving the affine seed vector of the current block by adding the affine seed vectors of the merge candidates.

Equation

Equation

[0212] In equations 11 and 12, sv4 represents the first affine seed vector of the current block, sv0 represents the first affine seed vector of the first merge candidate, and sv2 represents the first affine seed vector of the second merge candidate. Furthermore, sv5 represents the second affine seed vector of the current block, sv1 represents the second affine seed vector of the first merge candidate, and sv3 represents the second affine seed vector of the second merge candidate.

[0213] Equations 13 and 14 below illustrate an example of deriving the affine seed vector of the current block by weighted addition of the affine seed vectors of merge candidates.

number

number

[0214] As another example, based on the affine seed vectors of the first and second merge candidates, a first subblock motion vector and a second subblock motion vector can be generated for each subblock in the current block. Subsequently, the final subblock motion vector can be generated based on the sum, difference, average, or weighted addition of the first and second subblock motion vectors.

[0215] Equation 15 below shows an example of obtaining the final subblock motion vector by adding the first subblock motion vector and the second subblock motion vector.

number

[0216] In equation 15, V0 represents the first subblock motion vector, V1 represents the second subblock motion vector, and V2 represents the final subblock motion vector.

[0217] Furthermore, equation 16 below shows an example of deriving the final subblock motion vector by weighted addition of the first subblock motion vector and the second subblock motion vector.

number

[0218] Intra-prediction is the process of predicting the current block using encoded / decoded reconstructed samples from the surrounding area. In this case, the intra-prediction of the current block can use reconstructed samples before applying the in-loop filter.

[0219] Intra-prediction techniques include matrix-based intra-prediction and general intra-prediction that considers the orientation with respect to surrounding reconstructed samples. Information indicating the intra-prediction technique of the current block can be transmitted via a bitstream signal. This information may be a 1-bit flag. Alternatively, the intra-prediction technique of the current block can be determined based on at least one of the current block's location, size, shape, or the intra-prediction techniques of adjacent blocks. For example, if the current block straddles an image boundary, the current block may be configured not to have matrix-based intra-prediction applied.

[0220] Matrix-based intra-prediction is a method for obtaining a predicted block of the current block based on matrix multiplication of a matrix stored in an encoder and decoder with a reconstructed sample around the current block. Information indicating one of several stored matrices can be transmitted via a bitstream signal. The decoder can determine the matrix to be used for the intra-prediction of the current block based on this information and the size of the current block.

[0221] General intra prediction is a method of obtaining a predicted block of a current block based on a non - angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in detail while referring to the drawings.

[0222] FIG. 22 is a flowchart showing an intra prediction method according to an embodiment of the present invention.

[0223] A reference sample line of the current block can be determined (S2201). The reference sample line is a set of reference samples included in the K - th line shifted from above and / or to the left of the current block. The reference samples can be derived from the encoded / decoded reconstructed samples around the current block.

[0224] Through a bitstream, index information for labeling, with a signal, the reference sample line of the current block among a plurality of reference sample lines can be transmitted. The plurality of reference sample lines may be included in at least one of the first line, the second line, the third line, or the fourth line located above and / or to the left of the current block. Table 1 shows the indexes assigned to each reference sample line. Assume that in Table 1, the first line, the second line, and the fourth line are used as reference sample line candidates.

Table 1

[0225] The reference sample line of the current block can also be determined based on at least one of the position, size, shape of the current block, or the prediction coding mode of an adjacent block. For example, when the current block is adjacent to the boundary of an image, tile, slice, or coding tree unit, the first reference sample line can be determined as the reference sample line of the current block.

[0226] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper and left reference samples can be derived from reconstructed samples around the current block. The reconstructed samples may be in their state before the application of the in-loop filter.

[0227] Figure 23 shows the reference samples included in each reference sample line.

[0228] Currently, according to the intra-prediction mode of the block, a predicted sample can be obtained using at least one of the reference samples belonging to the reference sample line.

[0229] Next, the intra-prediction mode of the current block can be determined (S2202). With respect to the intra-prediction mode of the current block, at least one of the non-angle intra-prediction mode or the angle intra-prediction mode can be determined as the intra-prediction mode of the current block. The non-angle intra-prediction mode includes Planar and DC, and the angle intra-prediction mode includes 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

[0230] Figure 24 shows the intra-prediction mode.

[0231] Figure 24(a) shows 35 intra-prediction modes, and Figure 24(b) shows 67 intra-prediction modes.

[0232] You can also define more or fewer intra-prediction modes than the number shown in Figure 24.

[0233] The Most Probable Mode (MPM) can be set based on the intra-prediction mode of the adjacent blocks adjacent to the current block. Here, adjacent blocks may include the left adjacent block adjacent to the left of the current block and the upper adjacent block adjacent to the current block above it. If the coordinates of the upper left sample of the current block are (0,0), the left adjacent block may include a sample at the positions (-1,0), (-1,H-1), or (-1,(H-1) / 2), where H represents the height of the current block. The upper adjacent block may include a sample at the positions (0,-1), (W-1,-1), or ((W-1) / 2,-1), where W represents the width of the current block.

[0234] When encoding adjacent blocks using general intra-prediction, the MPM can be derived based on the intra-prediction mode of the adjacent block. Specifically, the intra-prediction mode of the left adjacent block can be denoted as variable candIntraPredModeA, and the intra-prediction mode of the upper adjacent block can be denoted as variable candIntraPredModeB.

[0235] In this case, when an adjacent block is unavailable (for example, when an adjacent block has not been encoded / decoded or when the adjacent block is located outside the image boundary), when an adjacent block is encoded by matrix-based intra-prediction, when an adjacent block is encoded by inter-prediction, or when an adjacent block is included in a different coding tree unit than the current block, the variable candIntraPredModeX (where X is A or B), derived based on the intra-prediction mode of the adjacent block, can be set as the default mode. Here, the default mode may include at least one of the planar mode, DC mode, vertical mode, or horizontal mode.

[0236] Alternatively, when encoding adjacent blocks by matrix-based intra-prediction, the intra-prediction mode corresponding to the index value for specifying any one of the matrices can be set to candIntraPredModeX. For this purpose, a lookup table showing the mapping relationship between the index value for specifying the matrix and the intra-prediction mode can be pre-stored in the encoder and decoder.

[0237] The MPM can be derived based on the variables candIntraPredModeA and candIntraPredModeB. In the encoder and decoder, the number of MPMs included in the MPM list can be predefined. For example, the number of MPMs may be 3, 4, 5, or 6. Alternatively, information representing the number of MPMs can be transmitted as a signal via the bitstream. Alternatively, the number of MPMs can be determined based on at least one of the predictive coding mode of the adjacent block, the size of the current block, or its shape.

[0238] In the embodiments described later, we assume that there are three MPMs, and these three MPMs are referred to as MPM[0], MPM[1], and MPM[2]. If there are more than three MPMs, the MPMs may include the three MPMs described in the embodiments described later.

[0239] If candIntraPredA is the same as candIntraPredB, and candIntraPredA is in planar mode or DC mode, then MPM[0] and MPM[1] can be in planar mode and DC mode, respectively. MPM[2] can be in vertical intra-prediction mode, horizontal intra-prediction mode, or diagonal intra-prediction mode. The diagonal intra-prediction mode may be the lower left diagonal intra-prediction mode, the upper left intra-prediction mode, or the upper right intra-prediction mode.

[0240] If candIntraPredA is the same as candIntraPredB and candIntraPredA is in intra-prediction mode, MPM[0] can be set to be the same as candIntraPredA. MPM[1] and MPM[2] can be set to intra-prediction modes similar to candIntraPredA. An intra-prediction mode similar to candIntraPredA may be an intra-prediction mode where the index difference value with candIntraPredA is ±1 or ±2. An intra-prediction mode similar to candIntraPredA can be derived using modulo operation (%) and offset.

[0241] If candIntraPredA is different from candIntraPredB, MPM[0] and candIntraPredA can be set to be the same, and MPM[1] and candIntraPredB can be set to be the same. In this case, if both candIntraPredA and candIntraPredB are in non-angle intra-prediction mode, MPM[2] can be set to vertical intra-prediction mode, horizontal intra-prediction mode, or diagonal intra-prediction mode. Alternatively, if at least one of candIntraPredA and candIntraPredB is in angle intra-prediction mode, MPM[2] can be set to an intra-prediction mode derived by adding or subtracting an offset to the larger of the following values: plane, DC, candIntraPredA, or candIntraPredB. Here, the offset may be 1 or 2.

[0242] An MPM list containing multiple MPMs can be generated, and information can be transmitted via a bitstream, in the form of a signal, indicating whether an MPM that is the same as the intra-prediction mode of the current block is included in the MPM list. This information is a 1-bit flag and may be called the MPM flag. If the MPM flag indicates that an MPM that is the same as the current block is included in the MPM list, an index information identifying one of the MPMs can be transmitted via a bitstream, in the form of a signal. The MPM specified by the index information can be set as the intra-prediction mode of the current block. If the MPM flag indicates that an MPM that is the same as the current block is not included in the MPM list, residual mode information indicating one of the residual intra-prediction modes other than an MPM can be transmitted via a bitstream, in the form of a signal. The residual mode information represents the index value corresponding to the intra-prediction mode of the current block when reassigning an index to a residual intra-prediction mode other than an MPM. The decoder can determine the intra-prediction mode of the current block by arranging the MPMs in ascending order and by comparing the residual mode information with the MPMs. For example, if the residual mode information is the same as or smaller than the MPM, the intra-predictive mode of the current block can be derived by adding 1 to the residual mode information.

[0243] Instead of setting the default mode to an MPM, a signal can be transmitted via a bitstream indicating whether the current block's intra-prediction mode is the default mode. This information is a 1-bit flag, which may also be called the default mode flag. The default mode flag can only be transmitted via a signal if the MPM flag indicates that the MPM that is the same as the current block is included in the MPM list. As described above, the default mode may include at least one of planar, DC, vertical, or horizontal modes. For example, if planar is the default mode, the default mode flag may indicate whether the current block's intra-prediction mode is planar. If the default mode flag indicates that the current block's intra-prediction mode is not the default mode, one of the MPMs indicated by the index information may be set as the current block's intra-prediction mode.

[0244] If multiple intra-prediction modes are set as the default mode, an index information indicating one of the default modes can be transmitted via a signal. The current block's intra-prediction mode can be set as the default mode indicated by the index information.

[0245] If the index of the reference sample line in the current block is not 0, the default mode is set not to be used. Therefore, if the index of the reference sample line is not 0, the default mode flag can be set to a predefined value (i.e., false) without sending a signal to the default mode flag.

[0246] If the intra-prediction mode for the current block is determined, prediction samples related to the current block can be obtained based on the determined intra-prediction mode (S2203).

[0247] When DC mode is selected, predicted samples related to the current block can be generated based on the mean value of the reference samples. Specifically, values ​​for all samples in the predicted block can be generated based on the mean value of the reference samples. The mean value can be derived using at least one of the upper reference samples located above the current block and the left reference samples located to the left of the current block.

[0248] The number or range of reference samples used to derive the mean may vary depending on the shape of the current block. For example, if the current block is a non-square block where the width is greater than the height, the mean can be calculated using only the upper reference samples. On the other hand, if the current block is a non-square block where the width is less than the height, the mean can be calculated using only the left reference samples. In other words, if the width and height of the current block are different, the mean can be calculated using only the reference samples adjacent to the longer length. Alternatively, it can be determined whether to calculate the mean using only the upper reference samples or only the left reference samples based on the ratio of the width to the height of the current block.

[0249] When planar mode is selected, prediction samples can be obtained using horizontal and vertical prediction samples. Here, horizontal prediction samples are obtained based on left and right reference samples located on the horizontal line that are the same as the prediction samples, and vertical prediction samples are obtained based on upper and lower reference samples located on the vertical line that are the same as the prediction samples. Here, a right reference sample can be generated by duplicating a reference sample adjacent to the upper right corner of the current block, and a lower reference sample can be generated by duplicating a reference sample adjacent to the lower left corner of the current block. Horizontal prediction samples can be obtained based on the weighted addition of the left and right reference samples, and vertical prediction samples can be obtained based on the weighted addition of the upper and lower reference samples. In this case, the weight value assigned to each reference sample can be determined based on the position of the prediction sample. Prediction samples can be obtained based on the averaging or weighted addition of the horizontal and vertical prediction samples. When performing weighted addition, the weight value assigned to the horizontal and vertical prediction samples can be determined based on the position of the prediction samples.

[0250] When an angle prediction mode is selected, you can determine the parameters that represent the prediction direction (or prediction angle) for the selected angle prediction mode. Table 2 below shows the intra-prediction parameters intraPredAng for each intra-prediction mode. [Table 2]

[0251] Table 2 shows the intra-direction parameters for each intra-prediction mode that has one index from 2 to 34 when 35 intra-prediction modes are defined. If more than 33 angle intra-prediction modes are defined, Table 2 further details the intra-direction parameters for each angle intra-prediction mode.

[0252] Currently, after arranging the upper and left reference samples of a block into a single row, a predicted sample can be obtained based on the value of the intra-direction parameter. In this case, if the value of the intra-direction parameter is negative, the left and upper reference samples can be arranged into a single row.

[0253] Figures 25 and 26 show examples of one-dimensional arrays in which reference samples are arranged in a single row.

[0254] Figure 25 shows an example of a vertical one-dimensional array in which reference samples are arranged vertically, and Figure 26 shows an example of a horizontal one-dimensional array in which reference samples are arranged horizontally. The embodiments of Figures 25 and 26 are described after determining that 35 intra-prediction modes have been defined.

[0255] If the intra-prediction mode index is any one between 11 and 18, a horizontal one-dimensional array can be applied, rotating the upper reference sample counterclockwise. Similarly, if the intra-prediction mode index is any one between 19 and 25, a vertical one-dimensional array can be applied, rotating the left reference sample clockwise. If the reference samples are arranged in a single row, the intra-prediction mode angle can be considered.

[0256] Based on the intra-direction parameter, the reference sample determination parameter can be determined. The reference sample determination parameter may include a reference sample index for specifying the reference sample and a weight parameter for determining the weight applied to the reference sample.

[0257] The reference sample index iIdx and the weighted parameter ifact are obtained by equations 17 and 18 below, respectively.

number

number

[0258] In equations 17 and 18, P ang This represents the intra-direction parameter. The reference sample specified by the reference sample index iIdx corresponds to an integer pixel (Integer pel).

[0259] To derive prediction samples, one or more reference samples can be specified. Specifically, the positions of the reference samples for deriving prediction samples can be specified, taking into account the gradient of the prediction mode. For example, the reference sample index iIdx can be used to specify the reference samples for deriving prediction samples.

[0260] In this case, if the gradient of the intra-prediction mode is not represented by a single reference sample, a prediction sample can be generated by interpolating across multiple reference samples. For example, if the gradient of the intra-prediction mode is a value between the gradient between the prediction sample and the first reference sample, and the gradient between the prediction sample and the second reference sample, a prediction sample can be obtained by interpolating across the first and second reference samples. In other words, if the angular line that follows the intra-prediction angle does not pass through a reference sample located at an integer pixel, a prediction sample can be obtained by interpolating across reference samples adjacent to the left, right, or above and below the position through which the angular line passes.

[0261] Equation 19 below shows an example of obtaining predicted samples based on reference samples.

number

[0262] In Equation 19, P represents a predicted sample, and Ref_1D represents one of the reference samples in the one-dimensional array. In this case, the position of the reference sample can be determined based on the position (x,y) of the predicted sample and the reference sample index iIdx.

[0263] If the gradient of the intra-prediction mode is represented as a single reference sample, the weight parameter ifact can be set to 0. Therefore, equation 19 can be simplified to equation 20 below.

number

[0264] It is also possible to perform intra-prediction on the current block based on multiple intra-prediction modes. For example, intra-prediction modes can be derived for different prediction samples, and prediction samples can be derived based on the intra-prediction modes assigned to each prediction sample.

[0265] Alternatively, intra-prediction modes can be derived for different regions, and intra-prediction can be performed for each region based on the intra-prediction mode assigned to each region. Here, the region may contain at least one sample. The size or shape of the region can be adaptively determined based on at least one of the current block size, shape, or intra-prediction mode. Alternatively, in the encoder and decoder, at least one of the size or shape of the region can be predefined regardless of the current block size or shape.

[0266] Alternatively, intra-prediction can be performed based on multiple intra-predictions, and the final predicted sample can be derived based on the averaging or weighted addition of multiple predicted samples obtained from multiple intra-predictions. For example, the first predicted sample can be obtained by performing intra-prediction based on the first intra-prediction mode, and the second predicted sample can be obtained by performing intra-prediction based on the second intra-prediction mode. Subsequently, the final predicted sample can be obtained based on the averaging or weighted addition of the first and second predicted samples. In this case, the weight values ​​assigned to the first and second predicted samples can be determined by considering whether the first intra-prediction mode is a non-angle / angle prediction mode, whether the second intra-prediction mode is a non-angle / angle prediction mode, or at least one of the intra-prediction modes of the adjacent block.

[0267] Multiple intra-prediction modes may be combinations of non-angle intra-prediction modes and angle prediction modes, combinations of angle prediction modes, or combinations of non-angle prediction modes.

[0268] Figure 27 shows the angle formed between the angle intra-prediction mode and a line perpendicular to the x-axis.

[0269] In the example shown in Figure 27, the angle prediction mode may exist between the lower left diagonal direction and the upper right diagonal direction. When described as the angle formed by the x-axis and the angle prediction mode, the angle prediction mode may exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0270] If the block is currently non-square in shape, based on the current block's intra-prediction mode, the predicted sample is derived using the reference sample furthest from the predicted sample, without using any reference samples close to the predicted sample, from among the reference samples located on the angle line that follows the intra-prediction angle.

[0271] Figure 28 shows an example of obtaining a prediction sample when the block is currently non-square.

[0272] For example, in the example shown in Figure 28(a), assume that the current block is a non-square shape with a width greater than its height, and that the current block's intra-prediction mode is an angle intra-prediction mode with angles from 0 to 45 degrees. In this case, when deriving a neighboring prediction sample A in the right column of the current block, a left-side reference sample L, which is far from the prediction sample, may be used instead of an upper reference sample T that is close to the prediction sample, from among the reference samples in the angle mode of the aforementioned angle.

[0273] In another example, in the example shown in Figure 28(b), assume that the current block is a non-square shape with height greater than width, and that the current block's intra-prediction mode is an angular intra-prediction mode with angles from -90 to -135 degrees. In the above case, when deriving a neighborhood predictive sample A in the lower row of the current block, an upper reference sample T, which is far from the predictive sample, may be used in place of a left-side reference sample L that is close to the predictive sample, from among the reference samples in the angular mode of the angle.

[0274] To solve the above-mentioned problem, if the block is currently non-square, the intra-prediction mode for the current block can be replaced with an intra-prediction mode in the reverse direction. Thus, for non-square shaped blocks, an angle prediction mode with an angle greater than or smaller than the angle of the angle prediction mode shown in Figure 24 can be used. Such an angle intra-prediction mode may be defined as a wide-angle intra-prediction mode. A wide-angle intra-prediction mode represents an angle intra-prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0275] Figure 29 shows the wide-angle intra-prediction mode.

[0276] In the example shown in Figure 29, the intra-prediction modes with indices from -1 to -14 and the intra-prediction modes with indices from 67 to 80 represent wide-angle intra-prediction modes.

[0277] Figure 29 shows 14 wide-angle intra-prediction modes with angles greater than 45 degrees (-1 to -14) and 14 wide-angle intra-prediction modes with angles less than -135 degrees (67 to 80), but more or fewer wide-angle intra-prediction modes can be defined.

[0278] When using wide-angle intra-prediction mode, the length of the upper reference sample is set to 2W+1, and the length of the left reference sample is set to 2H+1.

[0279] When using the wide-angle intra-prediction mode, sample A shown in Figure 28(a) can be predicted using reference sample T, and sample A shown in Figure 28(b) can be predicted using reference sample L.

[0280] By adding the existing intra-prediction modes to N wide-angle intra-prediction modes, a total of 67+N intra-prediction modes can be used. For example, Table 3 shows the intra-direction parameters of the intra-prediction modes when 20 wide-angle intra-prediction modes are defined. [Table 3]

[0281] If the current block is not square in shape and the intra-prediction mode of the current block obtained in step S2202 falls within the conversion range, the intra-prediction mode of the current block can be converted to a wide-angle intra-prediction mode. The conversion range can be determined based on at least one of the size, shape, or ratio of the current block. Here, the ratio can represent the ratio of the width to the height of the current block.

[0282] If the current block is a non-square with a width greater than its height, the conversion range can be set from the upper right diagonal intra-prediction mode index (e.g., 66) to (the upper right diagonal intra-prediction mode index - N), where N may be determined based on the proportions of the current block. If the intra-prediction mode of the current block falls within the conversion range, the intra-prediction mode can be converted to a wide-angle intra-prediction mode. This conversion can be performed by subtracting a predefined value from the intra-prediction mode. The predefined value may be the total number of intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 67).

[0283] According to the above embodiment, the intra-prediction modes between the 66th and 53rd can be converted to wide-angle intra-prediction modes between the -1st and -14th, respectively.

[0284] If the current block is a non-square with height greater than width, the conversion range can be set from the intra-prediction mode index in the lower-left diagonal direction (e.g., 2) to (index of the intra-prediction mode in the lower-left diagonal direction + M), where M may be determined based on the proportions of the current block. If the intra-prediction mode of the current block falls within the conversion range, the intra-prediction mode can be converted to a wide-angle intra-prediction mode. This conversion can be performed by adding a predefined value to the intra-prediction mode. The predefined value may be the total number of angle intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 65).

[0285] According to the above embodiment, the intra-prediction modes between the 2nd and 15th can be converted to wide-angle intra-prediction modes between the 67th and 80th, respectively.

[0286] Hereafter, intra-prediction modes that fall within the conversion range will be referred to as wide-angle intra-replacement prediction modes.

[0287] The conversion range may be determined based on the current block ratio. For example, Tables 4 and 5 show the conversion ranges when 35 intra-prediction modes other than the wide-angle intra-prediction mode and 67 intra-prediction modes are defined, respectively. [Table 4] [Table 5]

[0288] As shown in the examples in Tables 4 and 5, the number of wide-angle intra-replacement prediction modes that fall within the conversion range may vary depending on the ratio of the current blocks.

[0289] The addition of a wide-angle intra-prediction mode to the existing intra-prediction mode increases the resources required for encoding the wide-angle intra-prediction mode. This may reduce encoding efficiency. Therefore, instead of directly encoding the wide-angle intra-prediction mode, encoding is performed on a replacement intra-prediction mode related to the wide-angle intra-prediction mode to improve encoding efficiency.

[0290] For example, when encoding the current block using the 67th wide-angle intra-prediction mode, the number 2, which is the 67th wide-angle replacement intra-prediction mode, can be encoded into the intra-prediction mode of the current block. Similarly, when encoding the current block using the -1st wide-angle intra-prediction mode, the number 66, which is the -1st wide-angle replacement intra-prediction mode, can be encoded into the intra-prediction mode of the current block.

[0291] The decoder can decode the current block's intra-prediction mode and determine whether the decoded intra-prediction mode falls within the conversion range. If the decoded intra-prediction mode is a wide-angle replacement intra-prediction mode, the intra-prediction mode can be converted to a wide-angle intra-prediction mode.

[0292] Alternatively, when encoding the current block in wide-angle intra-predictive mode, you can also encode directly into wide-angle intra-predictive mode.

[0293] The encoding of the intra-prediction mode may be implemented based on the MPM list. The method for setting the MPM list will be described in detail later. In the embodiment described later, assume that 10 wide-angle intra-prediction modes with angles greater than 45 degrees (-1 to -10) and 10 wide-angle intra-prediction modes with angles less than -135 degrees (67 to 76) are defined.

[0294] When encoding adjacent blocks in wide-angle intra-prediction mode, the MPM can be set based on the wide-angle replacement intra-prediction mode that corresponds to the wide-angle intra-prediction mode. For example, when encoding adjacent blocks in wide-angle intra-prediction mode, the variable candIntraPredX (where X is A or B) can be set as the wide-angle replacement intra-prediction mode.

[0295] Alternatively, the method for deriving the MPM can be determined based on the current block shape. For example, if the current block is a square shape with equal width and height, candIntraPredX can be set to the wide-angle replacement intra-prediction mode. On the other hand, if the current block is not a square shape, candIntraPredX can be set to the wide-angle intra-prediction mode.

[0296] Alternatively, the decision of whether to set candIntraPredX as a wide-angle intra-prediction mode can be made based on whether the wide-angle intra-prediction mode of an adjacent block can be applied to the current block. For example, if the current block is a non-square shape where the width is greater than the height, then any wide-angle intra-prediction mode whose index is greater than the index of the intra-prediction mode in the upper right diagonal direction is set as candIntraPredX. For wide-angle intra-prediction modes whose index is less than the index of the intra-prediction mode in the lower left diagonal direction, the wide-angle replacement intra-prediction mode corresponding to the wide-angle intra-prediction mode is set as candIntraPredX. On the other hand, if the current block is a non-square shape where the height is greater than the width, then any wide-angle intra-prediction mode whose index is less than the index of the intra-prediction mode in the lower left diagonal direction is set as candIntraPredX. For wide-angle intra-prediction modes whose index is greater than the index of the intra-prediction mode in the upper right diagonal direction, the wide-angle replacement intra-prediction mode corresponding to the wide-angle intra-prediction mode is set as candIntraPredX.

[0297] In other words, based on whether the shape of the adjacent block encoded in wide-angle intra-prediction mode is the same as or similar to the shape of the current block, it is possible to decide whether to derive the MPM using wide-angle intra-prediction mode or wide-angle replacement intra-prediction mode.

[0298] Alternatively, regardless of the current block shape, the wide-angle intra-prediction mode for adjacent blocks can be set to candIntraPredX.

[0299] In short, candIntraPredX can be set to either a wide-angle intra-prediction mode or a wide-angle replacement intra-prediction mode for adjacent blocks.

[0300] The MPM can be derived based on candIntraPredA and candIntraPredB. In this case, the MPM can be derived in an intra-prediction mode similar to candIntraPredA or candIntraPredB. An intra-prediction mode similar to candIntraPredA or candIntraPredB can be derived based on modulo calculation and offset. In this case, the constants and offsets used in the modulo calculation can be determined differently depending on the shape of the current block.

[0301] Table 6 shows an example of deriving the MPM based on the current block shape. [Table 6]

[0302] Assume that candIntraPredA and candIntraPredB are the same, and that candIntraPredA is an angular intra-prediction mode. If the current block is square-shaped, an intra-prediction mode similar to candIntraPredA can be obtained by modulo calculation based on the value obtained by subtracting 1 from the total number of angular intra-prediction modes other than the wide-angle intra-prediction mode. For example, if the number of angular intra-prediction modes other than the wide-angle intra-prediction mode is 65, the MPM can be derived from the value obtained by modulo calculation based on candIntraPredA and 64. On the other hand, if the current block is not square-shaped, an intra-prediction mode similar to candIntraPredA can be obtained by modulo calculation based on the value obtained by subtracting 1 from the total number of angular intra-prediction modes including the wide-angle intra-prediction mode. For example, if the number of wide-angle intra-prediction modes is 20, the MPM can be derived from the value obtained by modulo calculation based on candIntrapredA and 84.

[0303] Since the constants used in modulo calculations are now set to differ depending on the shape of the block, it is possible to determine whether the wide-angle intra-prediction mode can be set to an angle intra-prediction mode similar to candIntraPredA. For example, in modulo calculations using 64, it may not be possible to set the wide-angle intra-prediction mode to an angle intra-prediction mode similar to candIntraPredA, but in modulo calculations using 84, it is possible to set the wide-angle intra-prediction mode to an angle intra-prediction mode similar to candIntraPredA.

[0304] Alternatively, if candIntraPredA and candIntraPredB are the same, the MPM can be derived by considering the current block shape and whether candIntraPredA is in wide-angle intra-prediction mode.

[0305] Table 7 shows an example of deriving the MPM based on the current block shape. [Table 7(1)]

[0306] [Table 7(2)]

[0307] Assume that candIntraPredA and candIntraPredB are the same.

[0308] If the block is currently square-shaped and candIntraPredA is in wide-angle intra-prediction mode, the MPM can be set as the default mode. For example, MPM[0], MPM[1], and MPM[2] can be set to planar mode, DC mode, and vertical intra-prediction mode, respectively.

[0309] Currently, if the block is square in shape and candIntraPredA is an angle intra-prediction mode other than the wide-angle intra-prediction mode, MPM can be set to an angle intra-prediction mode similar to candIntraPredA. For example, MPM[0] can be candIntraPredA, and MPM[1] and MPM[2] can be set to angle intra-prediction modes similar to candIntraPredA.

[0310] Currently, if the block is not square in shape and candIntraPredA is an angle intra-prediction mode, MPM can be set to an angle intra-prediction mode similar to candIntraPredA. For example, MPM[0] can be candIntraPredA, and MPM[1] and MPM[2] can be set to angle intra-prediction modes similar to candIntrapredA.

[0311] Using modulo calculations and offsets, an angle intra-prediction mode similar to candIntraPredA can be derived. In this case, the constants used in the modulo calculation may differ depending on the current block shape. Furthermore, the offsets used to derive an angle intra-prediction mode similar to candIntraPredA can be set differently based on the current block shape. For example, if the current block is a non-square shape where the width is greater than the height, an angle intra-prediction mode similar to candIntraPredA can be derived by using an offset of 2. On the other hand, if the current block is a non-square shape where the height is greater than the width, an angle intra-prediction mode similar to candIntraPredA can be derived by using offsets 2 and -8.

[0312] Alternatively, the MPM can be derived by considering whether candIntraPredX is a wide-angle intra-prediction mode with the maximum index or a wide-angle intra-prediction mode with the minimum index.

[0313] Table 8 shows an example of deriving the MPM by considering the wide-angle intra-predictive mode index. [Table 8(1)]

[0314] [Table 8(2)]

[0315] Let's assume that candIntraPredA and candIntraPredB are the same. For ease of explanation, we will call a wide-angle intra-prediction mode whose index value is less than the index value of the intra-prediction mode in the lower left diagonal direction a downward wide-angle intra-prediction mode, and a wide-angle intra-prediction mode whose index value is greater than the index value of the intra-prediction mode in the upper right diagonal direction a rightward wide-angle intra-prediction mode.

[0316] If candIntraPredA is a downward wide-angle intra-prediction mode, MPM can be an angle intra-prediction mode similar to candIntraPredA. In this case, if candIntraPredA is a downward wide-angle intra-prediction mode with a minimum value, MPM can be a downward wide-angle intra-prediction mode with a predefined index value. Here, the predefined index may be the index with the maximum value among the indices of the downward wide-angle intra-prediction modes. For example, if candIntraPredA is -10, MPM[0], MPM[1], and MPM[2] can be -10, -1, and -9, respectively.

[0317] If candIntraPredA is a rightward wide-angle intra-prediction mode, MPM can be an angle intra-prediction mode similar to candIntraPredA. In this case, if candIntraPredA is a rightward wide-angle intra-prediction mode with a maximum value, MPM can be a rightward wide-angle intra-prediction mode with a predefined index value. Here, the predefined index may be the index with the minimum value among the indices of the rightward wide-angle intra-prediction modes. For example, if candIntraPredA is 77, MPM[0], MPM[1], and MPM[2] can be 77, 76, and 67, respectively.

[0318] Alternatively, MPM can be set as the default mode if the index obtained by subtracting 1 from the index of candIntraPredA is less than the minimum value among the intra-prediction mode indices, or if the index obtained by adding 1 is greater than the maximum value. Here, the default mode may include at least one of the planar mode, DC mode, vertical intra-prediction mode, horizontal intra-prediction mode, and diagonal intra-prediction mode.

[0319] Alternatively, if the index obtained by subtracting 1 from the index of candIntraPredA is less than the minimum value among the intra-prediction mode indices, or if the index obtained by adding 1 is greater than the maximum value, then MPM can be set to an intra-prediction mode that is the inverse of candIntraPredA, or an intra-prediction mode similar to the intra-prediction mode that is the inverse of candIntraPredA.

[0320] Alternatively, MPM candidates can be derived by considering the shape of the current block and the shapes of adjacent blocks. For example, the method for deriving an MPM when both the current block and adjacent blocks are not square in shape may differ from the method for deriving an MPM when the current block is square but the adjacent block is not square.

[0321] The MPMs in the MPM list can be rearranged (or reordered) by considering at least one of the following: the size of the current block, the shape of the current block, the size of the adjacent block, and the shape of the adjacent block. Here, rearrangement means reassigning the index assigned to each MPM. For example, a smaller index can be assigned to an MPM that has the same intra-prediction mode as an adjacent block that has the same size or shape as the current block.

[0322] Assume that MPM[0] and MPM[1] are the intra-prediction modes for the left adjacent block (candIntraPredA) and the upper adjacent block (candIntraPredB), respectively.

[0323] If the current block and the adjacent block above are non-square in shape, with width greater than height, the MPM can be rearranged so that the intra-prediction mode of the adjacent block above, candIntraPredB, has a smaller index. That is, candIntraPredB can be rearranged to MPM[0] and candIntraPredA can be rearranged to MPM[1].

[0324] Alternatively, if the current block and the adjacent block above it are non-square in shape, with height greater than width, the MPM can be rearranged so that the intra-prediction mode of the adjacent block above it, candIntraPredB, has a smaller index. That is, candIntraPredB can be rearranged to MPM[0] and candIntraPredA can be rearranged to MPM[1].

[0325] Alternatively, if the current block and the adjacent block above it are square in shape, the MPM can be rearranged so that the intra-prediction mode of the adjacent block above it, candIntraPredB, has a smaller index. That is, candIntraPredB can be rearranged to MPM[0] and candIntraPredA can be rearranged to MPM[1].

[0326] If, instead of rearranging the MPM, we initially assigned candIntraPredX to the MPM, we could consider at least one of the following: the size of the current block, the shape of the current block, the size of the adjacent block, and the shape of the adjacent block.

[0327] The MPM can be rearranged based on the size or shape of the current block. For example, if the current block is a non-square shape where the width is greater than the height, the MPM can be rearranged in descending order. Conversely, if the current block is a non-square shape where the height is greater than the width, the MPM can be rearranged in ascending order.

[0328] The residual image can be derived by subtracting the original image from the predicted image. In this case, when the residual image is converted to the frequency domain, removing the high-frequency components does not significantly degrade the subjective image quality of the video. Therefore, converting the values ​​of the high-frequency components to smaller values ​​or setting the values ​​of the high-frequency components to 0 has the effect of improving compression efficiency without causing obvious visual distortion. To reflect the above characteristics, the residual image can be decomposed into two-dimensional frequency components by transforming the current block. This transformation can be performed using transformation techniques such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST).

[0329] DCT decomposes (or transforms) a residual image into two-dimensional frequency components using the cosine transform. DST decomposes (or transforms) a residual image into two-dimensional frequency components using the sine transform. As a result of the transformation of the residual image, the frequency components may be represented as the fundamental image. For example, when performing a DCT transform on a block of size N×N, N 2 N basic pattern components can be obtained. Through transformation, the size of each basic pattern component contained in a block of size N×N can be obtained. Depending on the transformation technique used, the size of the basic pattern component may be called the DCT coefficient or DST coefficient.

[0330] The DCT (Digital Transformation) technique is primarily used to transform images with a high distribution of non-zero low-frequency components. The DST (Digital Transformation) technique is primarily used to transform images with a high distribution of high-frequency components.

[0331] Residual images can also be transformed using transformation techniques other than DCT or DST.

[0332] Hereinafter, the process of converting a residual image into two-dimensional frequency components will be referred to as two-dimensional image conversion. The size of the fundamental pattern components obtained by the conversion may also be called the conversion coefficient. For example, the conversion coefficient may be a DCT coefficient or a DST coefficient. When the primary and secondary conversions described later are applied simultaneously, the conversion coefficient can represent the size of the fundamental pattern components generated by the result of the secondary conversion.

[0333] The transformation technique can be determined on a block-by-block basis. The transformation technique can be determined based on at least one of the following: the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, if coding is performed on the current block in intra-predictive mode and the size of the current block is less than N×N, DST can be performed using the transformation technique. On the other hand, if the above conditions cannot be met, the transformation can be performed using the DCT transformation technique.

[0334] In residual images, it is not necessary to perform a 2D image transformation on some blocks. This practice of not performing a 2D image transformation may be called a transform skip. When a transform skip is applied, quantization can be applied to residual values ​​that have not been transformed.

[0335] After transforming the current block using DCT or DST, it is possible to perform another transformation on the transformed current block. In this case, the transformation based on DCT or DST is defined as the primary transformation, and the process of performing another transformation on the block to which the primary transformation is applied is called the secondary transformation.

[0336] The main transformation can be performed using one of several candidate transformation kernels. For example, the main transformation can be performed using one of DCT2, DCT8, or DCT7.

[0337] Different conversion kernels can be used for the horizontal and vertical directions. Information representing the combination of the horizontal and vertical conversion kernels can also be transmitted via a bitstream signal.

[0338] The execution units for primary and secondary transformations are different. For example, a primary transformation can be performed on an 8x8 block, and a secondary transformation can be performed on a subblock of the transformed 8x8 block that is 4x4 in size. In this case, the transformation coefficient for the remaining area where the secondary transformation is not performed can also be set to 0.

[0339] Alternatively, a primary transformation can be performed on a 4x4 block, and then a secondary transformation can be performed on an 8x8 region containing the transformed 4x4 block.

[0340] A bitstream can transmit information in a signal indicating whether or not to perform a quadratic transformation.

[0341] In the decoder, the inverse of the quadratic transformation (second inverse transformation) can be performed, and the inverse of the primary transformation (first inverse transformation) can be performed on the result. As a result of performing the second and first inverse transformations, the residual signal of the current block can be obtained.

[0342] Quantization is used to reduce the energy of a block, and the quantization process includes a process of dividing the transformation coefficient by a specific constant. The constant may be derived from a quantization parameter, which may be defined as a value from 1 to 63.

[0343] When the encoder performs the transformation and quantization, the decoder can obtain the residual block by inverse quantization and inverse transformation. The decoder can then obtain the reconstructed block of the current block by adding the predicted block and the residual block.

[0344] Once a reconstructed block is obtained from the current block, in-loop filtering can be used to reduce the loss of information that occurred during the quantization and encoding processes. The in-loop filter may include at least one of the following: a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter may be referred to as the first reconstructed block, and the reconstructed block after applying the in-loop filter may be referred to as the second reconstructed block.

[0345] A second reconstructed block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block. In this case, the SAO or ALF can be applied after the deblocking filter.

[0346] A deblocking filter is used to mitigate image degradation (blocking artifact) that occurs at block boundaries when quantization is performed on a block-by-block basis. To apply a deblocking filter, the blocking strength (BS) between the first reconstructed block and adjacent reconstructed blocks can be determined.

[0347] Figure 30 is a flowchart showing the block strength determination process.

[0348] In the example shown in Figure 30, P represents the first reconfiguration block, and Q represents the adjacent reconfiguration block. Here, the adjacent reconfiguration block may be adjacent to the left or above the current block.

[0349] In the example shown in Figure 30, the block intensity can be determined by considering the prediction coding modes of P and Q, whether non-zero conversion coefficients are included, whether interpretation is performed using the same reference image, and whether the difference value of the motion vectors is greater than or equal to a threshold.

[0350] Based on the block strength, it is possible to determine whether a deblocking filter has been applied. For example, if the block strength is 0, filtering may not be performed.

[0351] SAO is used to mitigate ringing artifacts that occur when quantization is performed in the frequency domain. SAO can be performed using an offset determined by adding or subtracting a pattern that takes into account the first reconstructed image. Methods for determining the offset include edge offset (EO) or band offset. EO represents a method of determining the offset of the current sample based on the pattern of surrounding pixels. BO represents a method of applying a common offset to sets of pixels with similar luminance values ​​in a region. Specifically, pixel luminance can be divided into 32 equal intervals, and pixels with similar luminance values ​​can be grouped together. For example, four adjacent bands out of the 32 can be grouped together, and the same offset can be applied to samples belonging to these four bands.

[0352] ALF is a method for generating a second reconstructed image by applying a predefined size / shape filter to a first reconstructed image or a reconstructed image to which a deblocking filter has been applied. Equation 21 below shows an example of ALF application.

number

[0353] One of the predefined filter candidates can be selected for each image, coding tree unit, coding block, prediction block, or transformation block. The size or shape of each filter candidate may differ.

[0354] Figure 31 shows a diagram of predefined filter candidates.

[0355] In the example shown in Figure 31, at least one of the 5x5, 7x7, and 9x9 rhombuses can be selected.

[0356] For the chromaticity component, only rhombuses with a size of 5x5 can be used.

[0357] For high-resolution videos such as panoramic videos, 360-degree videos, or 4K / 8K UHD (ultra-high-definition) videos, real-time or low-latency encoding can be performed by dividing a single image into multiple regions and encoding / decoding each region. To achieve this, the image can be divided into tiles (i.e., basic units encoded / decoded in parallel), and these tiles can be processed in parallel.

[0358] Tiles can be restricted to those with a rectangular shape. When encoding / decoding a tile, data from other tiles is not used. A probability table for a context-adaptive binary arithmetic coding (CABAC) context can be initialized on a tile-by-tile basis, and it can be configured not to apply loop filters at tile boundaries.

[0359] Figure 32 shows an example of dividing an image into multiple tiles.

[0360] Each tile contains at least one coding tree unit, and the boundaries of the tile overlap with the boundaries of the coding tree units.

[0361] In the example shown in Figure 32, the image may be divided into multiple tile sets. Information for dividing the image into multiple tile sets can be transmitted via a bitstream signal.

[0362] Depending on the image subdivision type, tiles may have the same size in all areas except the image boundaries.

[0363] Alternatively, the image can be divided so that adjacent tiles have the same height horizontally, or so that adjacent tiles have the same width vertically.

[0364] When an image is divided using at least one line, either a vertical or horizontal line, that intersects the image, each tile belongs to a different and / or row. In the exemplary embodiments described below, the column to which a tile belongs is called the tile column, and the row to which a tile belongs is called the tile row.

[0365] Information for determining the shape of the image into tiles can be transmitted via a bitstream signal, using an image parameter set or a sequence parameter set. The information can be encoded and transmitted as a signal. This information is used to determine the number of tiles in the image and may also include information indicating the number of tile rows and the number of tile columns. For example, the syntax element num_tile_columns_minus1 represents the value obtained by subtracting 1 from the number of tile columns, and the syntax element num_tile_rows_minus1 represents the value obtained by subtracting 1 from the number of tile rows.

[0366] In the example shown in Figure 32, since the number of tile columns is 4 and the number of tile rows is 3, num_tile_columns_minus1 may be 3, and num_tile_rows_minus1 may be 2.

[0367] When dividing an image into multiple tiles, information indicating the size of each tile can be transmitted via a bitstream as a signal. For example, when dividing an image into multiple tile columns, information indicating the width of each tile column can be transmitted via a bitstream as a signal. Similarly, when dividing an image into multiple tile rows, information indicating the height of each tile row can be transmitted via a bitstream as a signal. For example, for each tile column, a syntax element `column_width_minus1` indicating the width of the tile column can be encoded and transmitted as a signal. Similarly, for each tile row, a syntax element `row_height_minus1` indicating the height of the tile row can be encoded and transmitted as a signal.

[0368] `column_width_minus1` can represent the value obtained by subtracting 1 from the width of the tile column. Similarly, `row_height_minus1` can represent the value obtained by subtracting 1 from the height of the tile row.

[0369] The encoding for column_width_minus1 may be omitted for the last tile column, and the encoding for row_height_minus1 may be omitted for the last tile row. The width of the last tile column and the height of the last row can be derived by considering the size of the image.

[0370] The decoder can determine the tile size based on column_width_minus1 and row_height_minus1.

[0371] Table 9 shows the syntax table for dividing an image into tiles. [Table 9]

[0372] Referring to Table 9, the signal can transmit the syntax element num_tile_columns_minus1 indicating the number of tile columns and the syntax element num_tile_rows_minus1 indicating the number of tile rows.

[0373] Next, the signal can send the syntax element `uniform_spacing_flag`, which indicates whether the image has been divided into tiles of equal size. If `uniform_spacing_flag` is true, tiles in areas other than the image boundaries can be divided into tiles of equal size.

[0374] If uniform_spacing_flag is false, the signal may send the syntax element column_width_minus1 indicating the width of each tile column and the syntax element row_height_minus1 indicating the height of each tile row.

[0375] The syntax element `loop_filter_across_tiles_enabled_flag` indicates whether loop filtering is allowed at tile boundaries.

[0376] A tile column having the minimum width may be called a minimum width tile, and a tile row having the minimum height may be called a minimum height tile. Information indicating the width of the minimum width tile and information indicating the height of the minimum height tile can be transmitted as signals via a bitstream. For example, the syntax element min_column_width_minus1 indicates the value obtained by subtracting 1 from the width of the minimum width tile, and the syntax element min_row_height_minus1 indicates the value obtained by subtracting 1 from the height of the minimum height tile.

[0377] For each tile column, a signal can be transmitted indicating the difference in width from the minimum tile width. For example, the syntax element diff_column_width indicates the width difference between the current tile column and the minimum tile column. The width difference may also be expressed as the difference in the number of encoded tree unit columns. The decoder can derive the width of the current tile by adding the width of the minimum width tile derived based on min_column_width_minus1 and the width difference derived based on diff_column_width.

[0378] Furthermore, for each tile row, information indicating the difference in height from the minimum tile height can be transmitted via a signal. For example, the syntax element diff_row_height indicates the height difference between the current tile row and the minimum tile row. The height difference may also be expressed as the difference in the number of encoded tree unit rows. The decoder can derive the current tile height by adding the height of the minimum height tile derived based on min_row_height_minus1 and the height difference derived based on diff_row_height.

[0379] Table 10 shows a syntax table containing information related to size differences. [Table 10]

[0380] An image can be divided so that horizontally adjacent tiles have different heights, or so that vertically adjacent tiles have different widths. The above image division method may also be called a flexible tile division method, and tiles divided by a flexible tile division method may be called flexible tiles.

[0381] Figure 33 shows an image segmentation mode using flexible tiling technology.

[0382] The search order of the tiles generated by dividing the image may follow a predetermined scan order. An index can be assigned to each tile according to the predetermined scan order.

[0383] The tile scanning order may be any one of raster scanning, diagonal scanning, vertical scanning, or horizontal scanning. Figures 33(a) to 33(d) show examples of assigning an index to each tile based on raster scanning, diagonal scanning, vertical scanning, and horizontal scanning, respectively.

[0384] The next scanning order can be determined based on the current tile's size or position. For example, if the height of the current tile differs from the height of the tile adjacent to its right (e.g., the height of the right-adjacent tile is greater than the height of the current tile), the leftmost tile on the vertical line that is the same as the vertical line of the tile adjacent below the current tile may be determined to be scanned after the current tile.

[0385] The scanning order of tiles can be determined based on an image or sequence.

[0386] Alternatively, the scanning order of tiles can be determined by considering the size of the first tile in the image. For example, if the width of the first tile is greater than its height, the scanning order can be horizontal. If the height of the first tile is greater than its width, the scanning order can be vertical. If the width of the first tile is the same as its height, the scanning order can be raster scanning or diagonal scanning.

[0387] Information indicating the total number of tiles can be transmitted via a bitstream signal. For example, when applying flexible tiling technology, the signal can transmit the syntax element `number_of_tiles_in_picture_minus2`, which is derived by subtracting 2 from the total number of tiles in the image. Based on `number_of_tiles_in_picture_minus2`, the decoder can recognize the number of tiles currently present in the image.

[0388] Table 11 shows a syntax table containing information related to the number of tiles. [Table 11]

[0389] To reduce the number of bits required to encode the size of a tile, information indicating the size of a subtile can be encoded and transmitted as a signal. A subtile is the basic unit that makes up a tile, and each tile may be configured to contain at least one subtile. A subtile may contain one or more encoded tree units.

[0390] For example, the syntax element `subtile_width_minus1` indicates the value obtained by subtracting 1 from the width of the subtile. The syntax element `subtile_height_minus1` indicates the value obtained by subtracting 1 from the height of the subtile.

[0391] Information indicating whether tiles other than the first tile have the same size as the previous tile can be encoded and transmitted as a signal. For example, the syntax element use_previous_tile_size_flag indicates whether the current tile's size is the same as the previous tile's size. If use_previous_tile_size_flag is true, it indicates that the current tile's size is the same as the previous tile's size. If use_previous_tile_size_flag is false, information indicating the current tile's size can be encoded and transmitted as a signal. For the first tile, encoding of use_previous_tile_size_flag may be omitted, and the flag value can be set to false.

[0392] Information indicating the size of a tile may include the syntax element tile_width_minus1[i] indicating the width of the i-th tile and the syntax element tile_height_minus1[i] indicating the height of the i-th tile.

[0393] Information indicating the size of a tile can show the difference between the size of the tile and the size of its child tiles. When using subtile size information, the encoding / decoding efficiency can be improved by reducing the number of bits required to encode the size of each tile. For example, the width of the i-th tile, tileWidth, can be derived based on equation 22 below, and the height of the i-th tile, tileHeight, can be derived based on equation 23 below.

number

number

[0394] Alternatively, the encoding of subtile size information may be omitted, or the size of the i-th tile may be directly encoded as tile size information. Subtile size information can be selectively encoded. Information indicating whether subtile size information has been encoded can be transmitted in the signal via a video parameter set, a sequence parameter set, or an image parameter set.

[0395] Information related to tile size may be encoded to indicate the number of coded tree units and transmitted as a signal. For example, column_width_minus1, min_column_width_minus1, subtile_width_minus1, tile_width_minus1, etc., can indicate the number of coded tree unit sequences contained in a tile. Diff_column_width can indicate the difference between the number of coded tree unit sequences in the minimum width tile and the number of coded tree unit sequences in the current tile.

[0396] Note that row_height_minus1, min_row_height_minus1, subtile_height_minus1, tile_height_minus1, etc., can indicate the number of coding tree unit rows contained in a tile. Note that diff_row_height can indicate the difference between the number of coding tree unit rows contained in the minimum height tile and the number of coding tree unit rows contained in the current tile.

[0397] The decoder can determine the size of a tile based on the number of coding tree unit columns and / or coding tree unit rows derived from the syntax elements, and the size of the coding tree units. For example, the width of the i-th tile may be (tile_width_minus1[i]+1) * (width of the coding tree unit), and the height of the i-th tile may be (tile_height_minus1[i]+1) * (height of the coding tree unit).

[0398] Simultaneously, information indicating the size of the coded tree unit can be transmitted in the signal via a sequence parameter set or an image parameter set.

[0399] Table 11 illustrates how the syntax element `use_previous_tile_size_flag` can be used to indicate whether the current tile's size is the same as the previous tile's size. As another example, information indicating whether the current tile's width is the same as the previous tile's width, or whether the current tile's height is the same as the previous tile's height, can be encoded and transmitted as a signal.

[0400] Table 12 shows a syntax table that includes information indicating whether the current tile width is the same as the width of the previous tile. [Table 12]

[0401] The syntax element `use_previous_tile_width_flag` indicates whether the current tile's width is the same as the previous tile's width. If `use_previous_tile_width_flag` is true, the current tile's width can be set to be equal to the previous tile's width. In this case, encoding the information indicating the current tile's width may be omitted, and the current tile's width can be derived from the previous tile's width.

[0402] If use_previous_tile_width_flag is false, the signal can send information indicating the current tile width. For example, tile_width_minus1[i] could represent the value obtained by subtracting 1 from the width of the i-th tile.

[0403] The syntax element `use_previous_tile_width_flag` can only be encoded and transmitted as a signal if it is determined that the current tile size is different from the size of the previous tile (for example, if the value of `use_previous_tile_size_flag` is 0).

[0404] tile_width_minus1[i] may have a value obtained by subtracting 1 from the number of coding tree unit sequences contained in the i-th tile. The decoder can derive the number of coding tree unit sequences belonging to the i-th tile by adding 1 to tile_width_minus1[i], and calculate the width of the tile by multiplying the derived value by the width of the coding tree units.

[0405] Table 13 shows a syntax table that includes further information indicating whether the current tile height is the same as the previous tile height. [Table 13(1)]

[0406] [Table 13(2)]

[0407] The syntax element `use_previous_tile_height_flag` indicates whether the current tile's height is the same as the previous tile's height. If `use_previous_tile_height_flag` is true, the current tile's height can be set to be equal to the previous tile's height. In this case, the encoding of the information indicating the current tile's height may be omitted, and the current tile's height can be derived from the previous tile's height.

[0408] If use_previous_tile_height_flag is false, the signal can send information indicating the current tile height. For example, tile_height_minus1[i] could represent the value obtained by subtracting 1 from the height of the i-th tile.

[0409] The syntax element `use_previous_tile_height_flag` can only be encoded and sent as a signal if it is determined that the current tile size is different from the size of the previous tile (for example, if the value of `use_previous_tile_size_flag` is 0). The syntax element `use_previous_tile_height_flag` is only sent as a signal if `use_previous_tile_width_flag` is false.

[0410] Table 12 shows an example using use_previous_tile_width_flag, and Table 13 shows an example using use_previous_tile_width_flag and use_previous_tile_height_flag. Although not shown in the above tables, the encoding of use_previous_tile_width_flag may be omitted, and only use_previous_tile_height_flag may be used.

[0411] The choice between use_previous_tile_height_flag and use_previous_tile_size_flag can be determined based on at least one of the following: the tile scanning order, the width and height of the first tile, and the width and height of the previous tile. For example, if the tile scanning order is vertical, use_previous_tile_height_flag can be used, and if the tile scanning order is horizontal, use_previous_tile_width_flag can be used. Alternatively, if the first tile or the previous tile is a non-square shape with a width greater than its height, use_previous_tile_width_flag can be used. If the first tile or the previous tile is a non-square shape with a height greater than its width, use_previous_tile_height_flag can be used.

[0412] When transmitting the number of tiles in an image via a signal, the encoding of information related to the tile size may be omitted for the last tile.

[0413] Table 14 shows an example where the encoding of tile size information is omitted for the last tile. [Table 14]

[0414] If you specify the size of any tile other than the last tile, the remaining area in the image can be designated as the last tile.

[0415] Each coding tree unit can be assigned an identifier (hereinafter referred to as Tile ID) to recognize the tile to which the coding tree unit belongs.

[0416] Figure 34 shows an example of assigning a tile ID to each coding tree unit.

[0417] The same tile ID can be assigned to the same coding tree unit belonging to the same tile. Specifically, the Nth Tile ID can be assigned to the coding tree unit belonging to tile N.

[0418] To determine the tile ID assigned to each coding tree unit, we can determine variables x and y that indicate the position of the coding tree unit in the image. Here, x represents the value obtained by dividing the x-axis coordinate of the position (x0, y0) of the top-left sample of the coding tree unit by the width of the coding tree unit, and y represents the value obtained by dividing the y-axis coordinate of the position (x0, y0) of the top-left sample of the coding tree unit by the height of the coding tree unit. Specifically, x and y can be derived from equations 24 and 25 below.

number

number

[0419] The process described below allows you to assign a tile ID to each coding tree unit.

[0420] i) Initialization of tile ID The tile ID of each encoded tree unit may be initialized to a value obtained by subtracting 1 from the number of tiles in the image. [Table 15]

[0421] ii) Derivation of tile ID [Table 16]

[0422] In the above embodiment, a flag indicating whether to allow the application of loop filtering at tile boundaries was transmitted via an image parameter set. However, setting the system to not use loop filtering at any tile boundaries can lead to problems such as a decrease in subjective image quality and a decrease in coding efficiency.

[0423] Therefore, information indicating whether each tile is allowed to have a loop filter applied can be encoded and transmitted as a signal.

[0424] Figure 35 shows an example of selectively deciding whether or not to apply a loop filter to each tile.

[0425] In the example shown in Figure 35, for each tile, it can be determined whether to allow the application of loop filters (e.g., deblocking filters, SAO and / or ALF) at the horizontal or vertical boundary.

[0426] Table 17 shows an example of encoding information for each tile indicating whether or not to allow the application of a loop filter. [Table 17]

[0427] In the example in Table 17, the syntax element loop_filter_across_tiles_flag[i] indicates whether the loop filter is allowed to be applied to the i-th tile. A value of loop_filter_across_tile_flag[i] of 1 indicates that the loop filter can be used at the horizontal and vertical boundaries of the tile with tile ID i. A value of loop_filter_across_tile_flag[i] of 0 indicates that the loop filter will not be used at the horizontal and vertical boundaries of the tile with tile ID i.

[0428] Information indicating whether loop filtering is permitted in both the horizontal and vertical directions can be encoded.

[0429] Table 18 shows examples of encoding information indicating whether the application of a loop filter is permitted in the horizontal and vertical directions, respectively. [Table 18]

[0430] In the example in Table 18, the syntax element loop_filter_hor_across_tiles_flag[i] indicates whether to allow the application of the loop filter at the position where it intersects with the i-th tile horizontally. The syntax element loop_filter_ver_across_tiles_flag[i] indicates whether to allow the application of the loop filter at the position where it intersects with the i-th tile vertically.

[0431] A value of 1 for loop_filter_hor_across_tile_flag[i] indicates that the loop filter can be used at the horizontal boundary of the tile with tile ID i. A value of 0 for loop_filter_hor_across_tile_flag[i] indicates that the loop filter will not be used at the vertical boundary of the tile with tile ID i.

[0432] A value of 1 for loop_filter_ver_across_tile_flag[i] indicates that the loop filter can be used at the vertical boundary of the tile with tile ID i. A value of 0 for loop_filter_ver_across_tile_flag[i] indicates that the loop filter will not be used at the vertical boundary of the tile with tile ID i.

[0433] Information indicating whether a group of tiles containing multiple tiles allows the application of a loop filter can be encoded and transmitted as a signal. Based on this information, it can be determined whether the multiple tiles in the group of tiles allow the application of a loop filter.

[0434] To determine a tile group, at least one of the following can be transmitted via a bitstream: the number of tiles belonging to the tile group, the size of the tile group, and image segmentation information. Alternatively, a region of a predefined size in the encoder and decoder can be used as the tile group.

[0435] The encoding of information indicating whether loop filtering is permitted may be omitted, and the decision on whether loop filtering is permitted may be based on at least one of the number of coded tree units in the tile, the width of the tile, and the height of the tile. For example, if the width of the tile is less than a reference value, loop filtering may be permitted horizontally, and if the height of the tile is less than a reference value, loop filtering may be permitted vertically.

[0436] When using a loop filter at tile boundaries, reconstructed data outside the tile can be generated based on the data contained within the tile. In this case, the reconstructed video outside the tile can be obtained by padding or interpolating the data contained within the tile. Subsequently, the loop filter can be applied to this reconstructed data outside the tile.

[0437] The embodiment described with an emphasis on the decoding or encoding process is also included in the scope of the present invention if it is used in the decoding or encoding process. The embodiment described in a different order from the one described is also included in the scope of the present invention if it is a modification of the plurality of embodiments described in a predetermined order.

[0438] Although embodiments have been described based on a series of steps or flowcharts, this does not limit the chronological order of the invention. Furthermore, they may be executed simultaneously or in other orders as needed. In the above embodiments, the components constituting the block diagram (e.g., units, modules, etc.) may each be realized as hardware devices or software. Furthermore, multiple components may be combined and executed as a single hardware device or software. The embodiments may be executed in the form of program instructions. The program instructions may be executed by various computer components and recorded on a computer-readable storage medium. The computer-readable storage medium may include program instructions, data files, data structures, etc. individually or in combination thereof. Examples of computer-readable storage media include magnetic media such as hard disks, flexible disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices such as ROMs, RAMs, and flash memories that are specifically arranged to store and execute program instructions. The hardware devices may be configured to operate as one or more software modules and execute the processing according to the present invention, and vice versa. [Industrial applicability]

[0439] The present invention is applicable to electronic devices that perform encoding / decoding of video.

Claims

1. A video decoding method, The current step is to generate a list of merge candidates for the block, The steps include: determining one merge candidate from among several merge candidates included in the merge candidate list; The steps include: deriving the first and second affine seed vectors of the current block based on the first and second affine seed vectors of the determined merge candidate; The steps include: deriving an affine vector for a subblock within the current block by using the first affine seed vector and the second affine seed vector of the current block; The step includes performing motion compensation prediction for the subblock based on the affine vector, The first affine seed vector and the second affine seed vector of the merge candidate are derived based on the movement information of the adjacent block adjacent to the current block. A video decoding method in which, if the adjacent block is contained in a coding tree unit different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate are derived based on the motion vectors of the lower left subblock and the lower right subblock of the adjacent block.

2. The lower left subblock includes a lower left reference sample located in the lower left corner of the adjacent block, and the lower right subblock includes a lower right reference sample located in the lower right corner of the adjacent block. The video decoding method according to claim 1.

3. The first affine seed vector and the second affine seed vector of the merge candidate are derived based on a value obtained by performing a shift operation using a scale factor on the difference between the motion vectors of the lower left subblock and the lower right subblock, and the scale factor is derived based on a value obtained by adding an offset to the horizontal distance between the lower left reference sample and the lower right reference sample. The video decoding method according to claim 2.

4. The first and second affine seed vectors of the merge candidate are derived based on values ​​obtained by performing a shift operation using a scale factor on the difference between the motion vectors of the lower left subblock and the lower right subblock, and the scale factor is derived based on the distance between the adjacent sample adjacent to the right of the lower right reference sample and the lower left reference sample. The video decoding method according to claim 2.

5. The merge candidate list includes a first merge candidate derived based on an upper adjacent block determined to be the first available block among the upper adjacent blocks located above the current block, and a second merge candidate derived based on a left adjacent block determined to be the first available block among the left adjacent blocks located to the left of the current block. The video decoding method according to claim 1.

6. If the adjacent block is included in a coding tree unit that is the same as the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate are derived based on the first affine seed vector and the second affine seed vector of the adjacent block. The video decoding method according to claim 1.

7. The first and second affine seed vectors of the current block are derived using the third affine seed vector for the lower left control point and the fourth affine seed vector for the lower right control point of the adjacent affine vector, where, [Math 1] Here, (x ｎ２ , y ｎ２ ) represents the coordinates of the lower left control point of the affine adjacent block, and (x ｎ３ , y ｎ３ ) represents the coordinates of the lower right control point of the affine adjacent block, (x ０ , y ０ ) currently represents the coordinates of the top-left control point of the block, and (x １ , y １ ) currently represents the coordinates of the upper right control point of the block, (nv ２ｘ , nv ２ｙ ) represents the affinity seed vector of the lower left control point of the third affinity seed vector, and (nv ３ｘ , nv ３ｙ ) represents the affinity seed vector of the lower right control point of the fourth affinity seed vector. (v ０ｘ ,v ０ｙ ) represents the affine seed vector of the upper left control point of the first affine seed vector, and (v １ｘ ,v １ｙ ) represents the affine seed vector of the upper right control point of the aforementioned second affine seed vector. The video decoding method according to claim 1.

8. A video encoding method, The current step is to generate a list of merge candidates for the block, The steps include: determining one merge candidate from among several merge candidates included in the merge candidate list; The steps include: deriving the first and second affine seed vectors of the current block based on the first and second affine seed vectors of the determined merge candidate; The steps include: deriving an affine vector for a subblock within the current block by using the first affine seed vector and the second affine seed vector of the current block; The step includes performing motion compensation prediction for the subblock based on the affine vector, The first affine seed vector and the second affine seed vector of the merge candidate are derived based on the movement information of the adjacent block adjacent to the current block. A video encoding method in which, if the adjacent block is included in a coding tree unit different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate are derived based on the motion vectors of the lower left subblock and the lower right subblock of the adjacent block.

9. The lower left subblock includes a lower left reference sample located in the lower left corner of the adjacent block, and the lower right subblock includes a lower right reference sample located in the lower right corner of the adjacent block. The video encoding method according to claim 8.

10. The first affine seed vector and the second affine seed vector of the merge candidate are derived based on a value obtained by performing a shift operation using a scale factor on the difference between the motion vectors of the lower left subblock and the lower right subblock, and the scale factor is derived based on a value obtained by adding an offset to the horizontal distance between the lower left reference sample and the lower right reference sample. The video encoding method according to claim 9.

11. The first and second affine seed vectors of the merge candidate are derived based on values ​​obtained by performing a shift operation using a scale factor on the difference between the motion vectors of the lower left subblock and the lower right subblock, and the scale factor is derived based on the distance between the adjacent sample adjacent to the right of the lower right reference sample and the lower left reference sample. The video encoding method according to claim 9.

12. The merge candidate list includes a first merge candidate derived based on an upper adjacent block determined to be the first available block among the upper adjacent blocks located above the current block, and a second merge candidate derived based on a left adjacent block determined to be the first available block among the left adjacent blocks located to the left of the current block. The video encoding method according to claim 8.

13. If the adjacent block is included in a coding tree unit that is the same as the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate are derived based on the first affine seed vector and the second affine seed vector of the adjacent block. The video encoding method according to claim 8.

14. The first and second affine seed vectors of the current block are derived using the third affine seed vector for the lower left control point and the fourth affine seed vector for the lower right control point of the adjacent affine vector, where, [Math 2] Here, (x ｎ２ , y ｎ２ ) represents the coordinates of the lower left control point of the affine adjacent block, and (x ｎ３ , y ｎ３ ) represents the coordinates of the lower right control point of the affine adjacent block, (x ０ , y ０ ) currently represents the coordinates of the top-left control point of the block, and (x １ , y １ ) currently represents the coordinates of the upper right control point of the block, (nv ２ｘ ,nv ２ｙ ) represents the affine seed vector of the lower left control point of the third affine seed vector, and (nv ３ｘ ,nv ３ｙ ) represents the affine seed vector of the lower right control point of the fourth affine seed vector, (v ０ｘ ,v ０ｙ ) represents the affine seed vector of the upper left control point of the first affine seed vector, and (v １ｘ ,v １ｙ ) represents the affine seed vector of the upper right control point of the aforementioned second affine seed vector. The video encoding method according to claim 8.

15. A computer-readable storage medium that stores instructions for causing a processor to generate a bitstream when executed by the processor, wherein the instructions cause the processor to perform the following steps: The current step is to generate a list of merge candidates for the block, The steps include: determining one merge candidate from among several merge candidates included in the merge candidate list; The steps include: deriving the first and second affine seed vectors of the current block based on the first and second affine seed vectors of the determined merge candidate; The steps include: deriving an affine vector for a subblock within the current block by using the first affine seed vector and the second affine seed vector of the current block; The step includes performing motion compensation prediction for the subblock based on the affine vector, The first affine seed vector and the second affine seed vector of the merge candidate are derived based on the movement information of the adjacent block adjacent to the current block. A computer-readable storage medium in which, if the adjacent block is contained in a coding tree unit different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate are derived based on the motion vectors of the lower left subblock and lower right subblock of the adjacent block.

Images