Video signal encoding / decoding method and device therefor

Abstract

To provide a video encoding / decoding method and a device for performing an inter prediction on a video signal using an affine model.SOLUTION: A method includes: a step of generating a merge candidate list of a current block; a step of specifying any one of a plurality of merge candidates included in the merge candidate list; a step of deriving a first affine seed vector and a second affine seed vector of the current block based on the first affine seed vector and the second affine seed vector of the specified merge candidate; a step of deriving an affine vector of a sub-block in a current block by using the first affine seed vector and the second affine seed vector of the current block; and a step of predicting a motion compensation for the sub-block based on the affine vector.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a video signal encoding / decoding method and an apparatus therefor. [Background technology]

[0002] As display panels continue to grow larger, video services with higher image quality are becoming increasingly necessary. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, active research is underway to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013. Its compression performance is approximately twice that of H.264 / AVC. With the rapid growth of high-definition video services, the performance limitations of HEVC are gradually becoming apparent. Summary of the Invention [Problem to be solved by the invention]

[0003] An object of the present invention is to provide an inter-prediction method that uses an affine model when encoding / decoding a video signal, and a device used in the inter-prediction method.

[0004] SUMMARY OF THE INVENTION An object of the present invention is to provide a method for deriving affine seed vectors using translational motion vectors of sub-blocks when encoding / decoding a video signal, and an apparatus for implementing said method.

[0005] Another object of the present invention is to provide a method for deriving an affine seed vector by converting the distance between an adjacent block and a current block into a power series of two when encoding / decoding a video signal, and an apparatus for performing the method.

[0006] The technical problems that the present invention aims to achieve are not limited to the technical problems mentioned above, and a person having ordinary skill in the art to which the present invention pertains will clearly understand other technical problems not mentioned in the following description. [Means for solving the problem]

[0007] A video signal decoding / encoding method according to the present invention includes the steps of generating a merge candidate list for a current block, designating one of a plurality of merge candidates included in the merge candidate list, deriving a first affine seed vector and a second affine seed vector for the current block based on a first affine seed vector and a second affine seed vector for the designated merge candidate, deriving an affine vector for a sub-block within the current block by using the first affine seed vector and the second affine seed vector for the current block, and performing motion compensation prediction on the sub-block based on the affine vector. In this case, the sub-block is an area having a size smaller than the current block. The first affine seed vector and the second affine seed vector for the merge candidate may be derived based on motion information of a neighboring block adjacent to the current block.

[0008] In the video signal decoding / encoding method according to the present invention, if the neighboring block is included in a coding tree unit different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merging candidate can be derived based on the motion vectors of the lower left sub-block and the lower right sub-block of the neighboring block.

[0009] In the video signal decoding / encoding method according to the present invention, the lower left sub-block may include a lower left reference sample located at the lower left corner of the adjacent block, and the lower right sub-block may include a lower right reference sample located at the lower right corner of the adjacent block.

[0010] In the video signal decoding / encoding method according to the present invention, the first affine seed vector and the second affine seed vector of the merging candidate can be derived based on a value obtained by using a scale factor to perform a shift operation on the difference value of the motion vector between the bottom-left sub-block and the bottom-right sub-block, and the scale factor can be derived based on a value obtained by adding the horizontal distance and offset between the bottom-left reference sample and the bottom-right reference sample.

[0011] In the video signal decoding / encoding method according to the present invention, the first affine seed vector and the second affine seed vector of the merging candidate can be derived based on a value obtained by performing a shift operation on the difference value of the motion vector between the lower-left sub-block and the lower-right sub-block using a scale factor, and the scale factor can be derived based on the distance between the adjacent sample adjacent to the right of the lower-right reference sample and the lower-left reference sample.

[0012] In the video signal decoding / encoding method according to the present invention, the merge candidate list includes a first merge candidate and a second merge candidate, and the first merge candidate is derived based on an upper neighboring block determined to be a first available block among upper neighboring blocks located above the current block, and the second merge candidate is derived based on a left neighboring block determined to be a first available block among left neighboring blocks located to the left of the current block.

[0013] In the video signal decoding / encoding method according to the present invention, when the neighboring block is included in a coding tree unit that is the same as the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on the first affine seed vector and the second affine seed vector of the neighboring block.

[0014] The above briefly summarized features of the present invention are merely exemplary embodiments of the detailed description of the invention that follows and are not intended to limit the scope of the invention. [Effects of the Invention]

[0015] According to the present invention, the inter prediction method using the affine model has the effect of improving prediction efficiency.

[0016] According to the present invention, the affine seed vector is derived using the translational motion vector of the sub-block, thereby improving the coding efficiency.

[0017] According to the present invention, the affine seed vector is derived by converting the distance between the neighboring block and the current block into a power series of two, thereby improving coding efficiency.

[0018] The effects obtainable by the present invention are not limited to the above effects, and a person having ordinary skill in the art to which the present invention pertains can clearly understand other effects not mentioned in the following description. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a block diagram illustrating a video encoder according to an embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram illustrating a video decoder according to an embodiment of the present invention. [Figure 3] FIG. 2 illustrates a basic coding tree unit according to an embodiment of the present invention. [Figure 4] FIG. 10 is a diagram illustrating multiple division types of a coding block. [Figure 5] FIG. 10 is a diagram illustrating an example of division of a coding tree unit. [Figure 6] 1 is a flowchart illustrating an inter-prediction method according to an embodiment of the present invention. [Figure 7] FIG. 1 illustrates non-linear motion of an object. [Figure 8] 1 is a flowchart illustrating an affine motion-based inter-prediction method according to an embodiment of the present invention. [Figure 9] 10A and 10B are diagrams illustrating examples of affine seed vectors for each affine motion model. [Figure 10] FIG. 10 is a diagram showing an example of affine vectors of sub-blocks in a four-parameter motion model. [Figure 11] FIG. 10 illustrates neighboring blocks that can be used to derive merge candidates. [Figure 12] FIG. 10 illustrates deriving an affine seed vector for a current block based on affine seed vectors of affine neighboring blocks. [Figure 13] FIG. 10 is a diagram showing an example in which the motion vector of a sub-block is used as the affine seed vector of an affine adjacent block. [Figure 14] FIG. 10 shows the location of a reference sample. [Figure 15] FIG. 10 shows the location of a reference sample. [Figure 16] FIG. 10 shows the location of a reference sample. [Figure 17]FIG. 10 illustrates an example application of the modified affine merge vector derivation method. [Figure 18] FIG. 10 illustrates an example of deriving an affine seed vector for a current block based on non-affine neighboring blocks. [Figure 19] FIG. 10 is a diagram illustrating an example of replacing an adjacent block with another adjacent block. [Figure 20] 1 is a flowchart illustrating a motion compensation prediction method using multiple merging candidates. [Figure 21] 1 is a flowchart illustrating a motion compensation prediction method using multiple merging candidates. [Figure 22] 1 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention. [Figure 23] FIG. 10 is a diagram showing reference samples included in each reference sample line. [Figure 24] FIG. 10 is a diagram illustrating intra-prediction modes. [Figure 25] FIG. 10 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Figure 26] FIG. 10 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Figure 27] FIG. 10 illustrates the angle formed between an angular intra prediction mode and a line parallel to the x-axis. [Figure 28] FIG. 10 is a diagram illustrating an example of obtaining predicted samples when the current block is non-square. [Figure 29] FIG. 10 is a diagram illustrating a wide-angle intra prediction mode. [Figure 30] 10 is a flowchart illustrating a process for determining block strength. [Figure 31] FIG. 1 illustrates predefined filter candidates. [Figure 32] FIG. 10 is a diagram illustrating an example of dividing an image into multiple tiles. [Figure 33] FIG. 1 illustrates an image division mode based on flexible tiling technology. [Figure 34] FIG. 10 is a diagram illustrating an example of assigning a tile ID to each coding tree unit. [Figure 35] FIG. 10 illustrates an example of selectively determining whether to apply a loop filter to each tile. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0021] Video encoding and decoding is performed on a block-by-block basis, for example, encoding / decoding operations such as transform, quantization, prediction, loop filtering, or reconstruction may be performed on a coding block, a transform block, or a prediction block.

[0022] Hereinafter, the block to be coded / decoded is referred to as a “current block.” For example, according to the current coding / decoding process step, the current block can represent a coding block, a transformation block, or a prediction block.

[0023] It should be noted that the term "unit" used in this specification may be understood to refer to a basic unit for performing a specific encoding / decoding process, and "block" may be understood to refer to a sample array of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described below, a coding block and a coding unit may be understood to have the same meaning.

[0024] FIG. 1 is a block diagram illustrating a video encoder according to an embodiment of the present invention.

[0025] Referring to FIG. 1, the video encoding device 100 may include an image division unit 110, prediction units 120 and 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0026] 1 are shown individually to represent different characteristic functions in a video encoding device, but do not represent that each component is composed of separate hardware or a single software assembly. That is, for ease of explanation, each component is shown as a representative component, and each component may be included, and at least two components may be combined into one component, or one component may be divided into multiple components to perform a function. As long as they do not deviate from the essence of the present invention, both embodiments in which each component is integrated and embodiments in which each component is separated are within the scope of the present invention.

[0027] It should be noted that some components are not essential components for performing the essential functions of the present invention, but are merely optional components for improving performance. The present invention may be implemented by including only the components necessary to realize the essence of the present invention (excluding components for improving performance), and a structure including only the necessary components (excluding components for improving performance) also falls within the scope of the present invention.

[0028] The image division unit 110 can divide an input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image division unit 110 divides one image into a plurality of combinations of coding units, prediction units, and transform units. The image can be coded by selecting a combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function).

[0029] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used to divide the coding units into other coding units, with one video or largest coding unit as the root. The coding unit may have a number of child nodes equal to the number of divided coding units. Coding units that cannot be divided due to some restrictions become leaf nodes. That is, assuming that one coding unit can only realize square division, one coding unit can be divided into a maximum of four other coding units.

[0030] Hereinafter, in the embodiments of the present invention, the encoding unit may refer to a unit that performs encoding, or may refer to a unit that performs decoding.

[0031] The prediction units in one coding unit can be divided into at least one square or rectangle of the same size, and one prediction unit in one coding unit can also be divided into one that has a different shape and / or size than another prediction unit.

[0032] Performing intra prediction based on a coding unit If the prediction unit is not the smallest coding unit, intra prediction can be performed without the need to divide it into multiple NxN prediction units.

[0033] The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It may be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference image, etc.) may be determined based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode may be determined by the prediction unit, and the prediction may be performed by a transform unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. Prediction mode information, motion vector information, etc. for prediction may be coded together with the residual value by the entropy coding unit 165 and transmitted to the decoder. When a specific coding mode is used, the original block may be directly coded and transmitted to the decoder without generating a prediction block by the prediction units 120 and 125.

[0034] The inter prediction unit 120 may predict a prediction unit based on information of at least one image immediately before or after the current image. In some cases, the prediction unit may also be predicted based on information of a coded portion of the current image. The inter prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0035] The reference image interpolation unit receives reference image information from memory 155 and can generate integer or fractional pixel information from the reference image. For luminance pixels, an 8-tap DCT-based interpolation filter with different filter coefficients can be used to generate fractional pixel information in units of 1 / 4 pixel. For chrominance signals, a 4-tap DCT-based interpolation filter with different filter coefficients can be used to generate fractional pixel information in units of 1 / 8 pixel.

[0036] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. A number of methods can be used to calculate a motion vector, such as a full search-based block matching algorithm (FBMA), a three-step search (TSS), and a new three-step search algorithm (NTS). According to the interpolated pixels, the motion vector may have a motion vector value in units of half pixels or quarter pixels. The motion prediction unit can predict the current prediction unit using different motion prediction methods. A number of motion prediction methods can be used, such as a skip method, a merge method, an advanced motion vector prediction (AMVP), and an intra block copy method.

[0037] The intra prediction unit 125 can generate a prediction unit based on neighboring reference pixel information of the current block (the reference pixel information is pixel information in the current image). When neighboring blocks of the current prediction unit are blocks on which inter prediction has been performed and the reference pixels are pixels on which inter prediction has been performed, the reference pixels included in the block on which inter prediction has been performed can be used as reference pixel information of neighboring blocks on which intra prediction has been performed. In other words, when reference pixels are unavailable, at least one reference pixel from available reference pixels can be used instead of the unavailable reference pixel information.

[0038] In intra prediction, prediction modes may include an angular prediction mode that uses reference pixel information based on the prediction direction and a non-angular mode that does not use direction information when performing prediction. The mode for predicting luma information may be different from the mode for predicting chroma information. To predict chroma information, intra prediction mode information for predicting luma information or predicted luma signal information may be used.

[0039] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixel located to the left, the pixel located to the upper left, and the pixel located above.However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed based on the reference pixel of the transform unit.Note that intra prediction using NxN division can only be applied to the smallest coding unit.

[0040] After applying an adaptive intra smoothing (AIS) filter to reference pixels based on the prediction mode, a predicted block can be generated using an intra prediction method. The type of adaptive intra smoothing filter applied to the reference pixels can vary. To perform the intra prediction method, the intra prediction mode of a current prediction unit can be predicted based on the intra prediction mode of prediction units located around the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted from surrounding prediction units, if the intra prediction mode of the current prediction unit is the same as that of the surrounding prediction units, information indicating that the prediction mode of the current prediction unit is the same as that of the surrounding prediction units can be transmitted using predetermined flag information. If the prediction mode of the current prediction unit is different from that of the surrounding prediction units, entropy coding can be performed to encode the prediction mode information of the current block.

[0041] In addition, a residual block including residual information can be generated. The residual information is a difference value between a prediction unit that performs prediction based on a prediction unit generated by the prediction units 120 and 125 and the original block of the prediction unit. The generated residual block can be input to the transform unit 130.

[0042] The transform unit 130 may transform the residual block using a transform method such as a discrete cosine transform (DCT), a discrete sine transform (DST), a transform skip, etc. The residual block includes residual information between the original block and the prediction unit generated by the prediction units 120 and 125. Whether to apply the DCT, the DST, or the KLT to transform the residual block may be determined based on intra-prediction mode information of the prediction unit used to generate the residual block.

[0043] The quantization unit 135 may quantize the values ​​transformed into the frequency domain by the transformation unit 130. The quantization coefficient may vary depending on the importance of the block or image. The values ​​calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0044] The rearrangement unit 160 may perform rearrangement of coefficient values ​​for the quantized residual values.

[0045] The rearrangement unit 160 may convert two-dimensional block shape coefficients into one-dimensional vector form using a coefficient scanning method. For example, the rearrangement unit 160 may scan DC coefficients to high-frequency region coefficients using a zig-zag scan method and convert them into one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, vertical scanning, which scans two-dimensional block shape coefficients along the column direction, and horizontal scanning, which scans two-dimensional block shape coefficients along the row direction, may be used instead of zig-zag scanning. That is, whether to use zig-zag scanning, vertical scanning, or horizontal scanning may be determined based on the size of the transform unit and the intra prediction mode.

[0046] The entropy coding unit 165 can perform entropy coding based on the values ​​calculated by the rearrangement unit 160. For example, the entropy coding can use a plurality of coding methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0047] The entropy coding unit 165 can encode multiple information such as residual value coefficient information and block type information of the coding unit from the rearrangement unit 160 and the prediction units 120 and 125, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0048] The entropy coding unit 165 can perform entropy coding on the coefficient values ​​of the coding unit input from the rearrangement unit 160 .

[0049] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the multiple values ​​quantized by the quantization unit 135, and perform inverse transform on the values ​​transformed by the transform unit 130. A reconstructed block can be generated by merging the residual values ​​generated by the inverse quantization unit 140 and the inverse transform unit 145 with the prediction units predicted by the motion prediction unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125.

[0050] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0051] A deblocking filter can remove block artifacts generated in a reconstructed image due to boundaries between blocks. To determine whether to perform deblocking, it can be determined whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied based on the required deblocking filtering strength. In addition, when performing vertical filtering or horizontal filtering in the process of using a deblocking filter, horizontal filtering and vertical filtering can be performed synchronously.

[0052] The offset correction unit can correct the offset between the deblocked image and the original image on a pixel-by-pixel basis. Offset correction can be performed on a specified image in the following manner: After dividing the pixels included in the image into a predetermined number of regions, regions that require offset correction are determined, and offset correction is applied to the corresponding regions or offset correction is applied taking into account edge information of each pixel.

[0053] Adaptive loop filtering (ALF) can be performed based on the comparison value between the filtered reconstructed image and the original image. After dividing the pixels in the image into predetermined groups, a filter to be used for the corresponding group is determined, and filtering can be performed differentially for each group. Information regarding whether adaptive loop filtering is to be applied can be transmitted to each coding unit (CU) via the luminance signal. The shape and filter coefficients of the adaptive loop filter to be applied may differ for each block. Note that the same type (constant type) of ALF can be applied regardless of the characteristics of the block to which it is applied.

[0054] The memory 155 can store the reconstructed block or blocks calculated by the filter unit 150 and can provide the stored reconstructed block or images to the prediction units 120, 125 when performing inter prediction.

[0055] FIG. 2 is a block diagram illustrating a video decoder according to an embodiment of the present invention.

[0056] Referring to FIG. 2, the video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0057] When a video bitstream is input from a video encoder, the input bitstream can be decoded in steps that are the reverse of those of the video encoder.

[0058] The entropy decoding unit 210 may perform entropy decoding in a step that is the reverse of the step of entropy encoding performed by the entropy encoding unit of the video encoder. For example, to correspond to the method performed by the video encoder, multiple methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied.

[0059] The entropy decoding unit 210 may perform decoding on information related to intra-prediction and inter-prediction performed by the encoder.

[0060] The rearrangement unit 215 may perform rearrangement by rearranging the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. The rearrangement may be performed by reconstructing a plurality of coefficients represented in a one-dimensional vector form into coefficients in a two-dimensional block shape. The rearrangement unit 215 may perform rearrangement in the following manner: It receives information related to coefficient scanning performed by the encoding unit, and performs reverse scanning according to the scanning order performed by the corresponding encoding unit.

[0061] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values ​​of the rearranged block.

[0062] For the quantization result performed by the video encoder, the inverse transform unit 225 can perform inverse transforms of the DCT, DST, and KLT performed by the transform unit. That is, it performs inverse DCT, inverse DST, and inverse KLT. The inverse transform may be performed using a transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder can selectively perform a transform method (e.g., DCT, DST, or KLT) based on multiple pieces of information such as a prediction method, a size of a current block, and a prediction direction.

[0063] The prediction units 230, 235 can generate prediction blocks based on information related to the generation of the prediction blocks provided by the entropy decoding unit 210 and previously decoded block or image information provided by the memory 245.

[0064] As described above, when intra prediction is performed in the same manner as the operation manner in a video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixel located to the left, the pixel located to the upper left, and the pixel located above the prediction unit. When intra prediction is performed, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed based on reference pixels of the transform unit. Note that intra prediction using NxN division can be applied only to the smallest coding unit.

[0065] The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives multiple pieces of information, such as prediction unit information input from the entropy decoding unit 210, prediction mode information for the intra prediction method, and motion prediction-related information for the inter prediction method, and classifies the prediction unit based on the current encoding unit and determines whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may perform inter prediction on the current prediction unit based on information included in at least one image immediately before or immediately after the current image to which the current prediction unit belongs, using information necessary for performing inter prediction on the current prediction unit provided by the video encoder. Alternatively, the inter prediction may be performed based on information on a reconstructed region of the current image to which the current prediction unit belongs.

[0066] To perform inter prediction, it is possible to determine, based on the coding unit, whether the motion prediction method of the prediction unit included in the corresponding coding unit is skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), or intra block duplication mode.

[0067] The intra prediction unit 235 may generate a prediction block based on pixel information in the current image. If the prediction unit is a prediction unit for which intra prediction has been performed, the intra prediction may be performed based on intra prediction mode information of the prediction unit provided from the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The adaptive intra smoothing filter is a part that performs filtering on reference pixels of the current block and may determine whether to apply a filter based on the prediction mode of the current prediction unit. Adaptive intra smoothing filtering may be performed on reference pixels of the current block using the prediction mode of the prediction unit and adaptive intra smoothing filter information provided from the video encoder. If the prediction mode of the current block is a mode that does not perform adaptive intra smoothing filtering, the adaptive intra smoothing filter may not be applied.

[0068] If the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values ​​to be interpolated for reference pixels, the reference pixel interpolation unit can generate reference pixels in pixel units of integer or decimal values ​​by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, interpolation of the reference pixels is not necessary. If the prediction mode of the current block is a DC mode, the DC filter can generate a prediction block by filtering.

[0069] The reconstructed block or image may be provided to a filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0070] The deblocking filter of the video decoder may receive, from the video encoder, information regarding whether to apply a deblocking filter to the corresponding block or image, and information regarding whether to use a strong filter or a weak filter when applying the deblocking filter. The deblocking filter of the video decoder may receive the information regarding the deblocking filter provided by the video encoder, and the video decoder may perform deblocking filtering on the corresponding block.

[0071] The offset correction unit can perform offset correction on the reconstructed image based on the offset information and the type of offset correction applied to the image when encoding.

[0072] ALF can be applied to a coding unit based on information provided by the encoder regarding whether to apply ALF, ALF coefficient information, etc. Such ALF information may be provided by being included in a particular parameter set.

[0073] The memory 245 stores the reconstructed image or block, makes it available as a reference image or block, and is capable of providing the reconstructed image to an output.

[0074] FIG. 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

[0075] A coding block of the largest size can be defined as a coding tree block. An image may be divided into multiple coding tree units (CTUs). A coding tree unit is a coding unit of the largest size and may also be called a largest coding unit (LCU). Figure 3 shows an example of dividing an image into multiple coding tree units.

[0076] The size of the coding tree unit may be defined at the picture level or the sequence level, so that the picture parameter set or the sequence parameter set can be used to signal information indicating the size of the coding tree unit.

[0077] For example, the size of the coding tree unit for all images in a sequence can be 128 x 128. Alternatively, the size of the coding tree unit can be determined to be either 128 x 128 or 256 x 256 at the image level. For example, the size of the coding tree unit for a first image can be 128 x 128, and the size of the coding tree unit for a second image can be 256 x 256.

[0078] Coding blocks can be generated by dividing the coding tree units. The coding blocks represent basic units for performing encoding / decoding processes. For example, prediction or transformation can be performed according to different coding blocks, or a predictive coding mode can be determined according to different coding blocks. Here, the predictive coding mode represents a method for generating a predicted image. For example, the predictive coding mode may include intra prediction (intra prediction), inter prediction (inter prediction), current picture referencing (CPR), intra block copy (IBC), or combined prediction. For a coding block, a predictive block related to the coding block can be generated using at least one predictive coding mode from intra prediction, inter prediction, current picture referencing, or combined prediction.

[0079] Information indicating the predictive coding mode of the current block can be transmitted as a signal via the bitstream. For example, the information can be a one-bit flag indicating whether the predictive coding mode is intra mode or inter mode. Only when it is determined that the predictive coding mode of the current block is inter mode, can current picture reference or combined prediction be used.

[0080] The current image reference is used to obtain a prediction block of the current block from an encoded / decoded region in the current image using the current image as a reference image. Here, the current image refers to an image including the current block. Information indicating whether the current image reference is applied to the current block may be transmitted as a signal via the bitstream. For example, the information may be a 1-bit flag. If the flag is true, the predictive coding mode of the current block may be determined as the current image reference. If the flag is false, the prediction mode of the current block may be determined as inter prediction.

[0081] Alternatively, the predictive coding mode of the current block may be determined based on a reference image index. For example, if the reference image index indicates the current image, the predictive coding mode of the current block may be determined as current image reference. If the reference image index indicates an image other than the current image, the predictive coding mode of the current block may be determined as inter prediction. In other words, current image reference is a prediction method that uses information on an encoded / decoded region in the current image, and inter prediction is a prediction method that uses information on another encoded / decoded image.

[0082] Combined prediction is a coding mode formed by combining two or more of intra prediction, inter prediction, and current image reference. For example, when combined prediction is applied, a first predicted block may be generated based on one of intra prediction, inter prediction, or current image reference, and a second predicted block may be generated based on the other. When generating the first predicted block and the second predicted block, a final predicted block may be generated by averaging and weighted addition of the first predicted block and the second predicted block. Information indicating whether combined prediction is applied may be transmitted as a signal via a bitstream. The information may be a 1-bit flag.

[0083] FIG. 4 is a diagram showing a plurality of division types of a coding block.

[0084] A coding block can be divided into a plurality of coding blocks based on quadtree division, binary tree division, or ternary tree division. Also, a divided coding block can be further divided into a plurality of coding blocks based on quadtree division, binary tree division, or ternary tree division.

[0085] Quadtree partitioning is a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block can be divided into four square partitions (see "SPLIT_QT" in Figure 4(a)).

[0086] Binary tree partitioning is a partitioning technique that divides a current block into two blocks. The process of dividing the current block into two blocks along the vertical direction (i.e., using a vertical line that crosses the current block) can be called vertical binary tree partitioning, and the process of dividing the current block into two blocks along the horizontal direction (i.e., using a horizontal line that crosses the current block) can be called horizontal binary tree partitioning. As a result of binary tree partitioning, the current block can be divided into two non-square partitions. "SPLIT_BT_VER" in Figure 4(b) represents the vertical binary tree partitioning result, and "SPLIT_BT_HOR" in Figure 4(c) represents the horizontal binary tree partitioning result.

[0087] Ternary tree partitioning is a partitioning technique that divides a current block into three blocks. The process of dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) is called vertical ternary tree partitioning, and the process of dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines crossing the current block) is called horizontal ternary tree partitioning. As a result of ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block may be twice the width / height of the other partitions. 'SPLIT_TT_VER' in Figure 4(d) represents the result of vertical ternary tree partitioning, and 'SPLIT_TT_HOR' in Figure 4(e) represents the result of horizontal ternary tree partitioning.

[0088] The number of times a coding tree unit is divided can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or image level. Therefore, the maximum partitioning depth of a coding tree unit may vary depending on the sequence or image.

[0089] Alternatively, a maximum splitting depth can be determined independently for each of multiple splitting techniques. For example, the maximum splitting depth that allows for quadtree splitting may be different from the maximum splitting depth that allows for binary tree and / or ternary tree splitting.

[0090] The encoder can transmit information indicative of at least one of the partition type or partition depth of the current block in a signal via the bitstream, and the decoder can determine the partition type and partition depth of the coding tree unit based on the information parsed from the bitstream.

[0091] FIG. 5 is a diagram illustrating an example of division of a coding tree unit.

[0092] The process of dividing a coding block using a division technique such as quad-tree division, binary tree division, and / or ternary tree division can be called multi-tree partitioning.

[0093] A coding block generated by applying multi-tree division to a coding block can be called a downstream coding block. If the division depth of a coding block is k, the division depth of multiple downstream coding blocks is k+1.

[0094] On the other hand, for a coding block with a division depth of k+1, a coding block with a division depth of k can be called an upstream coding block.

[0095] The division type of the currently coded block may be determined based on at least one of the division type of the upstream coded block or the division type of the adjacent coded block. Here, the adjacent coded block is adjacent to the currently coded block, and may include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the currently coded block. Here, the division type may include at least one of whether to perform quadtree division, whether to perform binary tree division, the binary tree division direction, whether to perform ternary tree division, and the ternary tree division direction.

[0096] To determine the split type of a coding block, a signal can be sent via the bitstream indicating whether the coding block is split or not. The signal is a one-bit flag "split_cu_flag", which indicates that the coding block is split using a multi-tree splitting technique when the flag is true.

[0097] If "split_cu_flag" is true, a signal can be sent via the bitstream indicating whether the coding block has been quadtree split. The information is a 1-bit flag "split_qt_flag", and if the flag is true, the coding block may be split into four blocks.

[0098] For example, in the example shown in Fig. 5, when a coding tree unit is quadtree divided, four coding blocks with a division depth of 1 are generated. Note that the illustration shows that quadtree division is again applied to the first and fourth coding blocks of the four coding blocks generated as a result of the quadtree division. Finally, four coding blocks with a division depth of 2 can be generated.

[0099] Note that by applying quadtree division to an encoding block with a division depth of 2 again, an encoding block with a division depth of 3 can be generated.

[0100] If quadtree partitioning has not been applied to a coding block, it may be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block by considering at least one of the size of the coding block, whether the coding block is located on an image boundary, the maximum partition depth, or the partition type of an adjacent block. If it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating a partitioning direction may be transmitted via a bitstream as a signal. The information may be a one-bit flag "mtt_split_cu_vertical_flag." Based on the flag, it may be determined whether the partitioning direction is vertical or horizontal. Note that information indicating whether binary tree partitioning or ternary tree partitioning is to be applied to the coding block may be transmitted via a bitstream as a signal. The information may be a one-bit flag "mtt_split_cu_binary_flag." Based on the flag, it may be determined whether to apply binary tree partitioning or ternary tree partitioning to the coding block.

[0101] For example, the example shown in Figure 5 illustrates the application of vertical binary tree partitioning to a coding block with a partitioning depth of 1. Vertical ternary tree partitioning is applied to the left coding block of the coding blocks generated as a result of the partitioning, and vertical binary tree partitioning is applied to the right coding block.

[0102] Inter-prediction is a predictive coding mode that predicts a current block using information of a previous image. For example, a block in the previous image that is located at the same position as the current block (hereinafter referred to as a collocated block) can be used as a prediction block for the current block. Hereinafter, a prediction block generated based on a block located at the same position as the current block can be referred to as a collocated prediction block.

[0103] On the other hand, if an object in the previous image moves to another position in the current image, the current block can be effectively predicted based on the symmetrical motion. For example, if the direction and size of the object's movement can be determined by comparing the previous image with the current image, a predicted block (or predicted image) of the current block can be generated taking into account the object's motion information. Hereinafter, the predicted block generated using the motion information may be referred to as a motion predicted block.

[0104] A residual block can be generated by subtracting a predicted block from a current block. In this case, if there is target motion, using a collocated predicted block instead of a motion predicted block can reduce the energy of the residual block and improve the compression performance of high residual blocks.

[0105] As described above, the process of generating a prediction block using motion information may be called motion compensated prediction. In most inter predictions, a prediction block may be generated based on motion compensated prediction.

[0106] The motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weight index. The motion vector represents the movement direction and size of an object. The reference image index specifies a reference image of the current block among multiple reference images included in a reference image list. The prediction direction indicates one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of L0 direction motion information or L1 direction motion information can be used based on the prediction direction of the current block. The bidirectional weight index specifies a weight to be applied to the L0 prediction block and a weight to be applied to the L1 prediction block.

[0107] FIG. 6 is a flowchart illustrating an inter prediction method according to an embodiment of the present invention.

[0108] Referring to FIG. 6, the inter prediction method includes a step of determining an inter prediction mode of a current block (S601), a step of obtaining motion information of the current block based on the determined inter prediction mode (S602), and a step of performing motion compensation prediction on the current block based on the obtained motion information (S603).

[0109] Here, the inter prediction mode represents a number of techniques for determining motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information may include a merge mode and an advanced motion vector prediction mode. The inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. According to the inter prediction mode, the motion information of the current block may be determined based on information analyzed from a neighboring block or a bitstream adjacent to the current block.

[0110] The inter prediction method using affine motion information will be described in detail below.

[0111] FIG. 7 is a diagram illustrating the nonlinear motion of an object.

[0112] Symmetrical motion in a video may be nonlinear motion. For example, as shown in the example of FIG. 7, symmetrical nonlinear motion such as camera zoom-in, zoom-out, rotation, or affine transformation may occur. When nonlinear motion of an object occurs, the object motion cannot be effectively represented by a translational motion vector. Therefore, in the part where nonlinear motion of an object occurs, affine motion is used instead of translational motion to improve coding efficiency.

[0113] FIG. 8 is a flowchart illustrating an affine motion-based inter prediction method according to an embodiment of the present invention.

[0114] Whether to apply an affine motion-based inter prediction technique to the current block may be determined based on information analyzed from the bitstream. Specifically, whether to apply an affine motion-based inter prediction technique to the current block may be determined based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0115] When applying an affine motion-based inter prediction technique to a current block, an affine motion model of the current block may be determined (S801). The affine motion model may be determined to be at least one of a six-parameter affine motion model or a four-parameter affine motion model. The six-parameter affine motion model represents affine motion using six parameters, and the four-parameter affine motion model represents affine motion using four parameters.

[0116] Equation 1 is a case where affine motion is expressed by six parameters: Affine motion represents the translational motion of a given region determined by an affine seed vector.

number

[0117] When affine motion is represented using six parameters, complex motion can be represented, but the number of bits required for encoding each parameter increases, reducing coding efficiency. Therefore, affine motion can also be represented using four parameters. Equation 2 shows the case where affine motion is represented using four parameters.

number

[0118] Information for determining an affine motion model of a current block may be coded and transmitted as a signal via a bitstream. For example, the information may be a 1-bit flag "affine_type_flag." A value of 0 of the flag indicates that a 4-parameter affine motion model is to be applied. A value of 1 of the flag indicates that a 6-parameter affine motion model is to be applied. The flag may be coded in units of slices, tiles, or blocks (e.g., coding blocks or coding tree units). When a flag is transmitted using a signal at the slice level, the affine motion model determined at the slice level may be applied to all blocks belonging to the slice.

[0119] Alternatively, the affine motion model of the current block may be determined based on the affine inter prediction mode of the current block. For example, when the affine merge mode is applied, the affine motion model of the current block may be determined as a four-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information for determining the affine motion model of the current block may be coded and transmitted as a signal via a bitstream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block may be determined based on a 1-bit flag "affine_type_flag."

[0120] Next, an affine seed vector for the current block can be derived (S802). If a four-parameter affine motion model is selected, motion vectors can be derived at two control points of the current block. If a six-parameter affine motion model is selected, motion vectors can be derived at three control points of the current block. The motion vectors at the control points can be called affine seed vectors. The control points may include at least one of the top left corner, the top right corner, or the bottom left corner of the current block.

[0121] FIG. 9 is a diagram showing examples of affine seed vectors for each affine motion model.

[0122] In a four-parameter affine motion model, an affine seed vector relating to two of the upper left corner, the upper right corner, or the lower left corner can be derived. For example, as shown in (a) of FIG. 9, when a four-parameter affine motion model is selected, an affine vector can be derived using an affine seed vector sv0 relating to the upper left corner of the current block (e.g., the upper left sample (x0, y0)) and an affine seed vector sv1 relating to the upper right corner of the current block (e.g., the upper right sample (x1, y1)). Also, instead of the affine seed vector relating to the upper left corner, an affine seed vector relating to the lower left corner can be used. Alternatively, instead of the affine seed vector relating to the upper right corner, an affine seed vector relating to the lower left corner can be used.

[0123] In a six-parameter affine motion model, affine seed vectors relating to the upper left corner, the upper right corner, and the lower left corner can be derived. For example, as shown in the example of FIG. 9(b), when a six-parameter affine motion model is selected, an affine vector can be derived using an affine seed vector sv0 relating to the upper left corner of the current block (e.g., the upper left sample (x0, y0)), an affine seed vector sv1 relating to the upper right corner of the current block (e.g., the upper right sample (x1, y1)), and an affine seed vector sv2 relating to the upper left corner of the current block (e.g., the upper left sample (x2, y2)).

[0124] In the embodiments described below, in a four-parameter affine motion model, the affine seed vectors of the top-left control point and the top-right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments described below that use the first affine seed vector and the second affine seed vector, at least one of the first affine seed vector and the second affine seed vector can be replaced with the affine seed vector of the bottom-left control point (third affine seed vector) or the affine seed vector of the bottom-right control point (fourth affine seed vector).

[0125] In the six-parameter affine motion model, the affine seed vectors of the top-left control point, the top-right control point, and the bottom-left control point are referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector, respectively. In an embodiment described below in which the first affine seed vector, the second affine seed vector, and the third affine seed vector are used, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with the affine seed vector of the bottom-right control point (fourth affine seed vector).

[0126] The affine seed vector can be used to derive affine vectors for different sub-blocks (S803), where the affine vectors represent translational motion vectors derived based on the affine seed vectors. The affine vectors of sub-blocks can be called affine sub-block motion vectors or sub-block motion vectors.

[0127] FIG. 10 is a diagram showing an example of affine vectors of sub-blocks in a four-parameter motion model.

[0128] The affine vector of a sub-block can be derived based on the control point positions, the sub-block positions, and the affine seed vector. For example, Equation 3 shows an example of deriving an affine sub-block vector.

number

[0129] In Equation 3, (x, y) represents the position of the sub-block. Here, the position of the sub-block represents the position of the reference sample included in the sub-block. The reference sample may be a sample located at the upper left corner of the sub-block, or at least one sample located at the center in the x-axis or y-axis coordinate. (x0, y0) represents the position of the first control point, and (sv 0x ,sv 0y ) represents the first affine seed vector. Note that (x1, y1) represents the position of the second control point, and (sv 1x ,sv 1y ) represents the second affine seed vector.

[0130] If the first and second control points correspond to the upper left and upper right corners of the current block, respectively, then x1-x0 can be set to a value equal to the width of the current block.

[0131] Next, motion compensation prediction can be performed on each sub-block using the affine vector of each sub-block (S804). After performing motion compensation prediction, a prediction block for each sub-block can be generated. The prediction block of the sub-block can be set as the prediction block of the current block.

[0132] The affine seed vector of the current block may be derived based on the affine seed vectors of neighboring blocks adjacent to the current block. If the inter prediction mode of the current block is the affine merge mode, the affine seed vector of a merge candidate included in the merge candidate list may be determined as the affine seed vector of the current block. In addition, if the inter prediction mode of the current block is the affine merge mode, motion information including at least one of a reference image index, a specific direction prediction flag, or a bidirectional weighting value of the current block may be set to be the same as that of the merge candidate.

[0133] Merge candidates may be derived based on neighboring blocks of the current block, which may include at least one of spatial neighboring blocks that are spatially neighboring to the current block and temporal neighboring blocks that are included in an image different from the current image.

[0134] FIG. 11 is a diagram showing neighboring blocks that can be used to derive merge candidates.

[0135] The neighboring blocks of the current block may include at least one of the neighboring block (A) adjacent to the left of the current block, the neighboring block (B) adjacent to the information of the current block, the neighboring block (C) adjacent to the upper right corner of the current block, the neighboring block (D) adjacent to the lower left corner of the current block, or the neighboring block adjacent to the upper left corner of the current block. If the coordinates of the upper left sample of the current block are (x0, y0), the left neighboring block A includes a sample at the position (x0-1, y0+H-1), and the upper neighboring block B includes a sample at the position (x0+W-1, y0-1), where W and H represent the width and height of the current block, respectively. The upper right neighboring block C includes a sample at the position (x0+W, y0-1), and the lower left neighboring block D includes a sample at the position (x0-1, y0+H). The upper left neighboring block E includes a sample at the position (x0-1, y0-1).

[0136] When encoding neighboring blocks in affine inter prediction mode, the affine seed vectors of the merging candidates can be derived based on the affine seed vectors of the corresponding neighboring blocks. In the following description, neighboring blocks encoded in affine inter prediction mode are referred to as affine neighboring blocks.

[0137] Merge candidates for a current block can be generated by searching neighboring blocks according to a predefined scan order. The scan order can be predefined in the encoder and decoder. For example, neighboring blocks can be searched in the order of A, B, C, D, and E. Merge candidates can be derived in order from the searched affine neighboring blocks. Alternatively, the scan order can be adaptively determined based on at least one of the size, shape, or affine motion model of the current block. That is, blocks with different sizes, shapes, or affine motion models have different scan orders.

[0138] Alternatively, the blocks located above the current block are searched in order, and a merge candidate is derived from the first affine neighboring block found, and the blocks located to the left of the current block are searched in order, and a merge candidate is derived from the first affine neighboring block found. Here, the multiple neighboring blocks located above the current block may include at least one of neighboring block E, neighboring block B, and neighboring block C, and the multiple blocks located to the left of the current block may include at least one of block A and block D. In this case, neighboring block E may be classified as a block located to the left of the current block.

[0139] Although not shown, merge candidates may be derived from temporally neighboring blocks of the current block. Here, the temporally neighboring blocks may include blocks located at the same position as the current block in the co-located image or blocks adjacent to the current block. Specifically, when encoding is performed on the temporally neighboring blocks of the current block in an affine inter-prediction mode, merge candidates may be derived based on the affine seed vectors of the temporal merge candidates.

[0140] A merge candidate list including merge candidates can be generated, and the affine seed vector of one of the merge candidates included in the merge candidate list can be determined as the affine seed vector of the current block. To this end, index information identifying one of the merge candidates can be coded and transmitted via a bitstream.

[0141] As another example, multiple neighboring blocks can be searched according to the scan order, and the affine seed vector of the current block can be derived from the affine seed vector of the first affine neighboring block found.

[0142] As described above, in affine merge mode, the affine seed vector of the current block can be derived using the affine seed vectors of neighboring blocks.

[0143] When the inter prediction mode of the current block is an affine motion vector prediction mode, an affine seed vector of a motion vector prediction candidate included in the motion vector prediction candidate list may be determined as an affine seed vector prediction value of the current block, and an affine seed vector of the current block may be derived by adding an affine seed vector difference value to the affine seed vector prediction value.

[0144] Affine seed vector prediction candidates can be derived based on neighboring blocks of the current block. Specifically, multiple neighboring blocks located above the current block can be searched according to a predetermined scanning order, and a first affine seed vector prediction candidate can be derived from the first affine neighboring block found. Additionally, multiple neighboring blocks located to the left of the current block can be searched according to a predetermined scanning order, and a second affine seed vector prediction candidate can be derived from the first affine neighboring block found.

[0145] Information for determining the affine seed vector differential value may be coded and transmitted via a bitstream. The information may include size information indicating the size of the affine seed vector differential value and symbol information indicating the symbol of the affine seed vector differential value. The affine seed vector differential value for each control point may be set to be the same. Alternatively, the affine seed vector differential value may be set to be different for each control point.

[0146] As described above, the affine seed vector of a merging candidate or an affine seed vector prediction candidate may be derived from the affine seed vector of an affine neighboring block, and the affine seed vector of the current block may be derived using the affine seed vector of the derived merging candidate or affine seed vector prediction candidate. Alternatively, after searching for a plurality of affine neighboring blocks according to a predetermined scanning order, the affine seed vector of the current block may be derived from the affine seed vector of the first affine neighboring block found.

[0147] In the following description, a method for deriving an affine seed vector of a current block, a merge candidate, or an affine seed vector prediction candidate from the affine seed vectors of affine neighboring blocks will be described in detail. In the following embodiments, deriving an affine seed vector of a current block may be understood as deriving an affine seed vector of a merge candidate or deriving an affine seed vector of an affine seed vector prediction candidate.

[0148] FIG. 12 is a diagram illustrating deriving an affine seed vector for a current block based on the affine seed vectors of affine neighboring blocks.

[0149] When a first affine seed vector nv0 for the top-left control point and a second affine seed vector nv1 for the top-right control point are stored in an affine neighboring block, a third affine seed vector nv2 for the bottom-left control point of the affine neighboring block can be derived based on the first and second affine seed vectors. Equation 4 shows an example of deriving the third affine seed vector.

number

[0150] In equation 4, (nv 0x ,nv 0y ) represents the first affine seed vector nv0, and (nv 1x ,nv 1y ) represents the second affine seed vector nv1, and (nv 2x ,nv 2y ) represents the third affine seed vector nv2. Note that (x n0 ,x n0 ) represents the position of the first control point, and (x n1 ,x n1 ) represents the position of the second control point, and (x n2 ,x n2 ) represents the position of the third control point.

[0151] Subsequently, an affine seed vector for the current block can be derived using the first affine seed vector, the second affine seed vector, and the third affine seed vector. Equation 5 shows an example of deriving the first affine seed vector v0 for the current block, and Equation 6 shows an example of deriving the second affine seed vector v1 for the current block.

number

number

[0152] In Equation 5 and Equation 6, (v 0x ,v0y ) represents the first affine seed vector sv0 of the current block, and (v 1x ,v 1y ) represents the second affine seed vector sv1 of the current block. Note that (x0, y0) represents the position of the first control point, and (x1, y1) represents the position of the second control point. For example, the first control point represents the upper left corner of the current block, and the second control point represents the upper right corner of the current block.

[0153] In the above example, it has been described that the multiple affine seed vectors of the current block are derived using three affine seed vectors related to the affine neighboring blocks. As another example, the affine seed vector of the current block can be derived using only two of the multiple affine seed vectors of the affine neighboring blocks.

[0154] Alternatively, the multiple affine seed vectors for the current block can be derived using the fourth affine seed vector related to the lower right corner without using the first affine seed vector related to the upper left corner, the second affine seed vector related to the upper right corner, or the third affine seed vector related to the lower left corner of the affine neighboring blocks.

[0155] In particular, when the upper boundary of the current block touches the upper boundary of a coding tree unit and the affine seed vector of the upper control point (e.g., the upper left corner or the upper right corner) of the affine neighboring block (hereinafter referred to as the upper affine neighboring block) adjacent to the current block above is to be used, these vectors must be pre-stored in memory. This may cause a problem of an increase in the number of line buffers. Therefore, when the upper boundary of the current block touches the upper boundary of a coding tree unit, the affine seed vector of the lower control point (e.g., the lower left corner or the lower right corner) can be set for the upper affine neighboring block without using the affine seed vector of the upper control point. For example, multiple affine seed vectors for the current block can be derived using the third affine seed vector associated with the lower left corner and the fourth affine seed vector associated with the lower right corner of the upper affine neighboring block. In this case, the affine seed vector associated with the lower corner can be derived by duplicating the affine seed vector associated with the upper corner, or can be derived from the affine seed vectors associated with multiple upper corners. For example, the first affine seed vector, the second affine seed vector or the third affine seed vector can be transformed / replaced with a fourth affine seed vector associated with the bottom right corner.

[0156] Equation 7 and Equation 8 show an example of deriving the first and second affine seed vectors of the current block using the third affine seed vector associated with the bottom-left control point and the fourth affine seed vector associated with the bottom-right control point of the neighboring affine vectors.

number

number

[0157] In equations 7 and 8, (x n2 ,y n2 ) represents the coordinate of the bottom-left control point of the affine neighboring block, and (x n3 ,y n3) represents the coordinates of the bottom right control point of the affine neighboring block. (x0,y0) represents the coordinates of the top left control point of the current block, and (x1,y1) represents the coordinates of the top right control point of the current block. (nv 2x ,nv 2y ) represents the affine seed vector of the bottom-left control point of the affine neighboring block (i.e., the third affine seed vector), and (nv 3x ,nv 3y ) represents the affine seed vector of the bottom-right control point of the affine neighboring block (i.e., the fourth affine seed vector). 0x ,v 0y ) represents the affine seed vector of the top-left control point of the current block (i.e., the first affine seed vector), and (v 1x ,v 1y ) represents the affine seed vector of the top-right control point of the current block (i.e., the second affine seed vector).

[0158] The divisions in Equation 7 and Equation 8 can be changed to shift operations. The shift operation is calculated by dividing the width between the bottom-left control point and the bottom-right control point (i.e., (x n3 -x n2 )) based on values ​​derived from

[0159] In the above example, multiple affine seed vectors for the current block can be derived based on multiple affine seed vectors of the coded / decoded affine neighboring blocks. To achieve this, multiple affine seed vectors for the coded / decoded affine neighboring blocks can be stored in memory. However, since multiple affine seed vectors for the affine neighboring blocks are stored in memory in addition to multiple translational motion vectors (i.e., multiple affine vectors) for multiple sub-blocks included in the affine neighboring blocks, this can result in increased memory usage. To solve this problem, the affine seed vector for the current block can be derived using the motion vector of a sub-block adjacent to a control point of the affine neighboring block. This replaces the affine seed vector of the affine neighboring block. In other words, the motion vector of the sub-block adjacent to the control point of the affine neighboring block can be set as the affine seed vector of the affine neighboring block. Here, the sub-block may be a block having a size / shape predefined by the encoder and decoder, or may be a block having a basic size / shape for storing motion vectors. For example, the sub-block may be a square block having a size of 4x4. Alternatively, the motion vector specifying the sample position can be set as the affine seed vector of the affine neighboring block.

[0160] FIG. 13 is a diagram showing an example in which the motion vector of a sub-block is set as the affine seed vector of an affine adjacent block.

[0161] The motion vector of a sub-block adjacent to a control point can be used as the affine seed vector of the corresponding control point. For example, in the example shown in FIG. 13, the motion vector (nv 4x ,nv 4y ) to the affine seed vector (nv 2x ,nv 2y ), and the motion vector (nv5x ,nv 5y ) is the affine seed vector (nv 3x ,nv 3y ), where the bottom-left sub-block is the bottom-left control point (x n2 ,y n2 ) adjacent samples (e.g., (x n2 ,y n2-1 ) position), and the bottom-right sub-block is the sub-block containing the bottom-right control point (x n3 ,y n3 ) adjacent samples (e.g., (x n3-1 ,y n3-1 )) position) is a block containing a sample at position )) When deriving the affine seed vector of the current block based on Equation 7 and Equation 8, the motion vector of the lower-left sub-block of the affine neighboring block can be used instead of the third affine seed vector, and the motion vector of the lower-right sub-block can be used instead of the fourth affine seed vector.

[0162] In the following description of the embodiments, a sub-block used as an affine seed vector for an affine neighboring block is called an affine sub-block.

[0163] According to one embodiment of the present invention, an affine sub-block can be determined based on a sample at a specific position. For example, a sub-block including a sample at a specific position can be defined as an affine sub-block. Hereinafter, the sample at the specific position will be referred to as an affine reference sample. Note that the reference sample for determining the affine sub-block of the bottom-left control point will be referred to as the bottom-left reference sample, and the reference sample for determining the affine sub-block of the bottom-right control point will be referred to as the bottom-right reference sample.

[0164] The lower-left reference sample and the lower-right reference sample may be selected from a plurality of samples included in the affine neighboring blocks. For example, at least one of the upper-left sample, the lower-left sample, the upper-right sample, or the lower-left sample of the lower-left sub-block may be the lower-left reference sample, and at least one of the upper-left sample, the lower-left sample, the upper-right sample, or the lower-left sample of the lower-right sub-block may be the lower-right reference sample. Therefore, the motion vectors of the lower-left sub-block including the lower-left reference sample and the lower-right sub-block including the lower-right reference sample may be the affine seed vector associated with the lower-left control point and the affine seed vector associated with the lower-right control point, respectively.

[0165] As another example, at least one of the bottom-left reference sample or the bottom-right reference sample may be a sample located outside the affine neighboring block, as will be described in detail with reference to FIGS.

[0166] 14 to 16 are diagrams showing the positions of the reference samples.

[0167] For example, as shown in FIG. 14(a), for the bottom-left control point, the top-left sample of the bottom-left sub-block is used as the reference sample (x n4 ,y n4 ) can be used. Therefore, the reference sample (x n4 ,y n4 ) can be the affine sub-block related to the lower-left control point.

[0168] For the bottom-right control point, the sample located to the right of the top-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be used. Therefore, the reference sample (x n5 ,y n5 ) can be the affine sub-block related to the lower-right control point.

[0169] Alternatively, as shown in the example of FIG. 14(b), for the lower-left control point, the sample located to the left of the upper-left sample of the lower-left sub-block is used as the reference sample (x n4 ,y n4 ) can be used. Therefore, the reference sample (x n4 ,y n4 ) can be used as an affine sub-block related to the lower-left control point.

[0170] For the bottom-right control point, the top-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be used. Therefore, the reference sample (x n5 ,y n5 ) can be the affine sub-block related to the lower-right control point.

[0171] Alternatively, as shown in the example of FIG. 15(a), for the bottom-left control point, the bottom-left sample of the bottom-left sub-block is used as the reference sample (x n4 ,y n4 ) can be used. Therefore, the reference sample (x n4 ,y n4 ) can be the affine sub-block related to the lower-left control point.

[0172] For the bottom-right control point, the sample located to the right of the bottom-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be used. Therefore, the reference sample (x n5 ,y n5 ) can be the affine sub-block related to the lower-right control point.

[0173] Alternatively, as shown in the example of FIG. 15(b), for the lower-left control point, the sample located to the left of the lower-left sample of the lower-left sub-block is used as the reference sample (x n4 ,y n4 ) can be used. Therefore, the reference sample (xn4 ,y n4 ) can be the affine sub-block related to the lower-left control point.

[0174] For the bottom-right control point, the bottom-right sample of the bottom-right sub-block is used as the reference sample (x n5 ,y n5 ) can be used. Therefore, the reference sample (x n5 ,y n5 ) can be the affine sub-block related to the lower-right control point.

[0175] Alternatively, as shown in the example of FIG. 16(a), for the lower-left control point, a sample located between the upper-left sample and the lower-left sample of the lower-left sub-block (for example, the left middle sample) is used as the reference sample (x n4 ,y n4 ) can be used. Therefore, the reference sample (x n4 ,y n4 ) can be the affine sub-block related to the lower-left control point.

[0176] For the lower-right control point, the sample located to the right of the sample between the upper-right sample and the lower-right sample of the lower-right sub-block (e.g., the right-side middle sample) is used as the reference sample (x n5 ,y n5 ) can be used. Therefore, the reference sample (x n5 ,y n5 ) can be the affine sub-block related to the lower-right control point.

[0177] Alternatively, as shown in the example of FIG. 16(b), for the lower-left control point, the sample located to the left of the sample between the upper-left sample and the lower-left sample of the lower-left sub-block is set as the reference sample (x n4 ,y n4 ) can be used. Therefore, the reference sample (x n4 ,y n4) can be the affine sub-block related to the lower-left control point.

[0178] For the bottom-right control point, the sample located between the top-right sample and the bottom-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be used. Therefore, the reference sample (x n5 ,y n5 ) can be the affine sub-block related to the lower-right control point.

[0179] When deriving the multiple affine seed vectors for the current block based on Equation 7 and Equation 8, the third affine seed vector of the affine neighboring block can be replaced with the motion vector of the affine sub-block related to the bottom-left control point, the fourth affine seed vector can be replaced with the motion vector of the affine sub-block related to the bottom-right control point, the position of the bottom-left control point can be replaced with the position of the bottom-left reference sample, and the position of the bottom-right control point can be replaced with the position of the bottom-right reference sample.

[0180] Unlike the descriptions in Figures 14 to 16, a sub-block including a sample adjacent to a reference sample can also be an affine sub-block. Specifically, a sample located outside an affine adjacent sub-block can be used as a reference sample, and a sub-block included in an affine adjacent block can be used as an affine sub-block. For example, in the example shown in Figure 14(a), a sample located to the right of the upper right sample in the lower right sub-block can be used as a reference sample (x n5 ,y n5 ), and the bottom right sub-block can be the affine sub-block associated with the bottom right corner. Alternatively, in the example shown in FIG. 14(b), the sample located to the left of the top left sample of the bottom left sub-block can be the reference sample (x n4 ,y n4), and the bottom-left sub-block may be the affine sub-block associated with the bottom-left corner.

[0181] 15 and 16 can be similarly applied. That is, in the example shown in FIG. 15(a) or FIG. 16(a), the sample located to the right of the bottom right sample or the right middle sample of the bottom right sub-block is used as the reference sample (x n5 ,y n5 ), and the lower right sub-block can be the affine sub-block related to the lower right corner. Alternatively, in the example shown in FIG. 15(b) or FIG. 16(b), the sample located to the left of the lower left sample or the left middle sample of the lower left sub-block can be used as the reference sample (x n4 ,y n4 ), and the bottom-left sub-block may be the affine sub-block associated with the bottom-left corner.

[0182] In the above example, the motion vectors of the affine sub-blocks can be used to derive the affine seed vectors of the affine neighboring blocks, so that for coded / decoded blocks, the motion vectors can be stored in units of sub-blocks.

[0183] As another example, after storing a minimum number of affine seed vectors for an affine neighboring block, the stored affine seed vectors can be used to derive a motion vector for an affine sub-block.

[0184] Equation 9 and Equation 10 show an example of deriving a motion vector for an affine sub-block using the affine seed vectors of affine neighboring blocks.

number

number

[0185] In Equation 9 and Equation 10, (nv 4x ,nv 4y ) represents the motion vector of the affine sub-block related to the bottom-left control point, and (nv 5x ,nv 5y ) represents the motion vector of the affine sub-block related to the bottom-right control point. Since the motion vector of the affine sub-block and the affine seed vector of the control point are set to be the same, (nv 4x ,nv 4y ), the affine seed vector (nv 2x ,nv 2y ) can be used, or (nv 5x ,nv 5y ), the affine seed vector (nv 3x ,nv 3y ) can be used.

[0186] (x n4 ,y n4 ) represents the position of the reference sample of the lower left sub-block. Alternatively, the center position of the lower left sub-block or the position of the lower left control point can be used instead. (x n5 ,y n5 ) represents the position of the reference sample of the lower right sub-block. Alternatively, the center position of the lower right sub-block or the position of the lower right control point can be used instead of the position.

[0187] When the current block does not contact the boundary of the coding tree unit, Equation 9 and Equation 10 can be applied. When the current block contacts the top boundary of the coding tree unit, instead of applying Equation 9 and Equation 10, the translational motion vector of the affine sub-block determined based on the bottom-left reference sample can be set as the third affine seed vector, and the translational motion vector of the affine sub-block determined based on the bottom-right reference sample can be set as the fourth affine seed vector.

[0188] In Equation 7 and Equation 8, (x n3 -x n2) represents the distance between the bottom left and bottom right control points. n3 Instead, the position of the bottom right reference sample x n5 can be used, and x n2 Instead, the position of the bottom left reference sample x n4 can be used. Below, (x n3 -x n2 ) or the value obtained by substituting the position of the reference sample for the equation (e.g., (x n5 -x n4 )) to variable W seed and the variable is called the subseed vector width.

[0189] According to the location of the reference sample, the subseed vector width is a power series of 2 (e.g., 2 n ) may not be a multiple of 2. For example, if the lower-left sample of the lower-left sub-block is the lower-left reference sample and the lower-right sample of the lower-right sub-block is the lower-right reference sample, the sub-seed vector width is not a multiple of 2. As described above, if the sub-seed vector width is not a power of 2, it can be converted to a power of 2. The conversion may include adding / subtracting an offset to / from the sub-seed vector width, or using the position of a sample adjacent to the reference sample instead of the position of the reference sample. For example, the converted sub-seed vector width can be derived by adding 1 to the width between the lower-left reference sample and the lower-right reference sample. Alternatively, the width between the adjacent reference sample adjacent to the right of the lower-right reference sample and the lower-left reference sample can be used as the converted sub-seed vector width. Subsequently, the affine seed vector of the current block can be derived by substituting the converted sub-seed vector width into Equations 7 and 8.

[0190] The division operations in Equation 7 and Equation 8 can also be changed to shift operations. The shift operations can be performed based on values ​​derived from the transformed subseed vector widths (i.e., values ​​expressed as a power of 2 series).

[0191] When the reference sample for determining the affine sub-block does not belong to the affine neighboring block, the affine seed vector of the affine neighboring block can be derived based on a sample adjacent to the reference sample among the multiple samples included in the affine neighboring block. Specifically, the translational motion vector of the sub-block including the sample adjacent to the reference sample in the affine neighboring block (hereinafter referred to as the adjacent reference sample) can be used as the affine seed vector of the affine neighboring block. As described above, the method of deriving the affine seed vector using the adjacent reference sample can be defined as a modified affine merge vector derivation method.

[0192] FIG. 17 is a diagram showing an application example of the modified affine merge vector derivation method.

[0193] The lower right reference sample (x n5 ,y n5 ) does not belong to the affine neighboring block, the sample (x n5-1 ,y n5 ), we can derive an affine seed vector based on the neighboring reference samples (x n5-1 ,y n5 ) can be taken as the affine seed vector of the bottom-right control point.

[0194] 17, the sample adjacent to the right of the upper right sample of the lower right sub-block is shown as the lower right reference sample. When the sample adjacent to the right of the lower right sample of the lower right sub-block or the sample adjacent to the right of the right middle sample of the lower right sub-block is used as the lower right reference sample, an affine seed vector can be derived based on the sample adjacent to the left of the adjacent reference sample.

[0195] It should be noted that if the bottom-left reference sample does not belong to an affine neighboring block, the affine seed vector can also be derived based on the neighboring sample to the right of the bottom-left reference sample, depending on the described embodiment.

[0196] By setting the positions of the reference samples and the sub-blocks for deriving the affine seed vectors in different ways, the width of the sub-seed vectors can be a power of two.

[0197] A merge candidate, an affine seed vector prediction candidate, or an affine seed vector for the current block can be derived by using neighboring blocks around the current block that are not coded in affine inter mode. Specifically, blocks that are not coded in affine inter mode can be combined, and the combination can be used as a merge candidate or an affine seed vector prediction candidate. For example, a combination of at least one of the motion vectors of any one of the neighboring blocks adjacent to the upper left corner of the current block, any one of the motion vectors of any one of the neighboring blocks adjacent to the upper right corner of the current block, and any one of the neighboring blocks adjacent to the lower left corner of the current block can be used as a merge candidate or an affine seed vector prediction candidate. In this case, the motion vectors of the neighboring blocks adjacent to the upper left corner, the motion vectors of the neighboring blocks adjacent to the upper right corner, and the motion vectors of the neighboring blocks adjacent to the lower left corner can be used as a first affine seed vector for the upper left control point, a second affine seed vector for the upper right control point, and a third affine seed vector for the lower left control point, respectively.

[0198] Alternatively, in the modified affine merge vector derivation method, a merge candidate, an affine seed vector prediction candidate, or an affine seed vector for the current block can be derived using a neighboring block that is not coded in affine inter mode. In the following, a neighboring block that is not coded in affine inter mode is referred to as a non-affine neighboring block.

[0199] FIG. 18 illustrates an example of deriving an affine seed vector for a current block based on non-affine neighboring blocks.

[0200] In the example shown in FIG. 18, it is assumed that all neighboring blocks adjacent to the current block are non-affine neighboring blocks.

[0201] When deriving an affine seed vector of the current block from a non-affine neighboring block A of the neighboring blocks adjacent to the current block, a bottom-left reference sample and a bottom-right reference sample of A can be set. For example, the sample adjacent to the left of the bottom-left sample of block A can be the bottom-left reference sample, and the bottom-right sample of block A can be the bottom-right reference sample. Because the bottom-left reference sample is outside block A, the motion vector of a sub-block including the sample adjacent to the right of the bottom-left reference sample can be the third affine seed vector of block A. Furthermore, the motion vector of a sub-block including the bottom-right reference sample can be the fourth affine seed vector of block A. Next, the first and second affine seed vectors of the current block can be derived from block A based on Equations 9 and 10.

[0202] Only when motion compensation prediction for non-affine neighboring blocks is performed on a sub-block basis, a method of deriving an affine seed vector from a non-affine neighboring block can be used. Here, the prediction technique for performing motion compensation prediction on a sub-block basis may include at least one of STMVP, ATMVP, bidirectional optical flow (BIO), overlapped block motion compensation (OBMC), and decoder-side motion vector modification (DMVR).

[0203] In the above embodiment, when the upper boundary of the current block contacts the boundary of the coding tree unit, the third affine seed vector of the bottom-left control point and the fourth affine seed vector of the bottom-right control point of the affine adjacent block located above the current block are used to derive a merge candidate, an affine seed vector prediction candidate, or an affine seed vector for the current block.

[0204] As another example, if the upper boundary of the current block touches the boundary of a coding tree unit and the neighboring block located above the current block belongs to a coding tree unit different from the coding tree unit of the current block, the neighboring block is not used, but the merge candidate, affine seed vector prediction candidate, or affine seed vector of the current block is derived using the neighboring block that is closest to the neighboring block among these blocks included in the coding tree unit to which the current block belongs.

[0205] 19, the current block touches the upper boundary of the coding tree unit, and blocks B, C, and E located above the current block belong to a coding tree unit different from the coding tree unit of the current block. Therefore, instead of block E, block F, which is adjacent to block E among these blocks included in the coding tree unit to which the current block belongs, is used to derive the affine seed vector of the current block.

[0206] Affine seed vectors of multiple blocks can be used for motion compensation prediction of the current block. For example, multiple merge candidates can be selected from a merge candidate list, and an affine seed vector or a sub-block vector of the current block can be derived based on the affine seed vectors of the selected merge candidates. Performing encoding / decoding of the current block using the affine seed vectors of multiple blocks may be referred to as a multi-affine merge encoding method.

[0207] Information indicating whether the multi-affine merge coding method has been applied to the current block may be coded and transmitted via a bitstream. Alternatively, it may be determined whether the multi-affine merge coding method has been applied to the current block based on at least one of the number of affine neighboring blocks among neighboring blocks of the current block, the number of merge candidates included in the merge candidate list, and the affine motion model of the current block.

[0208] 20 and 21 are flowcharts showing a motion compensation prediction method using multiple merging candidates.

[0209] Figure 20 illustrates an example of deriving an affine seed vector for a current block by using affine seed vectors of multiple merging candidates. Figure 21 illustrates an example of deriving a motion vector for each sub-block by using affine seed vectors of multiple merging candidates.

[0210] The affine seed vector for the current block can be generated based on the sum, difference, average, or weighted addition of the affine seed vectors of the two merge candidates.

[0211] Equations 11 and 12 below show an example of deriving the affine seed vector of the current block by adding the affine seed vectors of the merging candidates.

number

number

[0212] In Equations 11 and 12, sv4 represents the first affine seed vector of the current block, sv0 represents the first affine seed vector of the first merging candidate, sv2 represents the first affine seed vector of the second merging candidate, sv5 represents the second affine seed vector of the current block, sv1 represents the second affine seed vector of the first merging candidate, and sv3 represents the second affine seed vector of the second merging candidate.

[0213] Note that the following equations 13 and 14 show an example of deriving the affine seed vector of the current block by weighted addition of the affine seed vectors of the merging candidates.

number

number

[0214] As another example, a first sub-block motion vector and a second sub-block motion vector may be generated for each sub-block in the current block based on the affine seed vector of the first merging candidate and the affine seed vector of the second merging candidate, and then a final sub-block motion vector may be generated based on the sum, difference, average, or weighted sum of the first sub-block motion vector and the second sub-block motion vector.

[0215] Equation 15 below shows an example of obtaining the final sub-block motion vector by adding the first sub-block motion vector and the second sub-block motion vector.

number

[0216] In Equation 15, V0 represents the first sub-block motion vector, V1 represents the second sub-block motion vector, and V2 represents the final sub-block motion vector.

[0217] Note that the following equation 16 shows an example of deriving the final sub-block motion vector by weighted addition of the first sub-block motion vector and the second sub-block motion vector.

number

[0218] Intra prediction is a method of predicting a current block using coded / decoded reconstructed samples around the current block. In this case, the intra prediction of the current block can use reconstructed samples before applying an in-loop filter.

[0219] Intra prediction techniques include matrix-based intra prediction and general intra prediction that takes into account the directionality of surrounding reconstructed samples. Information indicating the intra prediction technique of the current block may be transmitted as a signal via a bitstream. The information may be a one-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, and shape of the current block or the intra prediction techniques of neighboring blocks. For example, if the current block straddles an image boundary, the current block may be set to not apply matrix-based intra prediction.

[0220] Matrix-based intra prediction is a method of obtaining a prediction block of a current block based on matrix multiplication of a matrix stored in an encoder and a decoder with reconstructed samples surrounding the current block. Information indicating one of a plurality of stored matrices can be transmitted as a signal via a bitstream. The decoder can determine the matrix to be used for intra prediction of the current block based on the information and the size of the current block.

[0221] The general intra prediction is a method of obtaining a predicted block of a current block based on a non-angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on the general intra prediction will be described in detail with reference to the accompanying drawings.

[0222] FIG. 22 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention.

[0223] A reference sample line of the current block can be determined (S2201). The reference sample line is a set of reference samples included in the Kth line shifted from the top and / or left side of the current block. The reference samples can be derived from the coded / decoded reconstructed samples around the current block.

[0224] Index information indicating a reference sample line of a current block among a plurality of reference sample lines may be transmitted as a signal via a bitstream. The plurality of reference sample lines may be included in at least one of the first, second, third, and fourth lines located above and / or to the left of the current block. Table 1 shows indexes assigned to each reference sample line. In Table 1, it is assumed that the first, second, and fourth lines are used as reference sample line candidates. [Table 1]

[0225] The reference sample line of the current block may also be determined based on at least one of the position, size, or shape of the current block or the predictive coding mode of a neighboring block. For example, if the current block borders a boundary of an image, a tile, a slice, or a coding tree unit, the first reference sample line may be determined as the reference sample line of the current block.

[0226] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper reference sample and the left reference sample may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before applying an in-loop filter.

[0227] FIG. 23 is a diagram showing reference samples included in each reference sample line.

[0228] According to the intra prediction mode of the current block, a prediction sample can be obtained using at least one of the reference samples belonging to the reference sample line.

[0229] Next, an intra prediction mode of the current block may be determined (S2202). Regarding the intra prediction mode of the current block, at least one of a non-angular intra prediction mode or an angular intra prediction mode may be determined as the intra prediction mode of the current block. The non-angular intra prediction modes include planar and DC, and the angular intra prediction modes include 33 or 65 modes from the lower-left diagonal to the upper-right diagonal.

[0230] FIG. 24 is a diagram showing intra prediction modes.

[0231] FIG. 24(a) shows 35 intra prediction modes, and FIG. 24(b) shows 67 intra prediction modes.

[0232] More or fewer intra-prediction modes than those shown in FIG. 24 may be defined.

[0233] A Most Probable Mode (MPM) may be set based on the intra-prediction modes of neighboring blocks adjacent to the current block. Here, the neighboring blocks may include a left neighboring block adjacent to the left of the current block and an upper neighboring block adjacent above the current block. If the coordinates of the upper left sample of the current block are (0,0), the left neighboring block may include a sample at a position (-1,0), (-1,H-1), or (-1,(H-1) / 2), where H represents the height of the current block. The upper neighboring block may include a sample at a position (0,-1), (W-1,-1), or ((W-1) / 2,-1), where W represents the width of the current block.

[0234] When encoding adjacent blocks using general intra prediction, the MPM can be derived based on the intra prediction modes of the adjacent blocks. Specifically, the intra prediction mode of the left adjacent block can be set as a variable candIntraPredModeA, and the intra prediction mode of the upper adjacent block can be set as a variable candIntraPredModeB.

[0235] In this case, when a neighboring block is unavailable (for example, when the neighboring block has not been coded / decoded or the position of the neighboring block is shifted from the image boundary), when the neighboring block is coded by matrix-based intra prediction, when the neighboring block is coded by inter prediction, or when the neighboring block is included in a coding tree unit different from that of the current block, a variable candIntraPredModeX (where X is A or B) derived based on the intra prediction mode of the neighboring block may be set as the default mode. Here, the default mode may include at least one of a planar mode, a DC mode, a vertical mode, or a horizontal mode.

[0236] Alternatively, when encoding neighboring blocks using intra prediction based on a matrix, the intra prediction mode corresponding to an index value for specifying one of the matrices can be set to candIntraPredModeX. For this purpose, a lookup table indicating a mapping relationship between index values ​​for specifying a matrix and intra prediction modes can be stored in advance in the encoder and decoder.

[0237] The MPM can be derived based on the variables candIntraPredModeA and candIntraPredModeB. The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs may be three, four, five, or six. Alternatively, information indicating the number of MPMs can be transmitted in a signal via the bitstream. Alternatively, the number of MPMs can be determined based on at least one of the predictive coding modes of neighboring blocks and the size or shape of the current block.

[0238] In the following examples, it is assumed that the number of MPMs is three, and the three MPMs are referred to as MPM[0], MPM[1], and MPM[2]. If the number of MPMs is more than three, the MPM may include three MPMs as described in the following examples.

[0239] If candIntraPredA is the same as candIntraPredB and candIntraPredA is planar mode or DC mode, MPM[0] and MPM[1] can be planar mode and DC mode, respectively. MPM[2] can be vertical intra prediction mode, horizontal intra prediction mode, or diagonal intra prediction mode. The diagonal intra prediction mode may be bottom-left diagonal intra prediction mode, top-left intra prediction mode, or top-right intra prediction mode.

[0240] If candIntraPredA is the same as candIntraPredB and candIntraPredA is an intra prediction mode, MPM[0] can be set to be the same as candIntraPredA. MPM[1] and MPM[2] can be set to intra prediction modes similar to candIntraPredA. An intra prediction mode similar to candIntraPredA may be an intra prediction mode whose index difference value from candIntraPredA is ±1 or ±2. An intra prediction mode similar to candIntraPredA can be derived using modulo arithmetic (%) and an offset.

[0241] If candIntraPredA is different from candIntraPredB, MPM[0] and candIntraPredA can be set to be the same, and MPM[1] and candIntraPredB can be set to be the same. In this case, if candIntraPredA and candIntraPredB are both non-angular intra prediction modes, MPM[2] can be a vertical intra prediction mode, a horizontal intra prediction mode, or a diagonal intra prediction mode. Alternatively, if at least one of candIntraPredA and candIntraPredB is an angular intra prediction mode, MPM[2] can be an intra prediction mode derived by adding or subtracting an offset to the larger value of plane, DC, candIntraPredA, or candIntraPredB. Here, the offset may be 1 or 2.

[0242] An MPM list including multiple MPMs may be generated, and information indicating whether an MPM that is the same as the intra prediction mode of the current block is included in the MPM list may be transmitted as a signal via a bitstream. The information may be a 1-bit flag and may be referred to as an MPM flag. If the MPM flag indicates that an MPM that is the same as the current block is included in the MPM list, index information indicating one of the MPMs may be transmitted as a signal via a bitstream. The MPM designated by the index information may be used as the intra prediction mode of the current block. If the MPM flag indicates that an MPM that is the same as the current block is not included in the MPM list, residual mode information indicating one of residual intra prediction modes other than the MPM may be transmitted as a signal via a bitstream. The residual mode information represents an index value corresponding to the intra prediction mode of the current block when reallocating indexes to residual intra prediction modes other than the MPM. The decoder may determine the intra prediction mode of the current block by arranging the MPMs in ascending order and comparing the residual mode information with the MPMs. For example, if the residual mode information is equal to or smaller than the MPM, the intra prediction mode of the current block can be derived by adding 1 to the residual mode information.

[0243] Instead of setting the default mode as the MPM, information indicating whether the intra prediction mode of the current block is the default mode may be transmitted via a signal through a bitstream. The information may be a 1-bit flag, and the flag may be referred to as a default mode flag. The MPM flag may transmit the default mode flag via a signal only if it indicates that the MPM that is the same as that of the current block is included in the MPM list. As described above, the default mode may include at least one of planar, DC, vertical mode, or horizontal mode. For example, if planar is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is planar. If the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

[0244] If multiple intra prediction modes are set as default modes, the signal may further transmit index information indicating one of the default modes, and the intra prediction mode of the current block may be set as the default mode indicated by the index information.

[0245] If the index of the reference sample line of the current block is not 0, the default mode is set to not be used. Therefore, if the index of the reference sample line is not 0, the default mode flag can be set to a predefined value (i.e., false) without transmitting the default mode flag by signal.

[0246] When the intra prediction mode of the current block is determined, a prediction sample for the current block can be obtained based on the determined intra prediction mode (S2203).

[0247] When the DC mode is selected, a predicted sample for the current block may be generated based on the average value of the reference samples. Specifically, values ​​of all samples in the predicted block may be generated based on the average value of the reference samples. The average value may be derived using at least one of an upper reference sample located above the current block and a left reference sample located to the left of the current block.

[0248] The number or range of reference samples for deriving the average value may vary depending on the shape of the current block. For example, if the current block is a non-square block whose width is greater than its height, the average value may be calculated using only the upper reference samples. On the other hand, if the current block is a non-square block whose width is less than its height, the average value may be calculated using only the left reference samples. That is, if the width and height of the current block are different, the average value may be calculated using only the reference samples adjacent to the longer side. Alternatively, it may be determined whether to calculate the average value using only the upper reference samples or only the left reference samples based on the ratio of the width and height of the current block.

[0249] When the planar mode is selected, a prediction sample can be obtained using a horizontal prediction sample and a vertical prediction sample. Here, the horizontal prediction sample is obtained based on a left reference sample and a right reference sample located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on an upper reference sample and a lower reference sample located on the same vertical line as the prediction sample. Here, the right reference sample can be generated by duplicating the reference sample adjacent to the upper right corner of the current block, and the lower reference sample can be generated by duplicating the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample can be obtained based on a weighted addition of the left reference sample and the right reference sample, and the vertical prediction sample can be obtained based on a weighted addition of the upper reference sample and the lower reference sample. In this case, a weight assigned to each reference sample can be determined based on the position of the prediction sample. The prediction sample can be obtained based on an average operation or weighted addition of the horizontal prediction sample and the vertical prediction sample. When performing weighted addition, the weight assigned to the horizontal prediction sample and the vertical prediction sample can be determined based on the position of the prediction sample.

[0250] When an angular prediction mode is selected, a parameter representing the prediction direction (or prediction angle) of the selected angular prediction mode can be determined. Table 2 below shows the intra prediction parameter intraPredAng for each intra prediction mode. [Table 2]

[0251] When 35 intra prediction modes are defined, Table 2 shows the intra direction parameters of each intra prediction mode having an index of any one of 2 to 34. When more than 33 angular intra prediction modes are defined, Table 2 sets the intra direction parameters of each angular intra prediction mode in more detail.

[0252] After arranging the upper reference samples and the left reference samples of the current block in a row, a predicted sample can be obtained based on the value of the intra direction parameter. In this case, if the value of the intra direction parameter is negative, the left reference samples and the upper reference samples can be arranged in a row.

[0253] 25 and 26 are diagrams showing examples of a one-dimensional array in which reference samples are arranged in one row.

[0254] 25 shows an example of a vertical one-dimensional array in which reference samples are arranged vertically, and FIG. 26 shows an example of a horizontal one-dimensional array in which reference samples are arranged horizontally. The examples of FIGS. 25 and 26 will be explained assuming that 35 intra prediction modes have been defined.

[0255] If the intra-prediction mode index is one of 11 to 18, a horizontal one-dimensional array in which the upper reference sample is rotated counterclockwise can be applied, and if the intra-prediction mode index is one of 19 to 25, a vertical one-dimensional array in which the left reference sample is rotated clockwise can be applied. When the reference samples are arranged in a row, the intra-prediction mode angle can be taken into consideration.

[0256] Based on the intra direction parameters, reference sample decision parameters can be determined, which may include a reference sample index for specifying the reference sample and a weight parameter for determining a weight to be applied to the reference sample.

[0257] The reference sample index iIdx and the weight parameter ifact are obtained by the following equations 17 and 18, respectively.

number

number

[0258] In equations 17 and 18, P ang represents an intra direction parameter. The reference sample specified by the reference sample index iIdx corresponds to an integer pel.

[0259] One or more reference samples may be designated to derive a prediction sample. Specifically, the position of the reference sample for deriving the prediction sample may be designated in consideration of the gradient of the prediction mode. For example, the reference sample for deriving the prediction sample may be designated using a reference sample index iIdx.

[0260] In this case, if the gradient of the intra prediction mode cannot be represented by one reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, if the gradient of the intra prediction mode is a value between the gradient between the prediction sample and a first reference sample and the gradient between the prediction sample and a second reference sample, the prediction sample can be obtained by interpolating the first reference sample and the second reference sample. In other words, if an angular line following the intra prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample can be obtained by interpolating reference samples adjacent to the left and right or above and below the position where the angular line passes.

[0261] Equation 19 below shows an example of obtaining a predicted sample based on a reference sample.

number

[0262] In Equation 19, P represents a predicted sample, and Ref_1D represents any one of the reference samples in the one-dimensional array, where the position of the reference sample can be determined based on the position (x, y) of the predicted sample and the reference sample index iIdx.

[0263] When the gradient of the intra prediction mode is represented as one reference sample, the weight parameter ifact can be set to 0. Therefore, Equation 19 can be simplified to Equation 20 below.

number

[0264] Intra prediction may be performed on the current block based on multiple intra prediction modes. For example, intra prediction modes may be derived for different prediction samples, and prediction samples may be derived based on the intra prediction modes assigned to each prediction sample.

[0265] Alternatively, intra-prediction modes may be derived for different regions, and intra-prediction may be performed for each region based on the intra-prediction mode assigned to each region. Here, the region may include at least one sample. At least one of the size or shape of the region may be adaptively determined based on at least one of the size, shape, or intra-prediction mode of the current block. Alternatively, at the encoder and decoder, at least one of the size or shape of the region may be predefined, regardless of the size or shape of the current block.

[0266] Alternatively, intra prediction may be performed based on multiple intra predictions, and a final predicted sample may be derived based on an average or weighted sum of multiple predicted samples obtained by multiple intra predictions. For example, a first predicted sample may be obtained by performing intra prediction based on a first intra prediction mode, and a second predicted sample may be obtained by performing intra prediction based on a second intra prediction mode. Subsequently, a final predicted sample may be obtained based on an average or weighted sum of the first predicted sample and the second predicted sample. In this case, the weights assigned to the first predicted sample and the second predicted sample may be determined by considering at least one of whether the first intra prediction mode is a non-angular / angular prediction mode, whether the second intra prediction mode is a non-angular / angular prediction mode, or the intra prediction mode of a neighboring block.

[0267] The multiple intra prediction modes may be a combination of a non-angular intra prediction mode and an angular prediction mode, a combination of angular prediction modes, or a combination of non-angular prediction modes.

[0268] FIG. 27 is a diagram illustrating the angle formed between an angular intra prediction mode and a line perpendicular to the x-axis.

[0269] In the example shown in Figure 27, the angular prediction modes may exist between the bottom left diagonal and the top right diagonal. When described as an angle formed by the x-axis and the angular prediction modes, the angular prediction modes may exist between 45 degrees (bottom left diagonal) and -135 degrees (top right diagonal).

[0270] If the current block is non-square, a prediction sample is derived based on the intra prediction mode of the current block using the reference sample farthest from the prediction sample, without using the reference sample closest to the prediction sample that is located on an angle line following the intra prediction angle.

[0271] FIG. 28 is a diagram illustrating an example of obtaining predicted samples when the current block is non-square.

[0272] For example, in the example shown in Figure 28(a), assume that the current block has a non-square shape in which the width is greater than the height, and the intra prediction mode of the current block is an angular intra prediction mode with an angle ranging from 0 to 45 degrees. In this case, when deriving a predicted sample A in the neighborhood of the right column of the current block, a left reference sample L farther from the predicted sample among reference samples in the angular mode of the angle may be used instead of an upper reference sample T close to the predicted sample.

[0273] In another example, assume that the current block has a non-square shape with its height greater than its width, and that the intra prediction mode of the current block is an angular intra prediction mode with an angle ranging from -90 degrees to -135 degrees. In this case, when deriving a predicted sample A in a neighboring row below the current block, an upper reference sample T far from the predicted sample among reference samples in the angular mode of the angle may be used instead of a left reference sample L close to the predicted sample.

[0274] To solve the above problem, if the current block is non-square, the intra prediction mode of the current block may be replaced with a backward intra prediction mode. Therefore, for a non-square block, an angular prediction mode having an angle larger or smaller than the angle of the angular prediction mode shown in FIG. 24 may be used. Such an angular intra prediction mode may be defined as a wide-angle intra prediction mode. A wide-angle intra prediction mode refers to an angular intra prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0275] FIG. 29 is a diagram illustrating the wide-angle intra prediction mode.

[0276] In the example shown in FIG. 29, the intra prediction modes with indexes from −1 to −14 and the intra prediction modes with indexes from 67 to 80 represent wide-angle intra prediction modes.

[0277] Figure 29 shows 14 wide-angle intra-prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra-prediction modes (67 to 80) with angles less than -135 degrees, but a greater or lesser number of wide-angle intra-prediction modes can be defined.

[0278] When using a wide-angle intra prediction mode, the length of the upper reference sample is set to 2W+1, and the length of the left reference sample is set to 2H+1.

[0279] When using the wide-angle intra prediction mode, the sample A shown in FIG. 28(a) can be predicted using the reference sample T, and the sample A shown in FIG. 28(b) can be predicted using the reference sample L.

[0280] By adding the existing intra prediction modes and the N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 3 shows the intra direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined. [Table 3]

[0281] If the current block has a non-square shape and the intra prediction mode of the current block obtained in step S2202 falls within the conversion range, the intra prediction mode of the current block may be converted to a wide-angle intra prediction mode. The conversion range may be determined based on at least one of the size, shape, and ratio of the current block. Here, the ratio may represent the ratio between the width and height of the current block.

[0282] If the current block is non-square, with its width greater than its height, the transformation range may be from the intra prediction mode index in the upper right diagonal direction (e.g., 66) to (the intra prediction mode index in the upper right diagonal direction - N), where N may be determined based on the ratio of the current block. If the intra prediction mode of the current block falls within the transformation range, the intra prediction mode may be converted to a wide-angle intra prediction mode. The transformation may be performed by subtracting a predefined value from the intra prediction mode. The predefined value may be the total number of intra prediction modes other than the wide-angle intra prediction mode (e.g., 67).

[0283] According to the embodiment, the intra prediction modes between 66th and 53rd can be converted to wide-angle intra prediction modes between −1st and −14th, respectively.

[0284] If the current block is non-square, with its height greater than its width, the transformation range may be from the lower-left diagonal intra prediction mode index (e.g., 2) to (lower-left diagonal intra prediction mode index + M), where M may be determined based on the ratio of the current block. If the intra prediction mode of the current block falls within the transformation range, the intra prediction mode may be converted to a wide-angle intra prediction mode. The transformation may be performed by adding a predefined value to the intra prediction mode. The predefined value may be the total number of angular intra prediction modes other than the wide-angle intra prediction mode (e.g., 65).

[0285] According to the embodiment, the intra prediction modes between the 2nd and 15th can be converted to wide-angle intra prediction modes between the 67th and 80th, respectively.

[0286] Hereinafter, an intra prediction mode that falls within the transformation range will be referred to as a wide-angle intra replacement prediction mode.

[0287] The transform range may be determined based on the ratio of the current block. For example, Tables 4 and 5 show transform ranges when 35 and 67 intra prediction modes other than the wide-angle intra prediction mode are defined, respectively. [Table 4] [Table 5]

[0288] As shown in the examples of Tables 4 and 5, the number of wide-angle intra-replacement prediction modes that fall within the transform range may vary depending on the ratio of the current block.

[0289] Using a wide-angle intra prediction mode in addition to an existing intra prediction mode increases the resources required for encoding the wide-angle intra prediction mode, which may reduce encoding efficiency. Therefore, encoding efficiency can be improved by encoding a replacement intra prediction mode related to the wide-angle intra prediction mode instead of directly encoding the wide-angle intra prediction mode.

[0290] For example, when encoding the current block using the 67th wide-angle intra prediction mode, the intra prediction mode of the current block can be coded as the 67th wide-angle replacement intra prediction mode, number 2. When encoding the current block using the −1th wide-angle intra prediction mode, the intra prediction mode of the current block can be coded as the −1st wide-angle replacement intra prediction mode, number 66.

[0291] The decoder may perform decoding on the intra prediction mode of the current block and determine whether the decoded intra prediction mode is included in the conversion range. If the decoded intra prediction mode is a wide-angle replacement intra prediction mode, the decoder may convert the intra prediction mode to a wide-angle intra prediction mode.

[0292] Alternatively, when encoding the current block in wide-angle intra prediction mode, encoding can be performed directly in wide-angle intra prediction mode.

[0293] Encoding of intra prediction modes may be implemented based on the MPM list. A method for setting the MPM list will be described in detail below. In the following example, it is assumed that 10 wide-angle intra prediction modes (-1 to -10) with angles greater than 45 degrees and 10 wide-angle intra prediction modes (67 to 76) with angles less than -135 degrees are defined.

[0294] When encoding a neighboring block in wide-angle intra prediction mode, the MPM can be set based on a wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode. For example, when encoding a neighboring block in wide-angle intra prediction mode, the variable candIntraPredX (X is A or B) can be set as the wide-angle replacement intra prediction mode.

[0295] Alternatively, the method for deriving the MPM may be determined based on the shape of the current block. For example, if the current block has a square shape with the same width and height, candIntraPredX may be the wide-angle replacement intra prediction mode. On the other hand, if the current block has a non-square shape, candIntraPredX may be the wide-angle intra prediction mode.

[0296] Alternatively, whether candIntraPredX is the wide-angle intra prediction mode may be determined based on whether the wide-angle intra prediction mode of a neighboring block is applicable to the current block. For example, if the current block has a non-square shape in which the width is greater than the height, the wide-angle intra prediction mode whose index is greater than the index of the intra prediction mode in the upper right diagonal direction is directly designated as candIntraPredX. For a wide-angle intra prediction mode whose index is less than the index of the intra prediction mode in the lower left diagonal direction, the wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode is designated as candIntraPredX. On the other hand, if the current block has a non-square shape in which the height is greater than the width, the wide-angle intra prediction mode whose index is less than the index of the intra prediction mode in the lower left diagonal direction is directly designated as candIntraPredX. For a wide-angle intra prediction mode whose index is greater than the index of the intra prediction mode in the upper right diagonal direction, the wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode is designated as candIntraPredX.

[0297] In other words, it is possible to determine whether to derive the MPM using a wide-angle intra-prediction mode or a wide-angle replacement intra-prediction mode based on whether the shape of a neighboring block coded in a wide-angle intra-prediction mode is the same as or similar to the shape of the current block.

[0298] Alternatively, regardless of the shape of the current block, the wide-angle intra prediction mode of the neighboring block can be set to candIntraPredX.

[0299] In short, candIntraPredX can be the wide-angle intra prediction mode or the wide-angle replacement intra prediction mode of the neighboring block.

[0300] The MPM may be derived based on candIntraPredA and candIntraPredB. In this case, the MPM may be derived for an intra prediction mode similar to candIntraPredA or candIntraPredB. An intra prediction mode similar to candIntraPredA or candIntraPredB may be derived based on modulo operation and offset. In this case, the constant and offset used in the modulo operation may be determined differently depending on the shape of the current block.

[0301] Table 6 shows an example of deriving the MPM based on the shape of the current block. [Table 6]

[0302] Assume that candIntraPredA and candIntraPredB are the same and candIntraPredA is an angular intra prediction mode. If the current block has a square shape, an intra prediction mode similar to candIntraPredA can be obtained by modulo calculation based on a value obtained by subtracting 1 from the total number of angular intra prediction modes other than the wide-angle intra prediction mode. For example, if the number of angular intra prediction modes other than the wide-angle intra prediction mode is 65, the MPM can be derived by modulo calculation based on candIntraPredA and 64. On the other hand, if the current block has a non-square shape, an intra prediction mode similar to candIntraPredA can be obtained by modulo calculation based on a value obtained by subtracting 1 from the total number of angular intra prediction modes including the wide-angle intra prediction mode. For example, if the number of wide-angle intra prediction modes is 20, the MPM can be derived by modulo calculation based on candIntraPredA and 84.

[0303] Since the constant used in the modulo operation is set to differ depending on the shape of the current block, it is possible to determine whether the wide-angle intra prediction mode can be set as an angular intra prediction mode similar to candIntraPredA. For example, in modulo operation using 64, the wide-angle intra prediction mode may not be set as an angular intra prediction mode similar to candIntraPredA, but in modulo operation using 84, the wide-angle intra prediction mode can be set as an angular intra prediction mode similar to candIntraPredA.

[0304] Alternatively, if candIntraPredA and candIntraPredB are the same, the MPM can be derived taking into account the shape of the current block and whether candIntraPredA is a wide-angle intra prediction mode.

[0305] Table 7 shows an example of deriving the MPM based on the shape of the current block. [Table 7(1)]

[0306] [Table 7(2)]

[0307] Assume that candIntraPredA and candIntraPredB are the same.

[0308] If the current block is square and candIntraPredA is a wide-angle intra prediction mode, MPM may be the default mode. For example, MPM[0], MPM[1], and MPM[2] may be the planar mode, DC mode, and vertical mode, respectively.

[0309] If the current block has a square shape and candIntraPredA is an angular intra prediction mode other than the wide-angle intra prediction mode, MPM may be set to an angular intra prediction mode similar to candIntraPredA. For example, MPM[0] may be set to candIntraPredA, and MPM[1] and MPM[2] may be set to angular intra prediction modes similar to candIntraPredA.

[0310] If the current block is non-square and candIntraPredA is an angular intra prediction mode, MPM may be set to an angular intra prediction mode similar to candIntraPredA. For example, MPM[0] may be set to candIntraPredA, and MPM[1] and MPM[2] may be set to angular intra prediction modes similar to candIntrapredA.

[0311] An angular intra prediction mode similar to candIntraPredA may be derived using modulo arithmetic and an offset. In this case, the constant used in the modulo arithmetic may vary depending on the shape of the current block. The offset for deriving an angular intra prediction mode similar to candIntraPredA may be set differently depending on the shape of the current block. For example, if the current block has a non-square shape in which the width is greater than the height, an angular intra prediction mode similar to candIntraPredA may be derived by using an offset of 2. On the other hand, if the current block has a non-square shape in which the height is greater than the width, an angular intra prediction mode similar to candIntraPredA may be derived by using offsets of 2 and −8.

[0312] Alternatively, the MPM can be derived by considering whether candIntraPredX is the wide-angle intra prediction mode with the maximum index or the wide-angle intra prediction mode with the minimum index.

[0313] Table 8 shows an example of deriving the MPM by considering the wide-angle intra-prediction mode index. [Table 8(1)]

[0314] [Table 8(2)]

[0315] Assume that candIntraPredA and candIntraPredB are the same. For ease of explanation, a wide-angle intra prediction mode whose index value is less than the index value of the diagonal left-bottom intra prediction mode is referred to as a bottom wide-angle intra prediction mode, and a wide-angle intra prediction mode whose index value is greater than the index value of the diagonal right-top intra prediction mode is referred to as a right wide-angle intra prediction mode.

[0316] If candIntraPredA is a bottom wide-angle intra prediction mode, MPM may be set to an angle intra prediction mode similar to candIntraPredA. In this case, if candIntraPredA is a bottom wide-angle intra prediction mode with a minimum value, MPM may be set to a bottom wide-angle intra prediction mode with a predefined index value. Here, the predefined index may be an index with the maximum value among the indices of the bottom wide-angle intra prediction modes. For example, if candIntraPredA is -10, MPM[0], MPM[1], and MPM[2] may be -10, -1, and -9, respectively.

[0317] If candIntraPredA is a right-angle wide-angle intra prediction mode, MPM may be set to an angle intra prediction mode similar to candIntraPredA. In this case, if candIntraPredA is a right-angle wide-angle intra prediction mode with the maximum value, MPM may be set to a right-angle wide-angle intra prediction mode with a predefined index value. Here, the predefined index may be an index with the smallest value among the indexes of the right-angle wide-angle intra prediction modes. For example, if candIntraPredA is 77, MPM[0], MPM[1], and MPM[2] may be set to 77, 76, and 67, respectively.

[0318] Alternatively, if the index obtained by subtracting 1 from the index of candIntraPredA is less than the minimum value of the intra prediction mode indexes, or if the index obtained by adding 1 is greater than the maximum value, the MPM can be set as the default mode. Here, the default mode may include at least one of a planar mode, a DC mode, a vertical intra prediction mode, a horizontal intra prediction mode, and a diagonal intra prediction mode.

[0319] Alternatively, if the index obtained by subtracting 1 from the index of candIntraPredA is less than the smallest value among the indexes of the intra-prediction modes, or if the index obtained by adding 1 is greater than the largest value, the MPM can be an intra-prediction mode that is the opposite of candIntraPredA or an intra-prediction mode similar to the intra-prediction mode that is the opposite of candIntraPredA.

[0320] Alternatively, MPM candidates may be derived taking into account the shapes of the current block and the neighboring blocks. For example, a method for deriving an MPM when both the current block and the neighboring blocks are non-square may differ from a method for deriving an MPM when the current block is square but the neighboring blocks are non-square.

[0321] The MPMs in the MPM list may be rearranged (or reordered) taking into account at least one of the size of the current block, the shape of the current block, the size of the neighboring block, and the shape of the neighboring block. Here, rearrangement refers to reallocating the indexes assigned to each MPM. For example, a smaller index may be assigned to an MPM that has the same intra-prediction mode as a neighboring block having the same size or shape as the size or shape of the current block.

[0322] Assume that MPM[0] and MPM[1] are the intra-prediction mode candIntraPredA of the left neighboring block and the intra-prediction mode candIntraPredB of the upper neighboring block, respectively.

[0323] If the current block and the upper neighboring block have a non-square shape with width greater than height, the MPMs can be rearranged so that the intra prediction mode candIntraPredB of the upper neighboring block has a smaller index, i.e., candIntraPredB can be rearranged to MPM[0] and candIntraPredA can be rearranged to MPM[1].

[0324] Alternatively, if the current block and the upper neighboring block have a non-square shape with height greater than width, the MPMs can be rearranged to make the intra prediction mode candIntraPredB of the upper neighboring block have a smaller index, i.e., candIntraPredB can be rearranged to MPM[0] and candIntraPredA can be rearranged to MPM[1].

[0325] Alternatively, if the current block and the upper neighboring block are square shaped, the MPMs can be rearranged to make the intra-prediction mode candIntraPredB of the upper neighboring block have a smaller index, i.e., candIntraPredB can be rearranged to MPM[0] and candIntraPredA can be rearranged to MPM[1].

[0326] Instead of rearranging the MPM, when candIntraPredX is assigned to the MPM first, at least one of the size of the current block, the shape of the current block, the size of the neighboring block, and the shape of the neighboring block can be taken into consideration.

[0327] The MPMs may be rearranged based on the size or shape of the current block. For example, if the current block is a non-square shape with its width greater than its height, the MPMs may be rearranged in descending order. On the other hand, if the current block is a non-square shape with its height greater than its width, the MPMs may be rearranged in ascending order.

[0328] A residual image can be derived by subtracting the original image from the predicted image. In this case, when the residual image is converted to the frequency domain, removing high-frequency components from the frequency components does not significantly degrade the subjective image quality of the video. Therefore, converting the high-frequency components to smaller values ​​or setting them to zero has the effect of improving compression efficiency without causing obvious visual distortion. To reflect this characteristic, the residual image can be decomposed into two-dimensional frequency components by transforming the current block. The transformation can be performed using a transform technique such as a discrete cosine transform (DCT) or a discrete sine transform (DST).

[0329] The DCT decomposes (or transforms) the residual image into two-dimensional frequency components using a cosine transform. The DST decomposes (or transforms) the residual image into two-dimensional frequency components using a sine transform. As a result of the transformation of the residual image, the frequency components may be represented as base images. For example, if a DCT transform is performed on a block of size NxN, then N 2 basic pattern components can be obtained. By transforming, the size of each basic pattern component contained in a block of size N×N can be obtained. Depending on the transform technique used, the size of the basic pattern component may be called a DCT coefficient or a DST coefficient.

[0330] The DCT transform technique is primarily used to transform images with a high distribution of non-zero low-frequency components, while the DST transform technique is primarily used for images with a high distribution of high-frequency components.

[0331] The residual image can also be transformed using transform techniques other than DCT or DST.

[0332] Hereinafter, the process of converting a residual image into two-dimensional frequency components is referred to as two-dimensional image conversion. The size of the basic pattern components obtained by conversion may be referred to as a transform coefficient. For example, the transform coefficient may be a DCT coefficient or a DST coefficient. When a primary transform and a secondary transform (described later) are applied simultaneously, the transform coefficient may represent the size of the basic pattern components generated as a result of the secondary transform.

[0333] A transform technique can be determined for each block. The transform technique can be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, if the current block is coded in intra prediction mode and the size of the current block is less than NxN, DST can be performed using a transform technique. On the other hand, if the above conditions cannot be met, DCT can be used to perform the transform.

[0334] In the residual image, a 2D image transform may not be performed on some blocks. Not performing the 2D image transform may be called a transform skip. When a transform skip is applied, quantization may be applied to the residual values ​​on which the transform has not been performed.

[0335] After transforming a current block using a DCT or DST, the transformed current block can be transformed again. In this case, the DCT or DST-based transformation is defined as a primary transformation, and the process of transforming the block to which the primary transformation is applied is called a secondary transformation.

[0336] The main transform can be performed using any one of several candidate transform kernels, for example, the main transform can be performed using any one of the DCT2, DCT8, or DCT7.

[0337] Different transform kernels can be used for the horizontal and vertical directions. A signal can also be sent via the bitstream indicating a combination of horizontal and vertical transform kernels.

[0338] The execution units for the primary transform and the secondary transform are different. For example, the primary transform can be performed on an 8x8 block, and the secondary transform can be performed on a sub-block of the transformed 8x8 block that is 4x4 in size. In this case, the transform coefficients of the remaining area where the secondary transform is not performed can be set to 0.

[0339] Alternatively, a primary transform can be performed on a 4x4 block and a secondary transform can be performed on a region of size 8x8 that contains the transformed 4x4 block.

[0340] Through the bitstream, a signal can be sent indicating whether or not a secondary transformation is to be performed.

[0341] In the decoder, the inverse transform (second inverse transform) of the secondary transform can be performed, and the inverse transform (first inverse transform) of the primary transform can be performed on the result. As a result of performing the second inverse transform and the first inverse transform, a residual signal of the current block can be obtained.

[0342] Quantization is used to reduce the energy of a block, and the quantization process involves dividing the transform coefficients by a certain constant, which may be derived from a quantization parameter, which may be defined as a value between 1 and 63.

[0343] After transform and quantization are performed in the encoder, the decoder can obtain a residual block by inverse quantization and inverse transform, and can obtain a reconstructed block of the current block by adding the prediction block and the residual block.

[0344] Once a reconstructed block of the current block is obtained, information loss occurring in the quantization and encoding processes can be reduced by in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter may be referred to as a first reconstructed block, and the reconstructed block after applying the in-loop filter may be referred to as a second reconstructed block.

[0345] The second reconstructed block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block, in which case the SAO or ALF can be applied after applying the deblocking filter.

[0346] The deblocking filter is used to reduce image quality degradation (blocking artifacts) that occur at block boundaries when quantization is performed in block units. To apply the deblocking filter, the blocking strength (BS) between a first reconstructed block and an adjacent reconstructed block can be determined.

[0347] FIG. 30 is a flowchart showing the process of determining block strength.

[0348] In the example shown in Figure 30, P represents the first reconstructed block, and Q represents the adjacent reconstructed block, which may be adjacent to the left or above the current block.

[0349] In the example shown in Figure 30, the block strength can be determined by taking into account the predictive coding modes of P and Q, whether they contain non-zero transform coefficients, whether inter-prediction is performed using the same reference image, and whether the difference value of the motion vector is greater than or equal to a threshold.

[0350] Based on the block strength, it can be determined whether to apply a deblocking filter. For example, if the block strength is 0, there is a possibility that no filtering is performed.

[0351] SAO is used to reduce ringing artifacts that occur when quantization is performed in the frequency domain. SAO can be performed using an offset determined by adding or subtracting a pattern that takes into account the first reconstructed image. Methods for determining the offset include edge offset (EO) and band offset. EO refers to a method of determining the offset of a current sample based on the pattern of surrounding pixels. BO refers to a method of applying a common offset to a group of pixels with similar luminance values ​​in a region. Specifically, pixel luminance can be divided into 32 equal intervals, and pixels with similar luminance values ​​can be grouped together. For example, four adjacent bands out of the 32 bands can be grouped together, and the same offset can be applied to samples belonging to the four bands.

[0352] ALF is a method of generating a second reconstructed image by applying a filter of a predefined size / shape to a first reconstructed image or a reconstructed image that has been subjected to a deblocking filter. The following equation 21 shows an example of applying ALF.

number

[0353] Any one of the predefined filter candidates can be selected for each image, coding tree unit, coding block, prediction block, or transform block, and any one of the sizes or shapes of each filter candidate may be different.

[0354] FIG. 31 is a diagram illustrating predefined filter candidates.

[0355] In the example shown in FIG. 31, at least one of a 5x5, 7x7 and 9x9 diamond may be selected.

[0356] For the chromaticity components, only diamonds of size 5x5 can be used.

[0357] To perform real-time or low-latency encoding for high-resolution videos such as panoramic videos, 360-degree videos, or 4K / 8K UHD (Ultra High Definition), an image can be divided into multiple regions and the multiple regions can be encoded / decoded. For this purpose, an image can be divided into tiles (i.e., basic units that are coded / decoded in parallel), and these tiles can be processed in parallel.

[0358] Tiles can be restricted to rectangular shapes. When encoding / decoding a tile, data from other tiles is not used. The probability table for the context-adaptive binary arithmetic coding (CABAC) context is initialized on a tile-by-tile basis, and it can be set to not apply the loop filter at tile boundaries.

[0359] FIG. 32 shows an example of dividing an image into multiple tiles.

[0360] A tile contains at least one coding tree unit, and the boundaries of the tile overlap with the boundaries of the coding tree units.

[0361] In the example shown in Figure 32, an image may be divided into multiple tile sets. Information for dividing an image into multiple tile sets can be transmitted in a signal via a bitstream.

[0362] According to the image division type, the tiles may have the same size in all areas except the image border.

[0363] Alternatively, the image can be divided so that horizontally adjacent tiles have the same height, or the image can be divided so that vertically adjacent tiles have the same width.

[0364] When an image is divided using at least one of a vertical line and a horizontal line that intersects the image, each tile belongs to a different column and / or row. In the exemplary embodiment described below, the column to which a tile belongs is called a tile column, and the row to which a tile belongs is called a tile row.

[0365] Information for determining the shape of the division of an image into tiles can be transmitted as a signal via a bitstream. This information can be coded and transmitted as a signal using an image parameter set or a sequence parameter set. This information is used to determine the number of tiles in an image and may include information indicating the number of tile rows and information indicating the number of tile columns. For example, the syntax element num_tile_columns_minus1 indicates a value obtained by subtracting 1 from the number of tile columns, and the syntax element num_tile_rows_minus1 indicates a value obtained by subtracting 1 from the number of tile rows.

[0366] In the example shown in FIG. 32, the number of tile columns is four and the number of tile rows is three, so num_tile_columns_minus1 may be three and num_tile_rows_minus1 may be two.

[0367] When an image is divided into multiple tiles, information indicating the size of the tiles can be transmitted as a signal via the bitstream. For example, when an image is divided into multiple tile columns, information indicating the width of each tile column can be transmitted as a signal via the bitstream. When an image is divided into multiple tile rows, information indicating the height of each tile row can be transmitted as a signal via the bitstream. For example, for each tile column, a syntax element column_width_minus1 indicating the width of the tile column can be coded and transmitted as a signal. For each tile row, a syntax element row_height_minus1 indicating the height of the tile row can be coded and transmitted as a signal.

[0368] column_width_minus1 can indicate the value obtained by subtracting 1 from the width of the tile column, and row_height_minus1 can indicate the value obtained by subtracting 1 from the height of the tile row.

[0369] For the last tile column, coding for column_width_minus1 may be omitted, and for the last tile row, coding for row_height_minus1 may be omitted. The width of the last tile column and the height of the last row can be derived taking into account the size of the image.

[0370] The decoder can determine the size of the tile based on column_width_minus1 plus row_height_minus1.

[0371] Table 9 shows the syntax table for dividing an image into tiles. [Table 9]

[0372] Referring to Table 9, the signal can transmit a syntax element num_tile_columns_minus1 indicating the number of tile columns and a syntax element num_tile_rows_minus1 indicating the number of tile rows.

[0373] Subsequently, the signal can send a syntax element uniform_spacing_flag that indicates whether the image is divided into tiles of equal size. If uniform_spacing_flag is true, tiles in other areas besides the image border can be divided into equal sizes.

[0374] If uniform_spacing_flag is false, the signal can send the syntax element column_width_minus1 indicating the width of each tile column and the syntax element row_height_minus1 indicating the height of each tile row.

[0375] The syntax element loop_filter_across_tiles_enabled_flag indicates whether to allow the application of loop filters at tile boundaries.

[0376] A tile column having the smallest width among tile columns may be referred to as a smallest-width tile, and a tile row having the smallest height among tile rows may be referred to as a smallest-height tile. Information indicating the width of the smallest-width tile and information indicating the height of the smallest-height tile may be transmitted as a signal via the bitstream. For example, a syntax element min_column_width_minus1 indicates a value obtained by subtracting 1 from the width of the smallest-width tile, and a syntax element min_row_height_minus1 indicates a value obtained by subtracting 1 from the height of the smallest-height tile.

[0377] For each tile column, information indicating a difference value from the minimum tile width can be transmitted by a signal. For example, the syntax element diff_column_width indicates a width difference value between the current tile column and the minimum tile column. The width difference value may be expressed as a difference value in the number of coding tree unit columns. The decoder can derive the width of the current tile by adding the width of the minimum-width tile derived based on min_column_width_minus1 and the width difference value derived based on diff_column_width.

[0378] For each tile row, information indicating a difference value from the minimum tile height can be transmitted by a signal. For example, the syntax element diff_row_height indicates a height difference value between the current tile row and the minimum tile row. The height difference value may be expressed as a difference value in the number of coding tree unit rows. The decoder can derive the height of the current tile by adding the height of the minimum height tile derived based on min_row_height_minus1 and the height difference value derived based on diff_row_height.

[0379] Table 10 shows a syntax table containing information regarding size differences. [Table 10]

[0380] An image can be divided into tiles having different heights than horizontally adjacent tiles, or an image can be divided into tiles having different widths than vertically adjacent tiles. The image division method may be called a flexible tile division method, and tiles divided by the flexible tile division method may be called flexible tiles.

[0381] FIG. 33 is a diagram illustrating an image division mode using flexible tiling technology.

[0382] The search order of tiles generated by dividing an image may follow a predetermined scanning order, and an index can be assigned to each tile according to the predetermined scanning order.

[0383] The scanning order of the tiles may be any one of raster scan, diagonal scan, vertical scan, and horizontal scan. Figures 33(a) to 33(d) show examples of assigning indices to each tile based on raster scan, diagonal scan, vertical scan, and horizontal scan, respectively.

[0384] The next scan order may be determined based on the size or position of the current tile. For example, if the height of the current tile is different from the height of the tile adjacent to the right of the current tile (e.g., if the height of the right-adjacent tile is greater than the height of the current tile), the leftmost tile among the tiles along the same vertical line as the vertical line of the tile adjacent to the current tile below may be determined as the tile to be scanned after the current tile.

[0385] The scanning order of tiles can be determined in units of images or sequences.

[0386] Alternatively, the scan order of the tiles can be determined by considering the size of the first tile in the image. For example, if the width of the first tile is greater than the height, the scan order of the tiles can be horizontal. If the height of the first tile is greater than the width, the scan order of the tiles can be vertical. If the width of the first tile is the same as the height, the scan order of the tiles can be raster or diagonal.

[0387] A signal may be sent via the bitstream indicating the total number of tiles. For example, when applying flexible tiling technology, a signal may be sent that indicates the number of tiles in the picture minus 2, which is derived from the total number of tiles in the picture. A decoder can recognize the number of tiles currently included in the picture based on the number_of_tiles_in_picture_minus2.

[0388] Table 11 shows a syntax table that contains information regarding the number of tiles. [Table 11]

[0389] To reduce the number of bits required to encode the size of a tile, information indicating the size of a subtile can be coded and signaled. A subtile is a basic unit that constitutes a tile, and each tile may be configured to include at least one subtile. A subtile may include one or more coding tree units.

[0390] For example, the syntax element subtile_width_minus1 indicates a value obtained by subtracting 1 from the width of the subtile. The syntax element subtile_height_minus1 indicates a value obtained by subtracting 1 from the height of the subtile.

[0391] Information indicating whether tiles other than the first tile have the same size as the previous tile may be coded and signaled. For example, the syntax element use_previous_tile_size_flag indicates whether the size of the current tile is the same as the size of the previous tile. If use_previous_tile_size_flag is true, it indicates that the size of the current tile is the same as the size of the previous tile. If use_previous_tile_size_flag is false, information indicating the size of the current tile may be coded and signaled. For the first tile, coding of use_previous_tile_size_flag may be omitted, or the value of the flag may be set to false.

[0392] The information indicating the size of the tile may include a syntax element tile_width_minus1[i] indicating the width of the ith tile and a syntax element tile_height_minus1[i] indicating the height of the ith tile.

[0393] The information indicating the size of a tile may indicate a difference value from the size of a child tile. When the size information of a subtile is used, the number of bits required to encode the size of each tile can be reduced, thereby improving encoding / decoding efficiency. For example, the width (tileWidth) of the i-th tile can be derived based on the following equation 22, and the height (tileHeight) of the i-th tile can be derived based on the following equation 23.

number

number

[0394] Alternatively, encoding of the subtile size information may be omitted, and the size of the i-th tile may be directly encoded in the tile size information. The subtile size information may be selectively encoded. Information indicating whether the subtile size information has been encoded may be transmitted in a signal via the video parameter set, the sequence parameter set, or the picture parameter set.

[0395] Information about the tile size may be coded into a value indicating the number of coding tree units and transmitted as a signal. For example, column_width_minus1, min_column_width_minus1, subtile_width_minus1, tile_width_minus1, etc. may indicate the number of coding tree unit sequences included in the tile. Note that diff_column_width may indicate the difference between the number of coding tree unit sequences included in the minimum-width tile and the number of coding tree unit sequences included in the current tile.

[0396] Note that row_height_minus1, min_row_height_minus1, subtile_height_minus1, tile_height_minus1, etc. can indicate the number of coding tree unit rows included in a tile. Note that diff_row_height can indicate the difference between the number of coding tree unit rows included in the minimum height tile and the number of coding tree unit rows included in the current tile.

[0397] The decoder may determine the size of the tile based on the number of coding tree unit columns and / or the number of coding tree unit rows derived based on the syntax elements and the size of the coding tree unit. For example, the width of the i-th tile may be (tile_width_minus1[i]+1) * (width of the coding tree unit), and the height of the i-th tile may be (tile_height_minus1[i]+1) * (height of the coding tree unit).

[0398] At the same time, information indicating the size of the coding tree unit can be transmitted in a signal via the sequence parameter set or the picture parameter set.

[0399] In Table 11, the syntax element use_previous_tile_size_flag is used to indicate whether the size of the current tile is the same as the size of the previous tile. As another example, information indicating whether the width of the current tile is the same as the width of the previous tile or information indicating whether the height of the current tile is the same as the height of the previous tile can be coded and signaled.

[0400] Table 12 shows a syntax table that includes information indicating whether the width of the current tile is the same as the width of the previous tile. [Table 12]

[0401] The syntax element use_previous_tile_width_flag indicates whether the width of the current tile is the same as the width of the previous tile. If use_previous_tile_width_flag is true, the width of the current tile can be set to be equal to the width of the previous tile. In this case, encoding information indicating the width of the current tile may be omitted, and the width of the current tile can be derived from the width of the previous tile.

[0402] If use_previous_tile_width_flag is false, the signal can send information indicating the width of the current tile. For example, tile_width_minus1[i] can indicate the value obtained by subtracting 1 from the width of the ith tile.

[0403] The syntax element use_previous_tile_width_flag may be coded and signaled only if it is determined that the size of the current tile is different from the size of the previous tile (eg, if the value of use_previous_tile_size_flag is 0).

[0404] tile_width_minus1[i] may have a value obtained by subtracting 1 from the number of coding tree unit columns included in the i-th tile. The decoder can derive the number of coding tree unit columns belonging to the i-th tile by adding 1 to tile_width_minus1[i], and calculate the tile width by multiplying the derived value by the width of the coding tree unit.

[0405] Table 13 shows a syntax table that further includes information indicating whether the height of the current tile is the same as the height of the previous tile. [Table 13(1)]

[0406] [Table 13(2)]

[0407] The syntax element use_previous_tile_height_flag indicates whether the height of the current tile is the same as the height of the previous tile. If use_previous_tile_height_flag is true, the height of the current tile can be set to be equal to the height of the previous tile. In this case, coding of information indicating the height of the current tile can be omitted, and the height of the current tile can be derived from the height of the previous tile.

[0408] If use_previous_tile_height_flag is false, the signal can send information indicating the height of the current tile. For example, tile_height_minus1[i] can indicate the value obtained by subtracting 1 from the height of the ith tile.

[0409] The syntax element use_previous_tile_height_flag can be coded and signaled only when it is determined that the size of the current tile is different from the size of the previous tile (e.g., when the value of the current use_previous_tile_size_flag is 0). Note that the syntax element use_previous_tile_height_flag is signaled only when the value of use_previous_tile_width_flag is false.

[0410] Table 12 shows an example of using use_previous_tile_width_flag, and Table 13 shows an example of using use_previous_tile_width_flag and use_previous_tile_height_flag. Although not shown in the above tables, encoding of use_previous_tile_width_flag may be omitted, and only use_previous_tile_height_flag may be used.

[0411] Whether to use use_previous_tile_height_flag or use_previous_tile_size_flag can be determined based on at least one of the tile scan order, the width and height of the first tile, and the width and height of the previous tile. For example, if the tile scan order is vertical, use_previous_tile_height_flag can be used, and if the tile scan order is horizontal, use_previous_tile_width_flag can be used. Alternatively, if the first tile or the previous tile is a non-square shape with its width greater than its height, use_previous_tile_width_flag can be used. If the first tile or the previous tile is a non-square shape with its height greater than its width, use_previous_tile_height_flag can be used.

[0412] When the signal transmits the number of tiles contained in an image, the coding of information about the size of the last tile may be omitted.

[0413] Table 14 shows an example in which encoding of tile size information is omitted for the last tile. [Table 14]

[0414] If the size of a tile other than the last tile is specified, the excess area in the image can be used as the last tile.

[0415] Each coding tree unit can be assigned an identifier (hereinafter referred to as a tile ID) for identifying the tile to which the coding tree unit belongs.

[0416] FIG. 34 is a diagram illustrating an example of assigning tile IDs to each coding tree unit.

[0417] The same tile ID can be assigned to coding tree units that belong to the same tile. Specifically, the Nth Tile ID can be assigned to a coding tree unit that belongs to tile N.

[0418] To determine the tile ID assigned to each coding tree unit, variables x and y indicating the position of the coding tree unit in the image can be determined. Here, x represents the value obtained by dividing the x-axis coordinate of the position (x0, y0) of the top-left sample of the coding tree unit by the width of the coding tree unit, and y represents the value obtained by dividing the y-axis coordinate of the position (x0, y0) of the top-left sample of the coding tree unit by the height of the coding tree unit. Specifically, x and y can be derived using the following equations 24 and 25.

number

number

[0419] The process described below allows for the assignment of a tile ID to each coding tree unit.

[0420] i) Initialize tile ID The tile ID of each coding tree unit may be initialized to a value obtained by subtracting one from the number of tiles in the image. [Table 15]

[0421] ii) Deriving the tile ID [Table 16]

[0422] In the above embodiment, a flag indicating whether to apply a loop filter at a tile boundary is transmitted as a signal via an image parameter set. However, if no loop filter is set to be used at any tile boundary, problems such as a decrease in subjective image quality and a decrease in coding efficiency may occur.

[0423] Therefore, information indicating whether each tile allows the loop filter to be applied can be coded and signaled.

[0424] FIG. 35 is a diagram showing an example of selectively determining whether or not to apply a loop filter to each tile.

[0425] In the example shown in FIG. 35, for each tile, it can be decided whether to allow the application of a loop filter (eg, a deblocking filter, SAO and / or ALF) at the horizontal or vertical boundary.

[0426] Table 17 shows an example of encoding information indicating whether or not application of a loop filter is permitted for each tile. [Table 17]

[0427] In the example of Table 17, the syntax element loop_filter_across_tiles_flag[i] indicates whether a loop filter is allowed to be applied to the i-th tile. A value of 1 for loop_filter_across_tile_flag[i] indicates that a loop filter can be used on the horizontal and vertical boundaries of the tile with tile ID i. A value of 0 for loop_filter_across_tile_flag[i] indicates that a loop filter is not used on the horizontal and vertical boundaries of the tile with tile ID i.

[0428] Information indicating whether or not the application of a loop filter is permitted in each of the horizontal and vertical directions can be coded.

[0429] Table 18 shows an example of encoding information indicating whether or not to allow the application of a loop filter in the horizontal and vertical directions. [Table 18]

[0430] In the example of Table 18, the syntax element loop_filter_hor_across_tiles_flag[i] indicates whether to allow the loop filter to be applied at the position that intersects with the i-th tile in the horizontal direction. The syntax element loop_filter_ver_across_tiles_flag[i] indicates whether to allow the loop filter to be applied at the position that intersects with the i-th tile in the vertical direction.

[0431] A value of 1 in loop_filter_hor_across_tile_flag[i] indicates that a loop filter can be used on the horizontal boundary of the tile with tile ID i. A value of 0 in loop_filter_hor_across_tile_flag[i] indicates that a loop filter should not be used on the vertical boundary of the tile with tile ID i.

[0432] A value of 1 in loop_filter_ver_across_tile_flag[i] indicates that a loop filter can be used on the vertical boundary of the tile with tile ID i. A value of 0 in loop_filter_ver_across_tile_flag[i] indicates that a loop filter should not be used on the vertical boundary of the tile with tile ID i.

[0433] Information indicating whether a tile group including multiple tiles allows the application of a loop filter can be coded and transmitted in a signal, and based on the information, it can be determined whether multiple tiles included in the tile group allow the application of a loop filter.

[0434] To determine a tile group, at least one of the number of tiles belonging to the tile group, the size of the tile group, and image division information can be transmitted as a signal via the bitstream. Alternatively, a tile group can be a region of a size predefined in the encoder and decoder.

[0435] The coding of the information indicating whether to allow application of a loop filter may be omitted, and whether to allow application of a loop filter may be determined based on at least one of the number of coding tree units included in a tile, the width of the tile, and the height of the tile. For example, if the width of the tile is less than the reference value, application of a loop filter in the horizontal direction may be allowed, and if the height of the tile is less than the reference value, application of a loop filter in the vertical direction may be allowed.

[0436] When using a loop filter at a tile boundary, reconstructed data outside the tile can be generated based on the data contained in the tile. In this case, reconstructed video outside the tile can be obtained by padding or interpolating the data contained in the tile. The loop filter can then be applied using the reconstructed data outside the tile.

[0437] It is within the scope of the present invention to use embodiments described with emphasis on a decoding or encoding process in a decoding or encoding process. It is also within the scope of the present invention to modify the embodiments described in a given order in a different order than that described.

[0438] Although the embodiments have been described based on a series of steps or flowcharts, this does not limit the chronological order of the invention. Furthermore, steps may be executed simultaneously or in a different order, as needed. In the above embodiments, the components (e.g., units, modules, etc.) constituting the block diagrams may each be realized as hardware devices or software. Furthermore, multiple components may be combined and executed as a single hardware device or software. The embodiments may be implemented in the form of program instructions. The program instructions may be executed by various computer components and recorded on a computer-readable storage medium. The computer-readable storage medium may include program instructions, data files, data structures, etc., alone or in combination. Examples of computer-readable storage media include magnetic media such as hard disks, flexible disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. The hardware devices may be configured to operate as one or more software modules to perform the processes according to the present invention, or vice versa. [Industrial Applicability]

[0439] The present invention is applicable to electronic devices that encode / decode video.

Claims

1. 1. A video decoding method comprising: In response to the upper boundary of the current block contacting the upper boundary of the coding tree unit of the current block, a first affine seed vector (nv ０ ), and the second affine seed vector (nv １ ) and The first affine seed vector (nv ０ ) and the second affine seed vector (nv １ ), the third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ ) and The third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ deriving an affine vector of a sub-block within the current block by using a vector (i.e., a vector of a region smaller than the current block), wherein the sub-block is an area having a size smaller than the size of the current block; performing motion compensated prediction on the sub-block based on the affine vector.

2. a sample located to the right of the lower right sample of the lower right sub-block of the upper affine neighboring block is set as a reference sample of the control point at the lower right corner, and the lower right sub-block is set as an affine sub-block of the control point at the lower right corner; 10. The video decoding method of claim 1.

3. a sample located to the left of the lower-left sample of the lower-left sub-block of the upper affine neighboring block is set as a reference sample of the lower-left corner control point, and the lower-left sub-block is set as an affine sub-block of the lower-left corner control point; 3. A video decoding method according to claim 1 or 2.

4. 1. A video encoding method comprising: In response to the upper boundary of the current block contacting the upper boundary of the coding tree unit of the current block, a first affine seed vector (nv ０ ), and the second affine seed vector (nv １ ) and The first affine seed vector (nv ０ ) and the second affine seed vector (nv １ ), the third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ ) and The third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ deriving an affine vector of a sub-block within the current block by using a vector (i.e., a vector of a region smaller than the current block), wherein the sub-block is an area having a size smaller than the size of the current block; performing motion compensated prediction on the sub-block based on the affine vector.

5. a sample located to the right of the lower right sample of the lower right sub-block of the upper affine neighboring block is set as a reference sample of the control point at the lower right corner, and the lower right sub-block is set as an affine sub-block of the control point at the lower right corner; 5. The video encoding method of claim 4.

6. a sample located to the left of the lower-left sample of the lower-left sub-block of the upper affine neighboring block is set as a reference sample of the lower-left corner control point, and the lower-left sub-block is set as an affine sub-block of the lower-left corner control point; A video encoding method according to claim 4 or 5.

7. 1. A video decoder, the video decoder configured to perform the following operations, the operations comprising: In response to the upper boundary of the current block contacting the upper boundary of the coding tree unit of the current block, a first affine seed vector (nv ０ ), and the second affine seed vector (nv １ ) and The first affine seed vector (nv ０ ) and the second affine seed vector (nv １ ), the third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ ) and The third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ deriving an affine vector of a sub-block within the current block by using a vector (i.e., a vector of a region smaller than the current block), wherein the sub-block is an area having a size smaller than the size of the current block; performing motion compensated prediction on the sub-block based on the affine vector.

8. a sample located to the right of the lower right sample of the lower right sub-block of the upper affine neighboring block is set as a reference sample of the control point at the lower right corner, and the lower right sub-block is set as an affine sub-block of the control point at the lower right corner; 8. A video decoder according to claim 7.

9. a sample located to the left of the lower-left sample of the lower-left sub-block of the upper affine neighboring block is set as a reference sample of the lower-left corner control point, and the lower-left sub-block is set as an affine sub-block of the lower-left corner control point; 9. A video decoder according to claim 7 or 8.

10. 1. A video encoder, the video encoder configured to perform the following operations: In response to the upper boundary of the current block contacting the upper boundary of the coding tree unit of the current block, a first affine seed vector (nv ０ ), and the second affine seed vector (nv １ ) and The first affine seed vector (nv ０ ) and the second affine seed vector (nv １ ), the third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ ) and The third affine seed vector (sv ０ ) and the fourth affine seed vector (sv １ deriving an affine vector of a sub-block within the current block by using a vector (i.e., a vector of a region smaller than the current block), wherein the sub-block is an area having a size smaller than the size of the current block; performing motion compensated prediction on the sub-block based on the affine vector.

11. a sample located to the right of the lower right sample of the lower right sub-block of the upper affine neighboring block is set as a reference sample of the control point at the lower right corner, and the lower right sub-block is set as an affine sub-block of the control point at the lower right corner; The video encoder of claim 10.

12. a sample located to the left of the lower-left sample of the lower-left sub-block of the upper affine neighboring block is set as a reference sample of the lower-left corner control point, and the lower-left sub-block is set as an affine sub-block of the lower-left corner control point; A video encoder according to claim 10 or 11.

13. A method for transmitting a bitstream, comprising: A method for transmitting a bitstream, wherein the bitstream is generated by a video coding method according to any one of claims 4 to 5.

Images