Video decoding / encoding method, video decoder / encoder, and bit stream transmission method

Abstract

The invention relates to a video decoding / encoding method, a video decoding / encoding device, and a bit stream transmission method. The video decoding method includes: in response to contact of an upper boundary of a current block and an upper boundary of a coding tree unit of the current block, acquiring a first affine seed vector (nv0) of a lower left corner control point of an upper affine neighbor block of the current block and a second affine seed vector (nv1) of a lower right corner control point of the upper affine neighbor block; deriving a third affine seed vector (sv0) and a fourth affine seed vector (sv1) of the current block based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the upper affine neighbor block; deriving an affine vector for a sub-block in the current block using the third affine seed vector (sv0) and the fourth affine seed vector (sv1) of the current block, where the sub-block is a region having a size smaller than that of the current block; and performing motion compensation prediction on the sub-block based on the affine vector.

Description

[0001] Cross-references

[0002] This application is a divisional application of Chinese patent application No. 201980054695.2, which entered the Chinese national phase of PCT international patent application PCT / KR2019 / 012290 with an application date of September 20, 2019, and the invention name is “Video signal encoding / decoding method and device for said method”. The PCT international patent application claims the priority of application No. 10-2018-0114342, application No. 10-2018-0114343, and application No. 10-2018-0114344 filed in the Korean Intellectual Property Office on September 21, 2018. All of the above patent applications are incorporated into this application document in their entirety by reference. Technical Field

[0003] The present invention relates to a video decoding method, a video encoding method, a video decoder, a video encoder and a bit stream transmission method. Background Art

[0004] As display panels become larger and larger, there is a growing demand for video services with higher image quality. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, research is being actively conducted to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding) and was approved on January 25, 2013. Its compression performance is approximately twice that of H.264 / AVC. With the rapid development of high-definition video services, the performance of HEVC has gradually exposed its limitations. Summary of the Invention

[0005] Technical problems to be solved

[0006] An object of the present invention is to provide an inter-frame prediction method using an affine model when encoding / decoding a video signal and an apparatus for the inter-frame prediction method.

[0007] An object of the present invention is to provide a method for deriving an affine seed vector using a translational motion vector of a sub-block when encoding / decoding a video signal and an apparatus for performing the method.

[0008] Another object of the present invention is to provide a method for deriving an affine seed vector by transforming the distance between a neighboring block and a current block into a power series of 2 when encoding / decoding a video signal, and an apparatus for performing the method.

[0009] The technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and a person skilled in the art in the technical field to which the present invention belongs will clearly understand other technical problems not mentioned through the following description.

[0010] Technical Solution

[0011] The video signal decoding / encoding method according to the present invention includes: generating a merge candidate list for a current block; specifying any one of a plurality of merge candidates included in the merge candidate list; deriving a first affine seed vector and a second affine seed vector of the current block based on the first affine seed vector and the second affine seed vector of the specified merge candidate; deriving an affine vector of a subblock in the current block by using the first affine seed vector and the second affine seed vector of the current block; and performing motion compensated prediction on the subblock based on the affine vector. In this case, the subblock is an area smaller than the size of the current block. In addition, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on motion information of a neighboring block adjacent to the current block.

[0012] In the video signal decoding / encoding method according to the present invention, when the neighboring block is included in a coding tree unit different from the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on the motion vectors of the lower left sub-block and the lower right sub-block of the neighboring block.

[0013] In the video signal decoding / encoding method according to the present invention, the lower left subblock may include a lower left reference sample located at the lower left corner of the adjacent block, and the lower right subblock may include a lower right reference sample located at the lower right corner of the adjacent block.

[0014] In the video signal decoding / encoding method according to the present invention, a scaling factor can be used to derive the first affine seed vector and the second affine seed vector of the merge candidate based on a value obtained by performing a shift operation on the difference in motion vector between the lower left sub-block and the lower right sub-block, and the scaling factor can be derived based on a value obtained by adding the horizontal distance between the lower left reference sample and the lower right reference sample to an offset.

[0015] In the video signal decoding / encoding method according to the present invention, a scaling factor can be used to derive the first affine seed vector and the second affine seed vector of the merge candidate based on a value obtained by performing a shift operation on the difference in motion vector between the lower left sub-block and the lower right sub-block, and the scaling factor can be derived based on the distance between the adjacent sample to the right of the lower right reference sample and the lower left reference sample.

[0016] In the video signal decoding / encoding method according to the present invention, the merge candidate list includes a first merge candidate and a second merge candidate, the first merge candidate is derived based on the upper adjacent block of the first available block among the upper adjacent blocks determined to be located above the current block, and the second merge candidate is derived based on the left adjacent block of the first available block among the left adjacent blocks determined to be located on the left side of the current block.

[0017] In the video signal decoding / encoding method according to the present invention, when the adjacent block is included in the same coding tree unit as the coding tree unit of the current block, the first affine seed vector and the second affine seed vector of the merge candidate can be derived based on the first affine seed vector and the second affine seed vector of the adjacent block.

[0018] The features briefly summarized above are merely exemplary embodiments of the detailed description of the present invention to be described later and do not limit the scope of the present invention.

[0019] Effects of the Invention

[0020] According to the present invention, prediction efficiency is improved by using an inter-frame prediction method using an affine model.

[0021] According to the present invention, encoding efficiency is improved by deriving an affine seed vector using the translational motion vector of a sub-block.

[0022] According to the present invention, encoding efficiency is improved by transforming the distance between the adjacent block and the current block into a power series of 2 to derive an affine seed vector.

[0023] Effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those having ordinary skill in the technical field to which the present invention pertains through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0025] Figure 2 FIG. 4 is a block diagram of a video decoder according to an embodiment of the present invention.

[0026] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

[0027] Figure 4 is a diagram showing various types of partitioning of coding blocks.

[0028] Figure 5 is a diagram showing an example of division of a coding tree unit.

[0029] Figure 6 is a flowchart of an inter-frame prediction method according to an embodiment of the present invention.

[0030] Figure 7 is a diagram illustrating the nonlinear motion of an object.

[0031] Figure 8 FIG. 4 is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0032] Figure 9 is a diagram showing an example of an affine seed vector for each affine motion model.

[0033] Figure 10 is a diagram showing an example of affine vectors of sub-blocks under a 4-parameter motion model.

[0034] Figure 11 It shows the merge candidates that can be used to derive Graph of adjacent blocks.

[0035] Figure 12 is a diagram illustrating derivation of an affine seed vector of a current block based on affine seed vectors of affine neighboring blocks.

[0036] Figure 13 is a diagram showing an example of setting the motion vector of a subblock as an affine seed vector of an affine neighboring block.

[0037] Figures 14 to 16 is a diagram showing the positions of reference samples.

[0038] Figure 17 is a diagram illustrating an example of an affine merged vector derivation method to which the modification is applied.

[0039] Figure 18 is a diagram illustrating an example of deriving an affine seed vector of a current block based on non-affine neighboring blocks.

[0040] Figure 19 is a diagram showing an example of replacing a neighboring block with another neighboring block.

[0041] Figure 20 and Figure 21is a flowchart illustrating a motion compensated prediction method using multiple merging candidates.

[0042] Figure 22 is a flowchart of an intra-frame prediction method according to an embodiment of the present invention.

[0043] Figure 23 is a diagram showing reference samples included in each reference sample line.

[0044] Figure 24 is a diagram showing intra prediction modes.

[0045] Figure 25 and Figure 26 is a diagram showing an example of a one-dimensional arrangement in which reference samples are arranged in a row.

[0046] Figure 27 is a diagram showing an angle formed between an angular intra prediction mode and a straight line parallel to the x-axis.

[0047] Figure 28 is a diagram showing an example of obtaining prediction samples when the current block is non-square.

[0048] Figure 29 is a diagram showing the wide-angle intra prediction mode.

[0049] Figure 30 is a flow chart illustrating a process for determining block strength.

[0050] Figure 31 is a diagram showing predefined filter candidates.

[0051] Figure 32 An example of dividing an image into multiple tiles is shown.

[0052] Figure 33 is a diagram showing an image partitioning pattern according to the flexible tile technique.

[0053] Figure 34 is a diagram showing an example of assigning a tile ID to each coding tree unit.

[0054] Figure 35 is a diagram illustrating an example of selectively determining whether to apply a loop filter to each tile. DETAILED DESCRIPTION

[0055] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0056] Video encoding and decoding are performed on a block-by-block basis. For example, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, or reconstruction can be performed on a coding block, a transform block, or a prediction block.

[0057] Hereinafter, a block to be encoded / decoded is referred to as a “current block.” For example, depending on the current encoding / decoding process step, the current block may refer to a coding block, a transform block, or a prediction block.

[0058] In addition, the term "unit" used in this specification refers to a basic unit for performing a specific encoding / decoding process, and "block" can be understood to mean a sample array of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described later, encoding blocks and encoding units can be understood to have the same meaning.

[0059] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0060] Reference Figure 1 The video encoding apparatus 100 may include an image division unit 110 , prediction units 120 , 125 , a transformation unit 130 , a quantization unit 135 , a rearrangement unit 160 , an entropy encoding unit 165 , an inverse quantization unit 140 , an inverse transformation unit 145 , a filter unit 150 , and a memory 155 .

[0061] Figure 1 The components shown are shown separately to represent distinct characteristic functions of the video encoding device, and do not represent that each component is composed of separate hardware or a single software component. Specifically, for ease of explanation, each structural element is listed and included as a representative structural element, and at least two structural elements may be combined into a single structural element, or a single structural element may be divided into multiple structural elements to perform functions accordingly. Such embodiments integrating and separating the structural elements also fall within the scope of the present invention, as long as they do not depart from the essence of the present invention.

[0062] In addition, some structural elements are not essential structural elements for performing the essential functions of the present invention, but are optional structural elements used only to improve performance. The present invention can be implemented by only including components required to achieve the essence of the present invention (excluding structural elements used to improve performance), and a structure including only essential structural elements (excluding structural elements used to improve performance) also falls within the scope of the present invention.

[0063] The image division unit 110 may divide an input image into at least one processing unit. In this case, a processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image division unit 110 divides an image into a combination of coding units, prediction units, and transform units, and may select a combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the image.

[0064] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used. A video or largest coding unit can be used as a root to divide the coding unit into additional coding units. The coding unit can have as many child nodes as the number of divided coding units. Coding units that are no longer divided according to certain restrictions become leaf nodes. That is, when assuming that a coding unit can only be divided into squares, a coding unit can be divided into a maximum of four additional coding units.

[0065] Hereinafter, in the embodiments of the present invention, a coding unit may mean a unit that performs coding, and may also mean a unit that performs decoding.

[0066] The prediction units within one coding unit may be divided into at least one square or rectangle of the same size, or one prediction unit within one coding unit may be divided to have a shape and / or size different from another prediction unit.

[0067] When a prediction unit on which intra prediction is performed based on a coding unit is not a minimum coding unit, intra prediction may be performed without being divided into a plurality of prediction units N×N.

[0068] The prediction units 120 and 125 may include an inter-frame prediction unit 120 for performing inter-frame prediction and an intra-frame prediction unit 125 for performing intra-frame prediction. Whether inter-frame prediction or intra-frame prediction is used for a prediction unit can be determined, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) can be determined based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode can be determined by the prediction unit, and the prediction can be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. In addition, the prediction mode information, motion vector information, etc. used for prediction can be encoded together with the residual value in the entropy coding unit 165 and transmitted to the decoder. When using a specific coding mode, the original block can also be directly encoded and transmitted to the decoder without generating a prediction block through the prediction units 120 and 125.

[0069] The inter-frame prediction unit 120 can predict a prediction unit based on information about at least one of the previous or next images of the current image. In some cases, it can also predict a prediction unit based on information about a portion of an already encoded region within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0070] The reference image interpolation unit receives reference image information from the memory 155 and can generate pixel information for integer pixels or fractional pixels from the reference image. For luma pixels, to generate pixel information for fractional pixels in units of 1 / 4 pixels, an 8th-order DCT-based interpolation filter with different filter coefficients can be used. For chroma signals, to generate pixel information for fractional pixels in units of 1 / 8 pixels, a 4th-order DCT-based interpolation filter with different filter coefficients can be used.

[0071] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. The method for calculating the motion vector can use a full search-based Block Matching Algorithm (FBMA), a three-step search method (TSS), a new three-step search algorithm (NTS), and a variety of other methods. Based on the interpolated pixels, the motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel. In the motion prediction unit, the current prediction unit can be predicted by adopting different motion prediction methods. The motion prediction method can use a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, and a variety of other methods.

[0072] The intra-frame prediction unit 125 can generate a prediction unit based on reference pixel information surrounding the current block (the reference pixel information is pixel information within the current image). If the neighboring block of the current prediction unit is an inter-frame predicted block and the reference pixel is an inter-frame predicted pixel, the reference pixel included in the inter-frame predicted block can be used as the reference pixel information of the surrounding intra-frame predicted block. In other words, if the reference pixel is unavailable, at least one reference pixel among the available reference pixels can be used to replace the unavailable reference pixel information.

[0073] In intra-frame prediction, the prediction mode can include an angular prediction mode that uses reference pixel information according to the prediction direction and a non-angular mode that does not use directional information when performing prediction. The mode used to predict luma information and the mode used to predict chroma information can be different. To predict chroma information, the intra-frame prediction mode information used to predict luma information or the predicted luma signal information can be used.

[0074] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixel to the left, the pixel to the upper left, or the pixel above the prediction unit. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed based on the reference pixels of the transform unit. In addition, intra prediction using N×N partitioning can be applied only to the minimum coding unit.

[0075] After applying an adaptive intra-frame smoothing (AIS) filter to the reference pixels according to the prediction mode, the intra-frame prediction method can generate a prediction block. The type of the adaptive intra-frame smoothing filter applied to the reference pixels may be different. In order to perform the intra-frame prediction method, the intra-frame prediction mode of the current prediction unit can be predicted based on the intra-frame prediction modes of the prediction units located around the current prediction unit. In the case of using the mode information predicted from the surrounding prediction units to predict the prediction mode of the current prediction unit, if the intra-frame prediction mode of the current prediction unit is the same as that of the surrounding prediction units, predetermined flag information can be used to transmit information indicating that the prediction mode of the current prediction unit is the same as that of the surrounding prediction units. If the prediction mode of the current prediction unit is different from that of the surrounding prediction units, the prediction mode information of the current block can be encoded by performing entropy coding.

[0076] In addition, a residual block including residual information that is a difference between a prediction unit predicted based on the prediction unit generated in the prediction units 120 and 125 and an original block of the prediction unit may be generated. The generated residual block may be input to the transform unit 130.

[0077] The transform unit 130 may use a transform method such as discrete cosine transform (DCT), discrete sine transform (DST), or transform skip to transform the residual block, which includes residual information between the original block and the prediction unit generated by the prediction units 120 and 125. Whether to apply DCT, DST, or KLT to transform the residual block may be determined based on intra prediction mode information of the prediction unit used to generate the residual block.

[0078] The quantization unit 135 may quantize the value transformed into the frequency domain in the transformation unit 130. The quantization coefficient may be changed according to the importance of the block or image. The value calculated in the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0079] The rearrangement part 160 may perform rearrangement of coefficient values on the quantized residual value.

[0080] The rearrangement unit 160 can change the 2D block shape coefficients into a 1D vector form using a coefficient scanning method. For example, the rearrangement unit 160 can use a zigzag scanning method to scan the DC coefficients and even the coefficients in the high frequency domain and change them into a 1D vector form. Depending on the size of the transform unit and the intra-frame prediction mode, instead of zigzag scanning, vertical scanning that scans the 2D block shape coefficients along the column direction and horizontal scanning that scans the 2D block shape coefficients along the row direction can also be used. That is, the zigzag scanning, vertical scanning, and horizontal scanning method to be used can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0081] The entropy coding unit 165 may perform entropy coding based on the value calculated by the rearrangement unit 160. For example, the entropy coding may use various coding methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0082] The entropy coding unit 165 can encode various information such as residual value coefficient information and block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc. of the coding units from the rearrangement unit 160 and the prediction units 120 and 125.

[0083] The entropy encoding part 165 may perform entropy encoding on the coefficient values of the coding unit input from the rearrangement part 160 .

[0084] The inverse quantization unit 140 and the inverse transformation unit 145 inversely quantize the multiple values quantized by the quantization unit 135 and inversely transform the values transformed by the transformation unit 130. A reconstructed block can be generated by merging the residual values generated in the inverse quantization unit 140 and the inverse transformation unit 145 with the prediction units predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125.

[0085] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0086] A deblocking filter can remove block distortion in the reconstructed image caused by block boundaries. To determine whether to perform deblocking, the application of a deblocking filter to the current block can be determined based on the pixels included in several columns or rows of the block. When a deblocking filter is applied to a block, a strong filter or a weak filter can be applied depending on the desired deblocking filter strength. Furthermore, when using a deblocking filter, when performing vertical and horizontal filtering, the horizontal and vertical filtering can be processed simultaneously.

[0087] The offset correction unit may correct the offset between the deblocked image and the original image on a pixel-by-pixel basis. Offset correction may be performed on a specified image by dividing the pixels included in the image into a predetermined number of regions, determining the regions in which offset is to be performed, and applying the offset to the corresponding regions, or by taking edge information of each pixel into consideration.

[0088] Adaptive loop filtering (ALF) can be performed based on the value of comparing the filtered reconstructed image with the original image. After the pixels included in the image are divided into predetermined groups, a filter to be used for the corresponding group can be determined, and filtering can be performed differently for each group. Information related to whether adaptive loop filtering is applied can be transmitted through the luminance signal for each coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied can be different for each block. In addition, the same type (fixed type) of ALF can also be applied regardless of the characteristics of the block to which it is applied.

[0089] The memory 155 may store the reconstructed block or image calculated by the filter unit 150 , and may provide the stored reconstructed block or image to the prediction units 120 and 125 when performing inter-frame prediction.

[0090] Figure 2 FIG. 4 is a block diagram of a video decoder according to an embodiment of the present invention.

[0091] Reference Figure 2 The video decoder 200 may include an entropy decoding unit 210 , a rearrangement unit 215 , an inverse quantization unit 220 , an inverse transformation unit 225 , a prediction unit 230 , a prediction unit 235 , a filter unit 240 , and a memory 245 .

[0092] When a video bit stream is input from a video encoder, the input bit stream may be decoded in the reverse order of the video encoder.

[0093] The entropy decoding unit 210 may perform entropy decoding in a manner opposite to the steps performed by the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied, corresponding to the method performed by the video encoder.

[0094] The entropy decoding part 210 may decode information related to intra prediction and inter prediction performed by the encoder.

[0095] The rearrangement unit 215 may perform rearrangement based on the method used by the encoder to rearrange the bitstream entropy-decoded by the entropy decoding unit 210. The rearrangement may be performed by reconstructing a plurality of coefficients represented by a one-dimensional vector form into coefficients in a two-dimensional block shape. The rearrangement unit 215 may perform the rearrangement by receiving information related to coefficient scanning performed by the encoder and performing reverse scanning based on the scanning order performed by the corresponding encoder.

[0096] The inverse quantization part 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient value of the rearranged block.

[0097] The inverse transform unit 225 can perform inverse transforms (i.e., inverse DCT, inverse DST, and inverse KLT) on the quantization results performed by the video encoder. The inverse transform can be performed based on the transfer unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, the transform method (e.g., DCT, DST, KLT) can be selectively selected based on various information, such as the prediction method, the size of the current block, and the prediction direction.

[0098] The prediction units 230 and 235 may generate a prediction block based on information related to prediction block generation provided by the entropy decoding unit 210 and previously decoded block or image information provided by the memory 245 .

[0099] As described above, when intra prediction is performed in the same manner as in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixel to the left, the pixel to the upper left, and the pixel above the prediction unit. If the size of the prediction unit when performing intra prediction is different from the size of the transform unit, intra prediction is performed using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can also be applied only to the minimum coding unit.

[0100] The prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit receives various information input from the entropy decoding unit 210, including prediction unit information, prediction mode information for the intra-frame prediction method, and motion prediction-related information for the inter-frame prediction method. It classifies the prediction unit according to the current coding unit and determines whether the prediction unit is performing inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 may use the information required to perform inter-frame prediction for the current prediction unit, provided by the video encoder, and perform inter-frame prediction on the current prediction unit based on information included in at least one of the previous or next images of the current image to which the current prediction unit belongs. Alternatively, inter-frame prediction may also be performed based on information about a reconstructed portion of the current image to which the current prediction unit belongs.

[0101] In order to perform inter-frame prediction, it can be determined based on the coding unit whether the motion prediction method of the prediction unit included in the corresponding coding unit is the skip mode (Skip Mode), merge mode (Merge Mode), advanced motion vector prediction mode (AMVP Mode), or intra-frame block copy mode.

[0102] The intra prediction unit 235 can generate a prediction block based on pixel information within the current image. When the prediction unit is a prediction unit that has already performed intra prediction, intra prediction can be performed based on the intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra smoothing filter is a part that performs filtering on the reference pixels of the current block, and whether to apply the filter can be determined based on the prediction mode of the current prediction unit. The prediction mode of the prediction unit provided by the video encoder and the adaptive intra smoothing filter information can be used to perform adaptive intra smoothing on the reference pixels of the current block. If the prediction mode of the current block is a mode in which adaptive intra smoothing is not performed, the adaptive intra smoothing filter may not be applied.

[0103] If the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on pixel values interpolated from reference pixels, the reference pixel interpolation unit may generate integer-valued or fractional-valued reference pixels by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. If the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block through filtering.

[0104] The reconstructed block or image may be provided to the filter part 240. The filter part 240 may include a deblocking filter, an offset correction part, and an ALF.

[0105] Information related to whether a deblocking filter is applied to a corresponding block or image and information related to whether a strong filter or a weak filter is applied when applying the deblocking filter can be received from the video encoder. A deblocking filter of a video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0106] The offset correction section may perform offset correction on the reconstructed image based on the type and offset amount information of the offset correction applied to the image at the time of encoding.

[0107] ALF may be applied to a coding unit based on information related to whether ALF is applied, ALF coefficient information, etc. provided by an encoder. Such ALF information may be provided by being included in a specific parameter set.

[0108] The memory 245 stores the reconstructed image or block so that the image or block can be used as a reference image or reference block, and can provide the reconstructed image to the output section.

[0109] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

[0110] The largest coding block can be defined as a coding tree block. A picture can be divided into multiple coding tree units (CTUs). A coding tree unit is the largest coding unit and can also be called the largest coding unit (LCU). Figure 3 An example of dividing an image into a plurality of coding tree units is shown.

[0111] The size of the coding tree unit can be defined at a picture level or a sequence level. To this end, information indicating the size of the coding tree unit can be signaled through a picture parameter set or a sequence parameter set.

[0112] For example, the size of the coding tree unit of the entire image in the sequence may be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level may be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image may be set to 128×128, and the size of the coding tree unit in the second image may be set to 256×256.

[0113] The coding block can be generated by dividing the coding tree unit. The coding block represents the basic unit for performing the encoding / decoding process. For example, prediction or transformation can be performed according to different coding blocks, or the prediction coding mode can be determined according to different coding blocks. Among them, the prediction coding mode represents a method for generating a predicted image. For example, the prediction coding mode may include intra prediction (Intra Prediction), inter prediction (Inter Prediction), current picture referencing (CPR, or Intra Block Copy (IBC)) or combined prediction (Combined Prediction). For a coding block, at least one prediction coding mode among intra prediction, inter prediction, current image reference or combined prediction can be used to generate a prediction block related to the coding block.

[0114] Information indicating the prediction coding mode of the current block may be signaled via the bitstream. For example, the information may be a 1-bit flag indicating whether the prediction coding mode is intra mode or inter mode. The current image reference or combined prediction may be used only when the prediction coding mode of the current block is determined to be inter mode.

[0115] The current picture reference is used to set the current picture as the reference picture and obtain the prediction block for the current block from the encoded / decoded region within the current picture. The current picture refers to the picture that includes the current block. Information indicating whether the current picture reference is applied to the current block can be signaled via the bitstream. For example, this information can be a 1-bit flag. When the flag is true, the prediction coding mode for the current block can be determined as current picture reference; when the flag is false, the prediction mode for the current block can be determined as inter-frame prediction.

[0116] Alternatively, the prediction coding mode for the current block can be determined based on a reference image index. For example, when the reference image index points to the current image, the prediction coding mode for the current block can be determined as current image reference. When the reference image index points to another image other than the current image, the prediction coding mode for the current block can be determined as inter-frame prediction. In other words, current image reference is a prediction method that uses information from an already coded / decoded area in the current image, while inter-frame prediction is a prediction method that uses information from another already coded / decoded image.

[0117] Combined prediction refers to a coding mode that combines two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, when combined prediction is applied, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or current image reference, and a second prediction block can be generated based on the other. When generating the first and second prediction blocks, the final prediction block can be generated by averaging or weighted summing the first and second prediction blocks. Information indicating whether combined prediction is applied can be signaled via the bitstream. This information can be a 1-bit flag.

[0118] Figure 4 is a diagram showing various types of partitioning of coding blocks.

[0119] The coding block may be divided into multiple coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. The divided coding block may be further divided into multiple coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0120] Quadtree partitioning refers to a partitioning technique that divides the current block into 4 blocks. As a result of quadtree partitioning, the current block can be divided into 4 square partitions (refer to Figure 4 'SPLIT_QT' in .

[0121] Binary tree partitioning refers to a partitioning technique that divides the current block into two blocks. The process of partitioning the current block into two blocks vertically (i.e., using a vertical line that passes through the current block) is referred to as vertical binary tree partitioning, and the process of partitioning the current block into two blocks horizontally (i.e., using a horizontal line that passes through the current block) is referred to as horizontal binary tree partitioning. As a result of the binary tree partitioning, the current block can be divided into two non-square partitions. Figure 4 The "SPLIT_BT_VER" in the vertical direction represents the binary tree partition result, and Figure 4 The "SPLIT_BT_HOR" in the figure represents the result of horizontal binary tree partitioning.

[0122] Ternary tree partitioning refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks vertically (i.e., using two vertical lines that cross the current block) is referred to as vertical ternary tree partitioning, and the process of dividing the current block into three blocks horizontally (i.e., using two horizontal lines that cross the current block) is referred to as horizontal ternary tree partitioning. As a result of ternary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. Figure 4 The "SPLIT_TT_VER" in the vertical direction represents the ternary tree division result, and Figure 4 The "SPLIT_TT_HOR" in the figure indicates the horizontal ternary tree division result.

[0123] The number of times a coding tree unit is divided can be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Therefore, the maximum partitioning depth of a coding tree unit can be different for different sequences or pictures.

[0124] Alternatively, the maximum partition depth may be determined separately for each of the plurality of partitioning techniques. For example, the maximum partition depth allowed for quadtree partitioning may be different from the maximum partition depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0125] The encoder may signal information indicating at least one of a partition type or a partition depth of the current block through a bitstream, and the decoder may determine a partition type and a partition depth of the coding tree unit based on the information parsed from the bitstream.

[0126] Figure 5 is a diagram showing an example of division of a coding tree unit.

[0127] The process of partitioning a coding block using a partitioning technique such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning may be referred to as multi-tree partitioning.

[0128] The coding blocks generated by applying multi-tree partitioning to the coding blocks may be referred to as downstream coding blocks. When the partition depth of the coding block is k, the partition depths of the multiple downstream coding blocks are set to k+1.

[0129] On the other hand, for a coding block split at a depth of k+1, the coding block split at a depth of k may be referred to as an upstream coding block.

[0130] The partition type of the current coding block may be determined based on at least one of a partition type of an upstream coding block or a partition type of an adjacent coding block. The adjacent coding block is adjacent to the current coding block and may include at least one of an upper adjacent block, a left adjacent block, or a neighboring block adjacent to the upper left corner of the current coding block. The partition type may include at least one of whether to partition into a quadtree, whether to partition into a binary tree, a binary tree partition direction, whether to partition into a ternary tree, or a ternary tree partition direction.

[0131] In order to determine the partition type of the coding block, information indicating whether the coding block is partitioned can be signaled through the bitstream. The information is a 1-bit flag "split_cu_flag", which indicates that the coding block is partitioned using a multi-tree partitioning technique when true.

[0132] When "split_cu_flag" is true, information indicating whether the coding block is divided into quadtrees can be signaled through the bitstream. The information is a 1-bit flag "split_qt_flag". When the flag is true, the coding block can be divided into 4 blocks.

[0133] For example, in Figure 5 The example shown illustrates that when the coding tree unit is quadtree-partitioned, four coding blocks with a partition depth of 1 are generated. Furthermore, quadtree partitioning is further applied to the first and fourth coding blocks of the four coding blocks generated as a result of the quadtree partitioning. Ultimately, four coding blocks with a partition depth of 2 are generated.

[0134] In addition, a coding block with a split depth of 3 may be generated by applying quadtree partitioning again to the coding block with a split depth of 2.

[0135] When quadtree partitioning is not applied to a coding block, whether binary tree partitioning or ternary tree partitioning is performed on the coding block can be determined by considering at least one of the size of the coding block, whether the coding block is located at the image boundary, the maximum partition depth, or the partition type of the adjacent blocks. When it is determined that binary tree partitioning or ternary tree partitioning is performed on the coding block, information indicating the partitioning direction can be sent by signal through the bitstream. The information can be a 1-bit flag "mtt_split_cu_vertical_flag". Based on the flag, it can be determined whether the partitioning direction is vertical or horizontal. In addition, information indicating whether binary tree partitioning or ternary tree partitioning is applied to the coding block can be sent by signal through the bitstream. The information can be a 1-bit flag "mtt_split_cu_binary_flag". Based on the flag, it can be determined whether binary tree partitioning or ternary tree partitioning is applied to the coding block.

[0136] For example, in Figure 5The example shown illustrates that vertical binary tree partitioning is applied to a coding block with a partition depth of 1, vertical ternary tree partitioning is applied to the left coding block in the coding block generated as a result of the partitioning, and vertical binary tree partitioning is applied to the right coding block.

[0137] Inter-frame prediction refers to a predictive coding mode that uses information from the previous image to predict the current block. For example, a block in the previous image that is co-located with the current block (hereinafter referred to as a collocated block) can be set as the prediction block for the current block. Hereinafter, a prediction block generated based on a block co-located with the current block is referred to as a collocated prediction block.

[0138] On the other hand, if an object that existed in the previous image has moved to a different location in the current image, the object's motion can be used to effectively predict the current block. For example, if the direction and size of the object's movement are known by comparing the previous image and the current image, the object's motion information can be used to generate a prediction block (or predicted image) for the current block. Hereinafter, a prediction block generated using motion information may be referred to as a motion prediction block.

[0139] A residual block can be generated by subtracting the prediction block from the current block. In this case, when there is motion of an object, the energy of the residual block can be reduced by using the motion prediction block instead of the co-located prediction block, thereby improving the compression performance of the residual block.

[0140] As described above, the process of generating a prediction block using motion information may be referred to as motion compensated prediction. In most inter-frame predictions, a prediction block may be generated based on motion compensated prediction.

[0141] Motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weighted value index. A motion vector indicates the direction and size of movement of an object. The reference image index specifies a reference image for the current block from among multiple reference images included in a reference image list. The prediction direction refers to either unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). Depending on the prediction direction of the current block, at least one of the L0 motion information or the L1 motion information may be used. The bidirectional weighted value index specifies the weighting value applied to the L0 prediction block and the weighting value applied to the L1 prediction block.

[0142] Figure 6 is a flowchart of an inter-frame prediction method according to an embodiment of the present invention.

[0143] refer to Figure 6The inter-frame prediction method includes the following steps: determining an inter-frame prediction mode of a current block (S601); obtaining motion information of the current block according to the determined inter-frame prediction mode (S602); and performing motion compensation prediction on the current block based on the obtained motion information (S603).

[0144] Among them, the inter-frame prediction mode represents a variety of technologies for determining the motion information of the current block, and can include an inter-frame prediction mode using translation motion information and an inter-frame prediction mode using affine motion information. For example, the inter-frame prediction mode using translation motion information can include a merge mode and an advanced motion vector prediction mode, and the inter-frame prediction mode using affine motion information can include an affine merge mode and an affine motion vector prediction mode. Depending on the inter-frame prediction mode, the motion information of the current block can be determined based on neighboring blocks adjacent to the current block or information parsed from the bitstream.

[0145] The following describes in detail the inter-frame prediction method using affine motion information.

[0146] Figure 7 is a diagram illustrating the nonlinear motion of an object.

[0147] The motion of objects in a video may be nonlinear. Figure 7 In the example shown, nonlinear motion of an object may occur, such as zoom-in, zoom-out, rotation, or affine transformation. When nonlinear motion occurs, translational motion vectors cannot effectively represent the object's motion. Therefore, affine motion can be used instead of translational motion in areas where nonlinear motion occurs, thereby improving coding efficiency.

[0148] Figure 8 FIG. 4 is a flowchart illustrating an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0149] Whether to apply the affine motion-based inter-frame prediction technology to the current block can be determined based on information parsed from the bitstream. Specifically, whether to apply the affine motion-based inter-frame prediction technology to the current block can be determined based on at least one of a flag indicating whether the affine merge mode is applied to the current block or a flag indicating whether the affine motion vector prediction mode is applied to the current block.

[0150] When an inter-frame prediction technique based on affine motion is applied to a current block, an affine motion model of the current block may be determined (S801). The affine motion model may be determined as at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent affine motion, and the 4-parameter affine motion model uses 4 parameters to represent affine motion.

[0151] Equation 1 represents the case of affine motion using six parameters. Affine motion represents translational motion with respect to a predetermined area determined by an affine seed vector.

[0152] Equation 1

[0153] v x =ax-by+e

[0154] v y =cx+dy+J

[0155] While using six parameters to represent affine motion can express complex motion, the number of bits required to encode each parameter increases, reducing encoding efficiency. Therefore, affine motion can also be represented using four parameters. Equation 2 shows the case of using four parameters to represent affine motion.

[0156] Equation 2

[0157] v x =ax-by+e

[0158] v y =bx+ay+f

[0159] Information used to determine the affine motion model for the current block can be encoded and signaled via the bitstream. For example, the information can be a 1-bit flag, "affine_type_flag." A value of 0 indicates that a 4-parameter affine motion model is applied, while a value of 1 indicates that a 6-parameter affine motion model is applied. The flag can be encoded per slice, tile, or block (e.g., coding block or coding tree unit). When the flag is signaled at the slice level, the affine motion model determined at the slice level can be applied to all blocks in the slice.

[0160] Alternatively, the affine motion model of the current block may be determined based on the affine inter prediction mode of the current block. For example, when the affine merge mode is applied, the affine motion model of the current block may be determined as a 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information used to determine the affine motion model of the current block may be encoded and signaled via the bitstream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block may be determined based on a 1-bit flag "affine_type_flag."

[0161] Next, an affine seed vector for the current block may be derived (S802). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block may be derived. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block may be derived. The motion vectors at the control points may be referred to as affine seed vectors. The control points may include at least one of the upper left corner, upper right corner, or lower left corner of the current block.

[0162] Figure 9 is a diagram showing an example of an affine seed vector for each affine motion model.

[0163] In the 4-parameter affine motion model, an affine seed vector associated with two of the top left corner, top right corner, or bottom left corner can be derived. Figure 9 In the example shown in part (a), when the 4-parameter affine motion model is selected, the affine vector can be derived by using the affine seed vector sv0 associated with the upper left corner of the current block (e.g., the upper left sample (x0, y0)) and the affine seed vector sv1 associated with the upper right corner of the current block (e.g., the upper right sample (x1, y1)). The affine seed vector associated with the lower left corner can also be used instead of the affine seed vector associated with the upper left corner, or the affine seed vector associated with the lower left corner can also be used instead of the affine seed vector associated with the upper right corner.

[0164] In the 6-parameter affine motion model, affine seed vectors related to the upper left corner, upper right corner, and lower left corner can be derived. Figure 9 In the example shown in part (b), when the 6-parameter affine motion model is selected, the affine vector can be derived by using the affine seed vector sv0 associated with the upper left corner of the current block (e.g., the upper left sample (x0, y0)), the affine seed vector sv1 associated with the upper right corner of the current block (e.g., the upper right sample (x1, y1)), and the affine seed vector sv2 associated with the upper left corner of the current block (e.g., the upper left sample (x2, y2)).

[0165] In the embodiments described later, under the 4-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments described later using the first affine seed vector and the second affine seed vector, at least one of the first affine seed vector and the second affine seed vector can be replaced by the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0166] In addition, under the 6-parameter affine motion model, the affine seed vectors of the upper left control point, the upper right control point, and the lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiment using the first affine seed vector, the second affine seed vector, and the third affine seed vector described later, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with the affine seed vector of the lower right control point (the fourth affine seed vector).

[0167] Affine seed vectors may be used to derive affine vectors for different sub-blocks (S803). The affine vectors represent translation motion vectors derived based on the affine seed vectors. The affine vectors of the sub-blocks may be referred to as affine sub-block motion vectors or sub-block motion vectors.

[0168] Figure 10 is a diagram showing an example of affine vectors of sub-blocks under a 4-parameter motion model.

[0169] The affine vector of the sub-block can be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. For example, Equation 3 represents an example of deriving an affine sub-block vector.

[0170] Equation 3

[0171]

[0172] In Equation 3, (x, y) represents the position of the sub-block. The position of the sub-block represents the position of the reference sample included in the sub-block. The reference sample can be a sample located at the upper left corner of the sub-block, or a sample at the center of at least one of its x-axis or y-axis coordinates. (x0, y0) represents the position of the first control point, and (sv 0x , sv 0y ) represents the first affine seed vector. In addition, (x1, y1) represents the position of the second control point, and (sv 1x , sv 1y ) represents the second affine seed vector.

[0173] When the first control point and the second control point correspond to the upper left corner and the upper right corner of the current block, respectively, x1-x0 may be set to the same value as the width of the current block.

[0174] Afterwards, motion compensation prediction can be performed on each sub-block using the affine vector of each sub-block (S804). After performing motion compensation prediction, a prediction block related to each sub-block can be generated. The prediction block of the sub-block can be set as the prediction block of the current block.

[0175] The affine seed vector of the current block may be derived based on the affine seed vectors of neighboring blocks adjacent to the current block. When the inter prediction mode of the current block is affine merge mode, the affine seed vector of the merge candidate included in the merge candidate list may be determined as the affine seed vector of the current block. In addition, when the inter prediction mode of the current block is affine merge mode, the motion information including at least one of the reference image index, the specific direction prediction flag, or the bidirectional weighted value of the current block may be set to be the same as that of the merge candidate.

[0176] The merging candidate may be derived based on neighboring blocks of the current block. The neighboring blocks may include at least one of spatially neighboring blocks spatially adjacent to the current block and temporally neighboring blocks included in an image different from the current image.

[0177] Figure 11 is a diagram showing neighboring blocks that can be used to derive merge candidates.

[0178] Neighboring blocks of the current block may include at least one of a neighboring block (A) adjacent to the left of the current block, a neighboring block (B) adjacent to the top of the current block, a neighboring block (C) adjacent to the upper right corner of the current block, a neighboring block (D) adjacent to the lower left corner of the current block, or a neighboring block adjacent to the upper left corner of the current block. If the coordinates of the upper left sample of the current block are (x0, y0), then the left neighboring block A includes a sample at (x0-1, y0+H-1), and the upper neighboring block B includes a sample at (x0+W-1, y0-1). Where W and H represent the width and height of the current block, respectively. The upper right neighboring block C includes a sample at (x0+W, y0-1), and the lower left neighboring block D includes a sample at (x0-1, y0+H). The upper left neighboring block E includes a sample at (x0-1, y0-1).

[0179] When neighboring blocks are encoded in affine inter prediction mode, the affine seed vector of the merge candidate can be derived based on the affine seed vector of the corresponding neighboring block. Hereinafter, the neighboring blocks encoded in affine inter prediction mode will be referred to as affine neighboring blocks.

[0180] Merge candidates for the current block can be generated by searching neighboring blocks in a predefined scanning order. The scanning order can be predefined in the encoder and decoder. For example, neighboring blocks can be searched in the order A, B, C, D, and E. Alternatively, merge candidates can be derived sequentially from the searched affine neighboring blocks. Alternatively, the scanning order can be adaptively determined based on at least one of the size, shape, or affine motion model of the current block. That is, the scanning order will differ for blocks with different sizes, shapes, or at least one of the affine motion models.

[0181] Alternatively, blocks located above the current block are sequentially searched and merge candidates are derived from the first affine neighboring block found, and blocks located to the left of the current block are sequentially searched and merge candidates are derived from the first affine neighboring block found. The multiple neighboring blocks located above the current block include at least one of neighboring block E, neighboring block B, or neighboring block C, and the multiple blocks located to the left of the current block may include at least one of block A or block D. In this case, neighboring block E may also be classified as a block located to the left of the current block.

[0182] Although not shown, merge candidates can be derived from temporally neighboring blocks of the current block. Temporally neighboring blocks may include blocks co-located with or adjacent to the current block in the co-located image. Specifically, when the temporally neighboring blocks of the current block are encoded in affine inter prediction mode, merge candidates can be derived based on the affine seed vectors of the temporal merge candidates.

[0183] A merge candidate list including merge candidates may be generated, and an affine seed vector of one of the merge candidates included in the merge candidate list may be determined as an affine seed vector of the current block. To this end, index information identifying any one of the plurality of merge candidates may be encoded and transmitted through a bitstream.

[0184] As another example, multiple neighboring blocks may be searched in a scanning order, and the affine seed vector of the current block may be derived from the affine seed vector of the first found affine neighboring block.

[0185] As described above, in the affine merge mode, the affine seed vectors of the neighboring blocks may be used to derive the affine seed vector of the current block.

[0186] When the inter prediction mode of the current block is an affine motion vector prediction mode, the affine seed vectors of the motion vector prediction candidates included in the motion vector prediction candidate list can be determined as the affine seed vector prediction value of the current block. The affine seed vector of the current block can be derived by adding the affine seed vector prediction value to the affine seed vector difference.

[0187] Affine seed vector prediction candidates may be derived based on neighboring blocks of the current block. Specifically, multiple neighboring blocks located above the current block are searched in a predetermined scanning order, and a first affine seed vector prediction candidate may be derived from the first affine neighboring block found. Additionally, multiple neighboring blocks located to the left of the current block are searched in a predetermined scanning order, and a second affine seed vector prediction candidate may be derived from the first affine neighboring block found.

[0188] Information used to determine the affine seed vector difference value can be encoded and transmitted via a bitstream. The information can include magnitude information indicating the magnitude of the affine seed vector difference value and a sign signal indicating the sign of the affine seed vector difference value. The affine seed vector difference value associated with each control point can be set to be the same. Alternatively, the affine seed vector difference value can be set differently for each control point.

[0189] As described above, the affine seed vector of the merge candidate or the affine seed vector prediction candidate can be derived from the affine seed vector of the affine neighboring block, and the affine seed vector of the derived merge candidate or the affine seed vector prediction candidate can be used to derive the affine seed vector of the current block. Alternatively, after searching for multiple affine neighboring blocks in a predetermined scanning order, the affine seed vector of the current block can be derived from the affine seed vector of the affine neighboring block that is first found.

[0190] Hereinafter, a method for deriving an affine seed vector of a current block, a merge candidate, or an affine seed vector prediction candidate from an affine seed vector of an affine neighboring block will be described in detail. In the embodiments described later, deriving an affine seed vector of a current block can be understood as deriving an affine seed vector of a merge candidate or deriving an affine seed vector of an affine seed vector prediction candidate.

[0191] Figure 12 is a diagram illustrating derivation of an affine seed vector of a current block based on affine seed vectors of affine neighboring blocks.

[0192] If the affine neighboring block stores a first affine seed vector nv0 associated with the upper left control point and a second affine seed vector nv1 associated with the upper right control point, a third affine seed vector nv2 associated with the lower left control point of the affine neighboring block can be derived based on the first and second affine seed vectors. Equation 4 represents an example of deriving the third affine seed vector.

[0193] Equation 4

[0194]

[0195] In equation 4, (nv 0x , nv 0y ) represents the first affine seed vector nv0, (nv1x , nv 1y ) represents the second affine seed vector nv1, and (nv 2x , nv 2y ) represents the third affine seed vector nv2. In addition, (x n0 , x n0 ) represents the position of the first control point, (x n1 , x n1 ) represents the position of the second control point, and (x n2 , x n2 ) represents the position of the third control point.

[0196] Afterwards, the first affine seed vector, the second affine seed vector, and the third affine seed vector can be used to derive the affine seed vector of the current block. Equation 5 shows an example of deriving the first affine seed vector v0 of the current block, and Equation 6 shows an example of deriving the second affine seed vector v1 of the current block.

[0197] Equation 5

[0198]

[0199] Equation 6

[0200]

[0201] In Equation 5 and Equation 6, (v 0x , v 0y ) represents the first affine seed vector sv0 of the current block, and (v 1x , v 1y ) represents the second affine seed vector sv1 of the current block. In addition, (x0, y0) represents the position of the first control point, and (x1, y1) represents the position of the second control point. For example, the first control point represents the upper left corner of the current block, and the second control point represents the upper right corner of the current block.

[0202] In the example, it is described that three affine seed vectors related to affine neighboring blocks are used to derive multiple affine seed vectors of the current block. As another example, only two of the multiple affine seed vectors of the affine neighboring blocks can be used to derive the affine seed vector of the current block.

[0203] Alternatively, instead of using the first affine seed vector at the upper left corner, the second affine seed vector at the upper right corner, or the third affine seed vector at the lower left corner associated with the affine neighboring block, the fourth affine seed vector associated with the lower right corner may be used to derive multiple affine seed vectors for the current block.

[0204] In particular, when the upper boundary of the current block contacts the upper boundary of the coding tree unit and the affine seed vector of the upper control point (e.g., the upper left corner or the upper right corner) of the affine neighboring block adjacent to the upper part of the current block (hereinafter referred to as the upper affine neighboring block) is to be used, it is necessary to pre-store them in the memory, which may cause the problem of increasing the number of line buffers. Therefore, when the upper boundary of the current block contacts the upper boundary of the coding tree unit, it can be set to use the affine seed vector of the lower control point (e.g., the lower left corner or the lower right corner) for the upper affine neighboring block instead of the affine seed vector of the upper control point. For example, a third affine seed vector associated with the lower left corner of the upper affine neighboring block and a fourth affine seed vector associated with the lower right corner can be used to derive multiple affine seed vectors of the current block. In this case, the affine seed vector associated with the lower corner can be derived by copying the affine seed vector associated with the upper corner, or can be derived from the affine seed vectors associated with multiple upper corners. For example, the first affine seed vector, the second affine seed vector, or the third affine seed vector may be transformed / replaced with a fourth affine seed vector associated with the lower right corner.

[0205] Equation 7 and Equation 8 represent examples of deriving the first and second affine seed vectors of the current block using the third affine seed vector associated with the lower left control point and the fourth affine seed vector associated with the lower right control point of the adjacent affine vector.

[0206] Equation 7

[0207]

[0208] Equation 8

[0209]

[0210] In equations 7 and 8, (x n2 ,y n2 ) represents the coordinates of the lower left control point of the affine adjacent block, and (x n3 ,y n3 ) represents the coordinates of the lower right control point of the affine adjacent block. (x0, y0) represents the coordinates of the upper left control point of the current block, and (x1, y1) represents the coordinates of the upper right control point of the current block. (nv 2x , nv 2y ) represents the affine seed vector of the lower left control point of the affine neighboring block (ie, the third affine seed vector), and (nv 3x , nv 3y ) represents the affine seed vector of the lower right control point of the affine neighboring block (ie, the fourth affine seed vector). (v 0x , v 0y) represents the affine seed vector of the upper left control point of the current block (ie, the first affine seed vector), and (v 1x , v 1y ) represents the affine seed vector of the upper right control point of the current block (ie, the second affine seed vector).

[0211] The division operations included in Equation 7 and Equation 8 can also be changed into shift operations. The shift operation can be based on the width between the lower left control point and the lower right control point (ie, (x n3 -x n2 ))Exported value to execute.

[0212] In the above example, multiple affine seed vectors for the current block can be derived based on multiple affine seed vectors for the encoded / decoded affine neighboring blocks. To this end, the multiple affine seed vectors for the encoded / decoded affine neighboring blocks need to be stored in memory. However, since the multiple affine seed vectors for the affine neighboring blocks are stored in memory in addition to the multiple translation motion vectors (i.e., multiple affine vectors) for the multiple sub-blocks included in the affine neighboring blocks, there is a problem of increased memory usage. To eliminate this problem, the motion vectors of the sub-blocks adjacent to the control points of the affine neighboring blocks can be used to derive the affine seed vectors for the current block, thereby replacing the affine seed vectors of the affine neighboring blocks. In other words, the motion vectors of the sub-blocks adjacent to the control points of the affine neighboring blocks can be set as the affine seed vectors of the affine neighboring blocks. A sub-block is a block with a size / shape predefined in the encoder and decoder, and can be a block of the basic size / shape for storing motion vectors. For example, a sub-block can be a 4×4 square block. Alternatively, the motion vector of a specified sample position can be set as the affine seed vector of the affine neighboring block.

[0213] Figure 13 is a diagram showing an example of setting the motion vector of a subblock as an affine seed vector of an affine neighboring block.

[0214] The motion vector of the sub-block adjacent to the control point can be set as the affine seed vector of the corresponding control point. Figure 13 In the example shown, the motion vector (nv 4x , nv 4y ) is set as the affine seed vector of the lower left control point (nv 2x , nv 2y ), and the motion vector (nv 5x , nv 5y ) is set as the affine seed vector of the control point at the lower right corner (nv 3x , nv 3y). Among them, the lower left sub-block refers to the lower left control point (x n2 ,y n2 ) adjacent samples (e.g., (x n2 ,y n2 -1) position), and the lower right sub-block refers to the sub-block including the lower right control point (x n3 ,y n3 ) adjacent samples (e.g., (x n3 -1,y n3 When deriving the affine seed vector of the current block based on Equations 7 and 8, the third affine seed vector of the affine neighboring block may be replaced by the motion vector of the lower left sub-block, and the fourth affine seed vector may be replaced by the motion vector of the lower right sub-block.

[0215] Hereinafter, in embodiments described later, a sub-block used as an affine seed vector of an affine adjacent block is referred to as an affine sub-block.

[0216] According to one embodiment of the present invention, an affine sub-block can be determined based on samples at a specific position. For example, a sub-block including samples at a specific position can be set as an affine sub-block. Hereinafter, samples at specific positions are referred to as affine reference samples. Furthermore, the reference samples of the affine sub-block used to determine the lower left control point are referred to as lower left reference samples, and the reference samples of the affine sub-block used to determine the lower right control point are referred to as lower right reference samples.

[0217] The lower-left reference sample and the lower-right reference sample can be selected from a plurality of samples included in the affine adjacent block. For example, at least one of the upper-left sample, lower-left sample, upper-right sample, or lower-left sample of the lower-left sub-block can be set as the lower-left reference sample, and at least one of the upper-left sample, lower-left sample, upper-right sample, or lower-left sample of the lower-right sub-block can be set as the lower-right reference sample. Therefore, the motion vectors of the lower-left sub-block including the lower-left reference sample and the lower-right sub-block including the lower-right reference sample can be set as the affine seed vector associated with the lower-left control point and the affine seed vector associated with the lower-right control point, respectively.

[0218] As another example, at least one of the lower left reference sample or the lower right reference sample may be set to a sample located outside the affine neighboring block. Figures 14 to 16 Provide detailed explanation.

[0219] Figures 14 to 16 is a diagram showing the positions of reference samples.

[0220] like Figure 14 In the example shown in part (a), for the lower left control point, the upper left sample of the lower left sub-block can be set as the reference sample (x n4 ,yn4 ). Therefore, the reference sample (x n4 ,y n4 ) is set to the affine sub-block associated with the lower left control point.

[0221] For the lower right control point, the sample to the right of the upper right sample of the lower right sub-block can be set as the reference sample (x n5 ,y n5 ). Therefore, the reference sample (x n5 ,y n5 ) is set as an affine sub-block associated with the lower right control point.

[0222] Alternatively, as Figure 14 In the example shown in part (b), for the lower left control point, the sample to the left of the upper left sample of the lower left sub-block can be set as the reference sample (x n4 ,y n4 ). Therefore, the reference sample (x n4 ,y n4 ) is set as an affine sub-block associated with the lower left control point.

[0223] For the lower right control point, the upper right sample of the lower right sub-block can be set as the reference sample (x n5 ,y n5 ). Therefore, the reference sample (x n5 ,y n5 ) is set to the affine sub-block associated with the lower right control point.

[0224] Alternatively, as Figure 15 In the example shown in part (a), for the lower left control point, the lower left sample of the lower left sub-block can be set as the reference sample (x n4 ,y n4 ). Therefore, the reference sample (x n4 ,y n4 ) is set to the affine sub-block associated with the lower left control point.

[0225] For the lower right control point, the sample to the right of the lower right sample of the lower right sub-block can be set as the reference sample (x n5 ,y n5 ). Therefore, the reference sample (x n5 ,y n5 ) is set as an affine sub-block associated with the lower right control point.

[0226] Alternatively, as Figure 15In the example shown in part (b), for the lower left control point, the sample to the left of the lower left sample of the lower left sub-block can be set as the reference sample (x n4 ,y n4 ). Therefore, the reference sample (x n4 ,y n4 ) is set as an affine sub-block associated with the lower left control point.

[0227] For the lower right control point, the lower right sample of the lower right sub-block can be set as the reference sample (x n5 ,y n5 ). Therefore, the reference sample (x n5 ,y n5 ) is set to the affine sub-block associated with the lower right control point.

[0228] Alternatively, as Figure 16 In the example shown in part (a), for the lower left control point, the sample between the upper left sample and the lower left sample of the lower left sub-block (for example, the left middle sample) can be set as the reference sample (x n4 ,y n4 ). Therefore, the reference sample (x n4 ,y n4 ) is set to the affine sub-block associated with the lower left control point.

[0229] For the lower right control point, the sample to the right of the sample between the upper right sample and the lower right sample of the lower right sub-block (eg, the right middle sample) may be set as the reference sample (x n5 ,y n5 ). Therefore, the reference sample (x n5 ,y n5 ) is set as an affine sub-block associated with the lower right control point.

[0230] Alternatively, as Figure 16 In the example shown in part (b), for the lower left control point, the sample to the left of the sample between the upper left sample and the lower left sample of the lower left sub-block can be set as the reference sample (x n4 ,y n4 ). Therefore, the reference sample (x n4 ,y n4 ) is set as an affine sub-block associated with the lower left control point.

[0231] For the lower right control point, the sample between the upper right sample and the lower right sample of the lower right sub-block can be set as the reference sample (x n5 ,y n5). Therefore, the reference sample (x n5 ,y n5 ) is set to the affine sub-block associated with the lower right control point.

[0232] When multiple affine seed vectors of the current block are derived based on Equations 7 and 8, the third affine seed vector of the affine neighboring block may be replaced by the motion vector of the affine sub-block associated with the lower left control point, and the fourth affine seed vector may be replaced by the motion vector of the affine sub-block associated with the lower right control point. In addition, the position of the lower left control point may be replaced by the position of the lower left reference sample, and the position of the lower right control point may be replaced by the position of the lower right reference sample.

[0233] and Figures 14 to 16 Different from the above, the sub-block including the samples adjacent to the reference sample can also be set as the affine sub-block. Specifically, the samples outside the affine adjacent sub-block can be set as the reference sample, and the sub-block included in the affine adjacent block can be set as the affine sub-block. For example, Figure 14 In the example shown in part (a), the sample to the right of the upper right sample of the lower right sub-block can be set as the reference sample (x n5 ,y n5 ), and the lower right sub-block can be set to an affine sub-block associated with the lower right corner. Alternatively, Figure 14 In the example shown in part (b), the sample to the left of the upper left sample of the lower left sub-block can be set as the reference sample (x n4 ,y n4 ), and the lower left sub-block can be set to an affine sub-block related to the lower left corner.

[0234] Can be applied equally Figure 15 and Figure 16 That is, in Figure 15 (a) Partial or Figure 16 In the example shown in part (a), the lower right sample of the lower right sub-block or the sample to the right of the middle sample on the right can be set as the reference sample (x n5 ,y n5 ), and the lower right sub-block can be set to an affine sub-block associated with the lower right corner. Alternatively, Figure 15 (b) part or Figure 16 In the example shown in part (b), the lower left sample of the lower left sub-block or the sample to the left of the left middle sample can be set as the reference sample (x n4 ,y n4 ), and the lower left sub-block can be set to an affine sub-block related to the lower left corner.

[0235] In the above example, the affine seed vector of the affine adjacent block can be derived by using the motion vector of the affine sub-block. To this end, for the encoded / decoded block, the motion vector can be stored in units of sub-blocks.

[0236] As another example, after the affine neighboring block stores a minimum number of affine seed vectors, the stored plurality of affine seed vectors may be used to derive the motion vector of the affine sub-block.

[0237] Equation 9 and Equation 10 represent examples of deriving a motion vector of an affine sub-block using an affine seed vector of an affine neighboring block.

[0238] Equation 9

[0239]

[0240] Equation 10

[0241]

[0242] In Equation 9 and Equation 10, (nv 4x , nv 4y ) represents the motion vector of the affine sub-block associated with the lower left control point, and (nv 5x , nv 5y ) represents the motion vector of the affine sub-block associated with the lower right control point. Since the motion vector of the affine sub-block is set to be the same as the affine seed vector of the control point, (nv 4x , nv 4y ) can be represented by the affine seed vector (nv 2x , nv 2y ) instead, or (nv 5x , nv 5y ) can be represented by the affine seed vector (nv 3x , nv 3y )replace.

[0243] (x n4 ,y n4 ) represents the position of the reference sample of the lower left sub-block. Alternatively, instead of this position, the center position of the lower left sub-block or the position of the lower left control point can also be used. (x n5 ,y n5 ) represents the position of the reference sample of the lower right sub-block. Alternatively, instead of this position, the center position of the lower right sub-block or the position of the lower right control point can also be used.

[0244] Equations 9 and 10 may be applied when the current block does not contact the boundary of the coding tree unit. When the current block contacts the upper boundary of the coding tree unit, instead of using Equations 9 and 10, the translation motion vector of the affine sub-block determined based on the lower left reference sample may be set as the third affine seed vector, and the translation motion vector of the affine sub-block determined based on the lower right reference sample may be set as the fourth affine seed vector.

[0245] In Equations 7 and 8, (x n3 -x n2 ) represents the width between the lower left control point and the lower right control point. As mentioned above, x n3 The position x of the lower right reference sample can be n5 Instead, and x n2 The position x of the lower left reference sample can be n4 Instead. Below, (x n3 -x n2 ) or a value obtained by replacing the position of the aforementioned equation by the position of the reference sample (e.g., (x n5 -x n4 )) is defined as variable W seed , the variable is called the sub-seed vector width.

[0246] Depending on the position of the reference sample, it may happen that the width of the subseed vector is not a power of 2 (e.g., 2 n ) case. For example, when the lower left sample of the lower left sub-block is set as the lower left reference sample and the lower right sample of the lower right sub-block is set as the lower right reference sample, the width of the sub-seed vector is not a multiple of 2. As described above, when the sub-seed vector width is not a power series of 2, the sub-seed vector width can be transformed into a power series of 2. The transformation may include adding / subtracting an offset to the sub-seed vector width, or replacing the position of the reference sample with the position of a sample adjacent to the reference sample. For example, the transformed sub-seed vector width can be derived by adding 1 to the width between the lower left reference sample and the lower right reference sample. Alternatively, the width between the adjacent reference sample adjacent to the right side of the lower right reference sample and the lower left reference sample can be set as the transformed sub-seed vector width. Thereafter, the affine seed vector of the current block can be derived by substituting the transformed sub-seed vector width into Equations 7 and 8.

[0247] It is also possible to change the division operations included in Equation 7 and Equation 8 into shift operations. The shift operations may be performed based on values derived from the transformed sub-seed vector widths (ie, values expressed as a power series of 2).

[0248] When the reference sample used to determine the affine sub-block does not belong to the affine neighboring block, the affine seed vector of the affine neighboring block can be derived based on samples adjacent to the reference sample among the multiple samples included in the affine neighboring block. Specifically, the translation motion vector of the sub-block including samples adjacent to the reference sample in the affine neighboring block (hereinafter referred to as the adjacent reference sample) can be set as the affine seed vector of the affine neighboring block. As described above, the method of deriving the affine seed vector using the adjacent reference samples can be defined as a modified affine merge vector derivation method.

[0249] Figure 17 is a diagram illustrating an example of an affine merged vector derivation method to which the modification is applied.

[0250] When the lower right reference sample (x n5 ,y n5 ) does not belong to the affine neighboring block, the sample (x n5 -1,y n5 ) to derive the affine seed vector. Specifically, the adjacent reference samples (x n5 -1,y n5 ) is set to the affine seed vector of the lower right control point.

[0251] exist Figure 17 In the example shown, the sample adjacent to the right side of the upper right sample of the lower right sub-block is shown as the lower right reference sample. When the sample adjacent to the right side of the lower right sample of the lower right sub-block or the sample adjacent to the right side of the right middle sample of the lower right sub-block is set as the lower right reference sample, the affine seed vector can be derived based on the sample adjacent to the left side of the adjacent reference sample.

[0252] In addition, when the lower left reference sample does not belong to an affine neighboring block, an affine seed vector may also be derived based on samples adjacent to the right side of the lower left reference sample according to the described embodiment.

[0253] By setting the positions of the reference samples and the sub-blocks used to derive the affine seed vectors in different ways, the sub-seed vector widths can be set to a power series of 2.

[0254] A merge candidate, an affine seed vector prediction candidate, or an affine seed vector of the current block can be derived by using neighboring blocks around the current block that are not encoded in affine inter mode. Specifically, blocks that are not encoded in affine inter mode can be combined, and their combination can be set as a merge candidate or an affine seed vector prediction candidate. For example, a combination of at least one of the motion vector of any neighboring block adjacent to the upper left corner of the current block, the motion vector of any neighboring block adjacent to the upper right corner of the current block, and the motion vector of any neighboring block adjacent to the lower left corner of the current block can be set as a merge candidate or an affine seed vector prediction candidate. In this case, the motion vector of the neighboring block adjacent to the upper left corner, the motion vector of the neighboring block adjacent to the upper right corner, and the motion vector of the neighboring block adjacent to the lower left corner can be set as the first affine seed vector of the upper left control point, the second affine seed vector of the upper right control point, and the third affine seed vector of the lower left control point, respectively.

[0255] Alternatively, in the modified affine merge vector derivation method described above, a neighboring block that is not encoded in affine inter mode can be used to derive a merge candidate, an affine seed vector prediction candidate, or an affine seed vector for the current block. Hereinafter, a neighboring block that is not encoded in affine inter mode will be referred to as a non-affine neighboring block.

[0256] Figure 18 is a diagram illustrating an example of deriving an affine seed vector of a current block based on non-affine neighboring blocks.

[0257] exist Figure 18 In the example shown, it is assumed that all neighboring blocks adjacent to the current block are non-affine neighboring blocks.

[0258] When the affine seed vector of the current block is to be derived from a non-affine neighboring block A among neighboring blocks adjacent to the current block, the lower left reference sample and the lower right reference sample of A can be set. For example, the sample adjacent to the left of the lower left sample of block A can be set as the lower left reference sample, and the lower right sample of block A can be set as the lower right reference sample. Since the lower left reference sample is outside block A, the motion vector of the subblock including the sample adjacent to the right of the lower left reference sample can be set as the third affine seed vector of block A. In addition, the motion vector of the subblock including the lower right reference sample can be set as the fourth affine seed vector of block A. Thereafter, the first and second affine seed vectors of the current block can be derived from block A based on Equations 9 and 10.

[0259] The method of deriving an affine seed vector from a non-affine neighboring block can be used only when motion compensation prediction for a non-affine neighboring block is performed in units of sub-blocks. Here, the prediction technology for performing motion compensation prediction in units of sub-blocks may include at least one of STMVP, ATMVP, bidirectional optical flow (BIO), overlapped block motion compensation (OBMC), and decoder-side motion vector correction (DMVR).

[0260] In the above embodiment, it has been described that when the upper boundary of the current block contacts the boundary of the coding tree unit, the third affine seed vector of the lower left control point and the fourth affine seed vector of the lower right control point of the affine adjacent block located above the current block are used to derive the merge candidate, affine seed vector prediction candidate or affine seed vector of the current block.

[0261] As another example, if the upper boundary of the current block touches the boundary of the coding tree unit, and the neighboring block located above the current block belongs to a coding tree unit different from the coding tree unit of the current block, then instead of using the neighboring block, the neighboring block closest to the neighboring block among the blocks included in the coding tree unit to which the current block belongs is used to derive the merge candidate, affine seed vector prediction candidate or affine seed vector of the current block.

[0262] exist Figure 19 In the example shown, the current block touches the upper boundary of the coding tree unit, and blocks B, C, and E located above the current block belong to a coding tree unit different from that of the current block. Therefore, instead of using block E, block F adjacent to block E among these blocks included in the coding tree unit to which the current block belongs can be used to derive the affine seed vector of the current block.

[0263] For motion compensation prediction of the current block, affine seed vectors of multiple blocks can be used. For example, multiple merge candidates can be selected from a merge candidate list, and the affine seed vector or sub-block vector of the current block can be derived based on the affine seed vectors of the selected merge candidates. Encoding / decoding the current block using the affine seed vectors of multiple blocks can be referred to as a multi-affine merge encoding method.

[0264] Information indicating whether a multi-affine merge encoding method is applied to the current block may be encoded and transmitted through a bitstream. Alternatively, whether a multi-affine merge encoding method is applied to the current block may be determined based on at least one of the number of affine neighboring blocks among neighboring blocks adjacent to the current block, the number of merge candidates included in the merge candidate list, and the affine motion model of the current block.

[0265] Figure 20 and Figure 21 is a flowchart illustrating a motion compensated prediction method using multiple merging candidates.

[0266] Figure 20 A diagram showing an example of deriving an affine seed vector of a current block by using affine seed vectors of a plurality of merge candidates. Figure 21 is a diagram illustrating an example of deriving a motion vector for each subblock by using affine seed vectors of a plurality of merging candidates.

[0267] The affine seed vector of the current block may be generated based on a sum, difference, average, or weighted sum operation of the affine seed vectors of two merging candidates.

[0268] The following Equations 11 and 12 show examples of deriving the affine seed vector of the current block by adding the affine seed vectors of the merge candidates.

[0269] Equation 11

[0270] (sv 4x , sv 4y )=(sv 0x , sv 0y )+(sv 2x , sv 2y )

[0271] Equation 12

[0272] (sv 5x , sv 5y )=(sv 1x , sv 1y )+(sv 3x , sv 3y )

[0273] In equations 11 and 12, sv4 represents the first affine seed vector of the current block, sv0 represents the first affine seed vector of the first merge candidate, and sv2 represents the first affine seed vector of the second merge candidate. In addition, sv5 represents the second affine seed vector of the current block, sv1 represents the second affine seed vector of the first merge candidate, and sv3 represents the second affine seed vector of the second merge candidate.

[0274] In addition, the following Equations 13 and 14 show examples of deriving the affine seed vector of the current block by merging the weighted sum operation of the candidate affine seed vectors.

[0275] Equation 13

[0276] (sv 4x , sv 4y )=(sv 0x , sv 0y )+((sv 2x , sv 2y )>>1)

[0277] Equation 14

[0278] (sv 5x , sv 5y )=(sv 1x , sv 1y )+((sv 3x , sv 3y )>>1)

[0279] As another example, a first sub-block motion vector and a second sub-block motion vector may be generated for each sub-block in the current block based on the affine seed vector of the first merge candidate and the affine seed vector of the second merge candidate. Thereafter, a final sub-block motion vector may be generated based on a sum, difference, average, or weighted sum operation of the first sub-block motion vector and the second sub-block motion vector.

[0280] The following Equation 15 shows an example of deriving a final subblock motion vector by adding the first subblock motion vector and the second subblock motion vector.

[0281] Equation 15

[0282] (V 2x , V 2y )=(V 0x , V 0y )+(V 1x , V 1y )

[0283] In Equation 15, V0 represents a first subblock motion vector, V1 represents a second subblock motion vector, and V2 represents a final subblock motion vector.

[0284] In addition, the following Equation 16 shows an example of deriving a final sub-block motion vector through a weighted sum operation of the first sub-block motion vector and the second sub-block motion vector.

[0285] Equation 16

[0286] (V 2x , V 2y )=(V 0x , V 0y )+((V 1x , V 1y )>>1)

[0287] Intra-frame prediction uses reconstructed samples of the surrounding coded / decoded blocks to predict the current block. In this case, intra-frame prediction of the current block can use reconstructed samples before applying the in-loop filter.

[0288] Intra-frame prediction techniques include matrix-based intra-frame prediction and general intra-frame prediction that takes into account directionality with surrounding reconstructed samples. Information indicating the intra-frame prediction technique for the current block can be signaled via the bitstream. The information can be a 1-bit flag. Alternatively, the intra-frame prediction technique for the current block can be determined based on at least one of the position, size, shape, or intra-frame prediction techniques of neighboring blocks of the current block. For example, when the current block exists across an image boundary, the current block can be set to not apply matrix-based intra-frame prediction.

[0289] Matrix-based intra prediction is a method that obtains a prediction block for the current block by performing matrix multiplication between matrices stored in the encoder and decoder and reconstructed samples around the current block. Information specifying any one of multiple stored matrices can be signaled via the bitstream. The decoder can determine the matrix to use for intra prediction of the current block based on this information and the size of the current block.

[0290] General intra prediction is a method of obtaining a prediction block of a current block based on a non-angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in more detail with reference to the accompanying drawings.

[0291] Figure 22 is a flowchart of an intra-frame prediction method according to an embodiment of the present invention.

[0292] A reference sample line for the current block may be determined (S2201). A reference sample line refers to a set of reference samples included in a K-th line offset from above and / or to the left of the current block. Reference samples may be derived from reconstructed samples that have been encoded / decoded around the current block.

[0293] Index information identifying a reference sample line for a current block among a plurality of reference sample lines may be signaled via a bitstream. The plurality of reference sample lines may include at least one of a first line, a second line, a third line, or a fourth line above and / or to the left of the current block. Table 1 shows the index assigned to each reference sample line. In Table 1, it is assumed that the first, second, and fourth lines are used as reference sample line candidates.

[0294]

Table 1

[0295] index Reference sample line 0 First reference sample line 1 Second reference sample line 2 Fourth reference sample line

[0296] The reference sample line of the current block may also be determined based on at least one of the position, size, shape of the current block, or a prediction coding mode of a neighboring block. For example, when the current block is adjacent to a boundary of an image, tile, slice, or coding tree unit, the first reference sample line may be determined as the reference sample line of the current block.

[0297] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper reference sample and the left reference sample may be derived from reconstructed samples surrounding the current block. The reconstructed samples may be in a state before an in-loop filter is applied.

[0298] Figure 23 is a diagram showing reference samples included in each reference sample line.

[0299] According to the intra prediction mode of the current block, a prediction sample may be obtained using at least one of the reference samples belonging to a reference sample line.

[0300] Next, an intra prediction mode for the current block may be determined (S2202). For the intra prediction mode for the current block, at least one of a non-angular intra prediction mode or an angular intra prediction mode may be determined as the intra prediction mode for the current block. The non-angular intra prediction modes include planar and DC, and the angular intra prediction modes include 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

[0301] Figure 24 is a diagram showing intra prediction modes.

[0302] Figure 24 (a) shows 35 intra prediction modes, and Figure 24 (b) shows 67 intra prediction modes.

[0303] You can also define Figure 24 A greater or lesser number of intra prediction modes as shown.

[0304] The most probable mode (MPM) can be set based on the intra prediction mode of the neighboring blocks adjacent to the current block. The neighboring blocks may include a left neighboring block adjacent to the left side of the current block and an upper neighboring block adjacent to the top of the current block. When the coordinates of the upper left sample of the current block are (0, 0), the left neighboring block may include samples at (-1, 0), (-1, H-1), or (-1, (H-1) / 2). H represents the height of the current block. The upper neighboring block may include samples at (0, -1), (W-1, -1), or ((W-1) / 2, -1). W represents the width of the current block.

[0305] When encoding a neighboring block with general intra prediction, the MPM can be derived based on the intra prediction mode of the neighboring block. Specifically, the intra prediction mode of the left neighboring block can be set to the variable candIntraPredModeA, and the intra prediction mode of the upper neighboring block can be set to the variable candIntraPredModeB.

[0306] At this time, when the neighboring block is not available (for example, when the neighboring block has not been encoded / decoded or the position of the neighboring block deviates from the image boundary), when the neighboring block is encoded using matrix-based intra prediction, when the neighboring block is encoded using inter prediction, or when the neighboring block is included in a coding tree unit different from that of the current block, a variable candIntraPredModeX (where X is A or B) derived based on the intra prediction mode of the neighboring block can be set to a default mode. The default mode may include at least one of a planar mode, a DC mode, a vertical direction mode, or a horizontal direction mode.

[0307] Alternatively, when encoding adjacent blocks using matrix-based intra prediction, the intra prediction mode corresponding to the index value for specifying any one of the matrices may be set to candIntraPredModeX. To this end, a lookup table indicating a mapping relationship between the index value for specifying the matrix and the intra prediction mode may be pre-stored in the encoder and decoder.

[0308] The MPMs can be derived based on the variables candIntraPredModeA and candIntraPredModeB. The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs can be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs can be signaled via the bitstream. Alternatively, the number of MPMs can be determined based on at least one of the predictive coding mode of neighboring blocks, the size of the current block, or the shape of the current block.

[0309] In the embodiment described later, assuming that the number of MPMs is 3, these 3 MPMs will be referred to as MPM[0], MPM[1], and MPM[2]. When the number of MPMs is greater than 3, the MPMs may include the 3 MPMs described in the embodiment described later.

[0310] When candIntraPredA is the same as candIntraPredB and candIntraPredA is in planar mode or DC mode, MPM[0] and MPM[1] can be set to planar mode and DC mode, respectively. MPM[2] can be set to vertical intra prediction mode, horizontal intra prediction mode, or diagonal intra prediction mode. The diagonal intra prediction mode can be bottom-left diagonal intra prediction mode, top-left intra prediction mode, or top-right intra prediction mode.

[0311] When candIntraPredA is the same as candIntraPredB and candIntraPredA is an intra prediction mode, MPM[0] can be set to be the same as candIntraPredA. MPM[1] and MPM[2] can be set to an intra prediction mode similar to candIntraPredA. An intra prediction mode similar to candIntraPredA can be an intra prediction mode whose index difference from candIntraPredA is ±1 or ±2. An intra prediction mode similar to candIntraPredA can be derived using a modulo operation (%) and an offset.

[0312] When candIntraPredA and candIntraPredB are different, MPM[0] and candIntraPredA may be set to be the same, and MPM[1] and candIntraPredB may be set to be the same. In this case, when both candIntraPredA and candIntraPredB are non-angular intra prediction modes, MPM[2] may be set to a vertical intra prediction mode, a horizontal intra prediction mode, or a diagonal intra prediction mode. Alternatively, when at least one of candIntraPredA and candIntraPredB is an angular intra prediction mode, MPM[2] may be set to an intra prediction mode derived by adding or subtracting an offset value from the larger value of planar, DC, or candIntraPredA or candIntraPredB. The offset value may be 1 or 2.

[0313] An MPM list including multiple MPMs may be generated, and information indicating whether the MPM identical to the intra prediction mode of the current block is included in the MPM list may be signaled via the bitstream. The information is a 1-bit flag, which may be referred to as an MPM flag. When the MPM flag indicates that the MPM identical to the current block is included in the MPM list, index information identifying one of the MPMs may be signaled via the bitstream. The MPM specified by the index information may be set as the intra prediction mode of the current block. When the MPM flag indicates that the MPM identical to the current block is not included in the MPM list, residual mode information indicating any one of the residual intra prediction modes other than the MPM may be signaled via the bitstream. The residual mode information indicates an index value corresponding to the intra prediction mode of the current block when reassigning indices to the residual intra prediction modes other than the MPM. The decoder may arrange the MPMs in ascending order and determine the intra prediction mode of the current block by comparing the residual mode information with the MPM. For example, when the residual mode information is the same as or smaller than the MPM, the intra prediction mode of the current block may be derived by adding 1 to the residual mode information.

[0314] Instead of setting the default mode to MPM, information indicating whether the intra prediction mode of the current block is the default mode may be signaled through the bitstream. The information is a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be signaled only when the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode may include at least one of plane, DC, vertical direction mode, or horizontal direction mode. For example, when plane is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is plane. When the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

[0315] When multiple intra prediction modes are set as default modes, index information indicating any one of the default modes may be further signaled. The intra prediction mode of the current block may be set to the default mode indicated by the index information.

[0316] When the index of the reference sample line of the current block is not 0, the default mode is not used. Therefore, when the index of the reference sample line is not 0, the default mode flag is not signaled and the value of the default mode flag may be set to a predefined value (i.e., false).

[0317] If the intra prediction mode of the current block is determined, a prediction sample related to the current block may be obtained based on the determined intra prediction mode ( S2203 ).

[0318] When DC mode is selected, prediction samples associated with the current block may be generated based on an average value of reference samples. Specifically, the values of all samples in the prediction block may be generated based on the average value of the reference samples. The average value may be derived using at least one of an upper reference sample located above the current block and a left reference sample located to the left of the current block.

[0319] The number or range of reference samples used to derive the average value may vary depending on the shape of the current block. For example, when the current block is a non-square block with a width greater than its height, only the upper reference samples may be used to calculate the average value. On the other hand, when the current block is a non-square block with a width less than its height, only the left reference samples may be used to calculate the average value. In other words, when the width and height of the current block are different, only the reference samples adjacent to the longer length may be used to calculate the average value. Alternatively, whether to use only the upper reference samples or only the left reference samples to calculate the average value may be determined based on the ratio of the width and height of the current block.

[0320] When planar mode is selected, prediction samples can be obtained using horizontal prediction samples and vertical prediction samples. The horizontal prediction samples are obtained based on left and right reference samples located on the same horizontal line as the prediction samples, and the vertical prediction samples are obtained based on upper and lower reference samples located on the same vertical line as the prediction samples. The right reference samples can be generated by copying the reference samples adjacent to the upper right corner of the current block, and the lower reference samples can be generated by copying the reference samples adjacent to the lower left corner of the current block. The horizontal prediction samples can be obtained based on a weighted sum of the left and right reference samples, and the vertical prediction samples can be obtained based on a weighted sum of the upper and lower reference samples. In this case, the weight assigned to each reference sample can be determined based on the position of the prediction sample. The prediction sample can be obtained based on an average or weighted sum of the horizontal and vertical prediction samples. When performing the weighted sum, the weight assigned to the horizontal and vertical prediction samples can be determined based on the position of the prediction samples.

[0321] When the angular prediction mode is selected, a parameter representing the prediction direction (or prediction angle) of the selected angular prediction mode may be determined. Table 2 below shows the intra prediction parameter intraPredAng for each intra prediction mode.

[0322]

Table 2

[0323]

[0324] Table 2 shows the intra direction parameters of each intra prediction mode with an index of any one of 2 to 34 when 35 intra prediction modes are defined. When more than 33 angular intra prediction modes are defined, Table 2 is further subdivided to set the intra direction parameters of each angular intra prediction mode.

[0325] After aligning the upper reference sample and the left reference sample of the current block, a prediction sample can be obtained based on the value of the intra-frame direction parameter. In this case, when the value of the intra-frame direction parameter is negative, the left reference sample and the upper reference sample can be aligned.

[0326] Figure 25 and Figure 26 is a diagram showing an example of a one-dimensional arrangement in which reference samples are arranged in a row.

[0327] Figure 25 An example of a vertical one-dimensional array in which reference samples are arranged in the vertical direction is shown, and Figure 26 An example of a horizontal one-dimensional array in which reference samples are arranged in the horizontal direction is shown. The following description will be made under the assumption that 35 intra prediction modes are defined. Figure 25 and 26 Example of .

[0328] When the intra prediction mode index is any one of 11 to 18, a horizontal one-dimensional arrangement of the upper reference sample rotated counterclockwise can be applied, and when the intra prediction mode index is any one of 19 to 25, a vertical one-dimensional arrangement of the left reference sample rotated clockwise can be applied. When arranging the reference samples in a row, the intra prediction mode angle can be taken into account.

[0329] The reference sample determination parameter may be determined based on the intra direction parameter. The reference sample determination parameter may include a reference sample index for specifying a reference sample and a weight value parameter for determining a weight value applied to the reference sample.

[0330] The reference sample index iIdx and the weight value parameter ifact can be obtained by the following equations 17 and 18, respectively.

[0331] Equation 17

[0332] iIdx=(y+1)*P ang / 32

[0333] Equation 18

[0334] i fact =[(y+1)*P ang ]&31

[0335] In Equations 17 and 18, P ang Indicates the intra-frame direction parameter. The reference sample specified by the reference sample index iIdx is equivalent to an integer pixel (Integer pel).

[0336] In order to derive the prediction sample, more than one reference sample can be specified. Specifically, the position of the reference sample used to derive the prediction sample can be specified considering the slope of the prediction mode. For example, the reference sample index iIdx can be used to specify the reference sample used to derive the prediction sample.

[0337] In this case, when the slope of the intra-frame prediction mode is not represented by a single reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, when the slope of the intra-frame prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample can be obtained by interpolating the first reference sample and the second reference sample. In other words, when the angular line (AngularLine) following the intra-frame prediction angle does not pass through the reference sample located at an integer pixel, the prediction sample can be obtained by interpolating the reference samples adjacent to the left, right, or above and below the position where the angular line passes.

[0338] The following Equation 19 represents an example of obtaining a predicted sample based on a reference sample.

[0339] Equation 19

[0340] P(x, y) = ((32-i fact ) / 32)*Ref_1D(x+ildx+1)+(i fact / 32)*Ref_1D(x+iIdx+2)

[0341] In Equation 19, P represents a prediction sample, and Ref_1D represents any one of the reference samples arranged in one dimension. In this case, the position of the reference sample can be determined according to the position (x, y) of the prediction sample and the reference sample index iIdx.

[0342] When the slope of the intra prediction mode can be represented as a reference sample, the weighting parameter ifact can be set to 0. Therefore, Equation 19 can be simplified as shown in the following Equation 20.

[0343] Equation 20

[0344] P(x, y) = Ref_1D(x+iIdx+1)

[0345] Intra-frame prediction may also be performed on the current block based on multiple intra-frame prediction modes. For example, intra-frame prediction modes may be derived for different prediction samples, and prediction samples may be derived based on the intra-frame prediction modes assigned to the respective prediction samples.

[0346] Alternatively, intra-frame prediction modes may be derived for different regions, and intra-frame prediction may be performed on each region based on the intra-frame prediction mode assigned to each region. The region may include at least one sample. At least one of the size or shape of the region may be adaptively determined based on at least one of the size or shape of the current block, or the intra-frame prediction mode. Alternatively, at least one of the size or shape of the region may be predefined in the encoder and decoder, regardless of the size or shape of the current block.

[0347] Alternatively, intra-frame predictions may be performed based on multiple intra-frame predictions, and a final prediction sample may be derived based on an average operation or a weighted sum operation of the multiple prediction samples obtained through the multiple intra-frame predictions. For example, intra-frame predictions may be performed based on a first intra-frame prediction mode to obtain a first prediction sample, and intra-frame predictions may be performed based on a second intra-frame prediction mode to obtain a second prediction sample. Thereafter, the final prediction sample may be obtained based on an average operation or a weighted sum operation between the first prediction sample and the second prediction sample. In this case, the weight values assigned to the first prediction sample and the second prediction sample, respectively, may be determined taking into account at least one of whether the first intra-frame prediction mode is a non-angle / angle prediction mode, whether the second intra-frame prediction mode is a non-angle / angle prediction mode, or the intra-frame prediction mode of a neighboring block.

[0348] The multiple intra prediction modes may be a combination of a non-angular intra prediction mode and an angular prediction mode, a combination of angular prediction modes, or a combination of non-angular prediction modes.

[0349] Figure 27 is a diagram showing an angle formed between an angular intra prediction mode and a straight line parallel to the x-axis.

[0350] like Figure 27 In the example shown, the angular prediction mode can exist between the lower left diagonal direction and the upper right diagonal direction. When described as the angle formed by the x-axis and the angular prediction mode, the angular prediction mode can exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0351] If the current block is non-square in shape, prediction samples are derived using reference samples farther away from the prediction samples rather than reference samples closer to the prediction samples among reference samples located on an angle line following the intra prediction angle according to the intra prediction mode of the current block.

[0352] Figure 28 is a diagram showing an example of obtaining prediction samples when the current block is non-square.

[0353] For example, as in Figure 28 In the example shown in (a), it is assumed that the current block has a non-square shape with a width greater than a height, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between 0 and 45 degrees. In this case, when deriving a prediction sample A near the right column of the current block, a left reference sample L farther from the prediction sample among the reference samples in the angular mode is used instead of an upper reference sample T closer to the prediction sample.

[0354] As another example, in Figure 28In the example shown in (b), it is assumed that the current block has a non-square shape in which the height is greater than the width, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between -90 degrees and -135 degrees. In this case, when deriving a prediction sample A near the lower row of the current block, an upper reference sample T farther from the prediction sample among the reference samples in the angular mode is used instead of a left reference sample L closer to the prediction sample.

[0355] In order to solve the above problem, when the current block is non-square, the intra prediction mode of the current block can be replaced with the intra prediction mode in the opposite direction. Figure 24 The angle prediction mode shown is an angle prediction mode with a larger or smaller angle. This angular intra prediction mode can be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode refers to an angular intra prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0356] Figure 29 is a diagram showing the wide-angle intra prediction mode.

[0357] exist Figure 29 In the illustrated example, the intra prediction modes with indexes of -1 to -14 and the intra prediction modes with indexes of 67 to 80 represent the wide-angle intra prediction mode.

[0358] Despite Figure 29 , 14 wide-angle intra prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) with angles less than -135 degrees are shown, but a greater or lesser number of wide-angle intra prediction modes may be defined.

[0359] When the wide-angle intra prediction mode is used, the length of the upper reference sample is set to 2W+1, and the length of the left reference sample is set to 2H+1.

[0360] When using wide-angle intra prediction mode, the reference sample T can be used to predict Figure 28 (a) shows sample A, and can be predicted using reference sample L Figure 28 (b) Sample A shown.

[0361] By adding the existing intra prediction mode and N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 3 shows the intra direction parameters of the intra prediction mode when 20 wide-angle intra prediction modes are defined.

[0362]

Table 3

[0363] PredModeIntra -10 -9 -8 -7 -6 -5 -4 -3 -2 intraPredAngle 114 93 79 68 60 54 49 45 39 PredModeIntra -1 2 3 4 5 6 7 8 9 intraPredAngle 35 32 29 26 23 21 19 17 15 PredModeIntra 10 11 12 13 14 15 16 17 18 intraPredAngle 13 11 9 7 5 3 2 1 0 PredModeIntra 19 20 21 22 23 24 25 26 27 intraPredAngle -1 -2 -3 -5 -7 -9 -11 -13 -15 PredModeIntra 28 29 30 31 32 33 34 35 36 intraPredAngle -17 -19 -21 -23 -26 -29 -32 -29 -26 PredModeIntra 37 38 39 40 41 42 43 44 45 intraPredAngle -23 -21 -19 -17 -15 -13 -11 -9 -7 PredModeIntra 46 47 48 49 50 51 52 53 54 intraPredAngle -5 -3 -2 -1 0 1 2 3 5 PredModeIntra 55 56 57 58 59 60 61 62 63 intraPredAngle 7 9 11 13 15 17 19 21 23 PredModeIntra 64 65 66 67 68 69 70 71 72 intraPredAngle 26 29 32 35 39 45 49 54 60 PredModeIntra 73 74 75 76 intraPredAngle 68 79 93 114

[0364] When the current block is non-square and the intra prediction mode of the current block obtained in step S2202 falls within the transformation range, the intra prediction mode of the current block may be transformed into the wide-angle intra prediction mode. The transformation range may be determined based on at least one of the size, shape, or ratio of the current block. The ratio may represent the ratio between the width and height of the current block.

[0365] When the current block is non-square and has a width greater than a height, the transform range may be set from the intra prediction mode index in the upper right diagonal direction (e.g., 66) to (the index of the intra prediction mode in the upper right diagonal direction - N). N may be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transform range, the intra prediction mode may be transformed into the wide-angle intra prediction mode. The transform may be performed by subtracting a predefined value from the intra prediction mode, and the predefined value may be the total number of intra prediction modes excluding the wide-angle intra prediction mode (e.g., 67).

[0366] According to the embodiment, the intra prediction modes between the 66th and 53rd may be transformed into wide-angle intra prediction modes between the -1st and -14th, respectively.

[0367] When the current block is a non-square block with a height greater than a width, the transformation range may be set from the intra prediction mode index in the lower left diagonal direction (e.g., 2) to (the index of the intra prediction mode in the lower left diagonal direction + M). M may be determined based on the ratio of the current block. When the intra prediction mode of the current block falls within the transformation range, the intra prediction mode may be transformed into the wide-angle intra prediction mode. The transformation may be performed by adding a predefined value to the intra prediction mode, which may be the total number of angular intra prediction modes other than the wide-angle intra prediction mode (e.g., 65).

[0368] According to the embodiment, the intra prediction modes between the 2nd to the 15th may be transformed into the wide-angle intra prediction modes between the 67th to the 80th, respectively.

[0369] Hereinafter, the intra prediction mode falling within the transformation range is referred to as a wide-angle intra replacement prediction mode.

[0370] The transform range may be determined based on the ratio of the current block. For example, Table 4 and Table 5 each show a transform range when 35 intra prediction modes and 67 intra prediction modes other than the wide-angle intra prediction mode are defined.

[0371]

Table 4

[0372]

[0373]

[0374]

Table 5

[0375] condition Replace intra prediction mode W / H=2 Mode 2, 3, 4, 5, 6, 7 W / H>2 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H=1 none H / W=1 / 2 Modes 61, 62, 63, 64, 65, 66 H / W<1 / 2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0376] As shown in the examples of Table 4 and Table 5, the number of wide-angle intra replacement prediction modes falling within the transform range may differ according to the ratio of the current block.

[0377] As the Wide intra prediction mode is used in addition to the existing intra prediction modes, the resources required to encode the Wide intra prediction mode increase, which may reduce encoding efficiency. Therefore, instead of directly encoding the Wide intra prediction mode, an alternative intra prediction mode related to the Wide intra prediction mode is encoded, thereby improving encoding efficiency.

[0378] For example, when the 67th Wide-angle intra-frame prediction mode is used to encode the current block, the number 2, which is the 67th Wide-angle alternative intra-frame prediction mode, may be encoded as the intra-frame prediction mode of the current block. Furthermore, when the -1th Wide-angle intra-frame prediction mode is used to encode the current block, the number 66, which is the -1th Wide-angle alternative intra-frame prediction mode, may be encoded as the intra-frame prediction mode of the current block.

[0379] The decoder may decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode is included in the transformation range. When the decoded intra prediction mode is the wide-angle replacement intra prediction mode, the intra prediction mode may be transformed into the wide-angle intra prediction mode.

[0380] Alternatively, when the current block is encoded in the wide-angle intra prediction mode, the wide-angle intra prediction mode may also be directly encoded.

[0381] The encoding of the intra prediction mode can be implemented based on the MPM list. Hereinafter, the method of configuring the MPM list will be described in detail. In the embodiment described below, it is assumed that 10 wide-angle intra prediction modes (-1 to -10) with an angle greater than 45 degrees and 10 wide-angle intra prediction modes (67 to 76) with an angle less than -135 degrees are defined.

[0382] When a neighboring block is encoded in Wide intra prediction mode, the MPM can be set based on the Wide alternative intra prediction mode corresponding to the Wide intra prediction mode. For example, when a neighboring block is encoded in Wide intra prediction mode, the variable candIntraPredX (where X is A or B) can be set to the Wide alternative intra prediction mode.

[0383] Alternatively, the method for deriving the MPM can be determined based on the shape of the current block. For example, when the current block is a square with the same width and height, candIntraPredX can be set to the wide-angle alternative intra prediction mode. On the other hand, when the current block is non-square, candIntraPredX can be set to the wide-angle intra prediction mode.

[0384] Alternatively, whether to set candIntraPredX as the wide-angle intra prediction mode may be determined based on whether the wide-angle intra prediction mode of the neighboring block is applicable to the current block. For example, if the current block has a non-square shape with a width greater than a height, a wide-angle intra prediction mode with an index greater than the index of the intra prediction mode in the upper right diagonal direction is set as candIntraPredX as is, but for a wide-angle intra prediction mode with an index less than the index of the intra prediction mode in the lower left diagonal direction, a wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode is set as candIntraPredX. On the other hand, if the current block has a non-square shape with a height greater than a width, a wide-angle intra prediction mode with an index less than the index of the intra prediction mode in the lower left diagonal direction is set as candIntraPredX as is, but for a wide-angle intra prediction mode with an index greater than the index of the intra prediction mode in the upper right diagonal direction, a wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode is set as candIntraPredX.

[0385] That is, depending on whether the shape of the neighboring block encoded in the wide-angle intra prediction mode is the same as or similar to the shape of the current block, it can be determined whether to use the wide-angle intra prediction mode to derive the MPM or whether to use the wide-angle replacement intra prediction mode to derive the MPM.

[0386] Alternatively, the wide-angle intra prediction mode of the neighboring block may be set to candIntraPredX regardless of the shape of the current block.

[0387] In summary, candIntraPredX can be set to the wide-angle intra prediction mode or the wide-angle replacement intra prediction mode of the neighboring block.

[0388] The MPM can be derived based on candIntraPredA and candIntraPredB. In this case, the MPM can be derived in an intra prediction mode similar to candIntraPredA or candIntraPredB. The intra prediction mode similar to candIntraPredA or candIntraPredB can be derived based on a modulo operation and an offset. In this case, the constant and offset used for the modulo operation can be determined differently depending on the shape of the current block.

[0389] Table 6 shows an example of deriving an MPM according to the shape of the current block.

[0390]

Table 6

[0391]

[0392]

[0393] Assume that candIntraPredA and candIntraPredB are the same and candIntraPredA is an angular intra prediction mode. When the current block is square, an intra prediction mode similar to candIntraPredA can be obtained by a modulo operation based on a value obtained by subtracting 1 from the total number of angular intra prediction modes excluding the wide-angle intra prediction mode. For example, when the number of angular intra prediction modes excluding the wide-angle intra prediction mode is 65, the MPM can be derived based on a value obtained by a modulo operation based on candIntraPredA and 64. On the other hand, when the current block is non-square, an intra prediction mode similar to candIntraPredA can be obtained by a modulo operation based on a value obtained by subtracting 1 from the total number of angular intra prediction modes including the wide-angle intra prediction mode. For example, when the number of wide-angle intra prediction modes is 20, the MPM can be derived based on a value obtained by a modulo operation based on candIntrapredA and 84.

[0394] Since the constant used for the modulo operation is set differently depending on the shape of the current block, it is possible to determine whether the wide-angle intra prediction mode can be set to an angular intra prediction mode similar to candIntraPredA. For example, in a modulo operation using 64, the wide-angle intra prediction mode may not be set to an angular intra prediction mode similar to candIntraPredA, while in a modulo operation using 84, the wide-angle intra prediction mode can be set to an angular intra prediction mode similar to candIntraPredA.

[0395] Alternatively, when candIntraPredA and candIntraPredB are the same, the MPM may be derived taking into consideration the shape of the current block and whether candIntraPredA is a wide-angle intra prediction mode.

[0396] Table 7 shows an example of deriving an MPM according to the shape of the current block.

[0397]

Table 7

[0398]

[0399] Assume candIntraPredA and candIntraPredB are the same.

[0400] When the current block is square and candIntraPredA is in wide-angle intra prediction mode, MPM can be set to the default mode. For example, MPM[0], MPM[1], and MPM[2] can be set to planar mode, DC mode, and vertical intra prediction mode, respectively.

[0401] When the current block is square-shaped and candIntraPredA is an angular intra prediction mode other than the wide-angle intra prediction mode, the MPM may be set to an angular intra prediction mode similar to candIntraPredA. For example, MPM[0] may be set to candIntraPredA, and MPM[1] and MPM[2] may be set to angular intra prediction modes similar to candIntraPredA.

[0402] When the current block is non-square and candIntraPredA is an angular intra prediction mode, MPM may be set to an angular intra prediction mode similar to candIntraPredA. For example, MPM[0] may be set to candIntraPredA, and MPM[1] and MPM[2] may be set to angular intra prediction modes similar to candIntrapredA.

[0403] An angular intra prediction mode similar to candIntraPredA can be derived using a modulo operation and an offset. In this case, the constant used for the modulo operation may differ depending on the shape of the current block. In addition, the offset used to derive the angular intra prediction mode similar to candIntraPredA can be set differently depending on the shape of the current block. For example, when the current block is a non-square shape with a width greater than a height, an angular intra prediction mode similar to candIntraPredA can be derived by using an offset of 2. On the other hand, when the current block is a non-square shape with a height greater than a width, an angular intra prediction mode similar to candIntraPredA can be derived using offsets of 2 and -8.

[0404] Alternatively, the MPM may be derived by considering whether candIntraPredX is the wide-angle intra prediction mode with the largest index or the smallest index.

[0405] Table 8 shows an example of deriving the MPM by considering the wide-angle intra prediction mode index.

[0406]

Table 8

[0407]

[0408] Assume that candIntraPredA and candIntraPredB are the same. For ease of explanation, a wide-angle intra prediction mode whose index value is less than the index value of the intra prediction mode in the lower left diagonal direction is referred to as a downward wide-angle intra prediction mode, and a wide-angle intra prediction mode whose index value is greater than the index value of the intra prediction mode in the upper right diagonal direction is referred to as a rightward wide-angle intra prediction mode.

[0409] When candIntraPredA is a downward wide-angle intra prediction mode, MPM can be set to an angular intra prediction mode similar to candIntraPredA. In this case, when candIntraPredA is a downward wide-angle intra prediction mode with a minimum value, MPM can be set to a downward wide-angle intra prediction mode with a predefined index value. Here, the predefined index can be an index with a maximum value among the indices of the downward wide-angle intra prediction mode. For example, when candIntraPredA is -10, MPM[0], MPM[1], and MPM[2] can be set to -10, -1, and -9, respectively.

[0410] When candIntraPredA is a right wide-angle intra prediction mode, MPM can be set to an angular intra prediction mode similar to candIntraPredA. In this case, when candIntraPredA is a right wide-angle intra prediction mode with a maximum value, MPM can be set to a right wide-angle intra prediction mode with a predefined index value. Here, the predefined index can be an index with a minimum value among the indices of the right wide-angle intra prediction mode. For example, when candIntraPredA is 77, MPM[0], MPM[1], and MPM[2] can be set to 77, 76, and 67, respectively.

[0411] Alternatively, when the index obtained by subtracting 1 from the index of candIntraPredA is less than the minimum value of the intra prediction mode index or the index obtained by adding 1 is greater than the maximum value, the MPM may be set to a default mode. Here, the default mode may include at least one of a planar mode, a DC mode, a vertical intra prediction mode, a horizontal intra prediction mode, and a diagonal intra prediction mode.

[0412] Alternatively, when the index obtained by subtracting 1 from the index of candIntraPredA is less than the minimum value among the indices of the intra-frame prediction mode or the index obtained by adding 1 is greater than the maximum value, the MPM can be set to an intra-frame prediction mode opposite to candIntraPredA or to an intra-frame prediction mode similar to the intra-frame prediction mode opposite to candIntraPredA.

[0413] Alternatively, the shape of the current block and the shape of the neighboring blocks may be considered to derive the MPM candidate. For example, the method of deriving the MPM when both the current block and the neighboring blocks are non-square shapes may be different from the method of deriving the MPM when the current block is square but the neighboring blocks are non-square shapes.

[0414] The MPMs in the MPM list may be rearranged (or reordered) taking into account at least one of the size of the current block, the shape of the current block, the size of the neighboring blocks, and the shape of the neighboring blocks. Here, rearranging means reallocating the index assigned to each MPM. For example, a smaller index may be assigned to an MPM having the same intra prediction mode as a neighboring block having the same size or shape as the current block.

[0415] Assume that MPM[0] and MPM[1] are set to the intra prediction mode candIntraPredA of the left neighboring block and the intra prediction mode candIntraPredB of the above neighboring block, respectively.

[0416] When the current block and the upper adjacent block are non-square in shape, the MPMs may be rearranged so that the intra prediction mode candIntraPredB of the upper adjacent block has a smaller index. That is, candIntraPredB may be rearranged to MPM[0], and candIntraPredA may be rearranged to MPM[1].

[0417] Alternatively, when the current block and the upper adjacent block are non-square in shape, the MPMs may be rearranged so that the intra prediction mode candIntraPredB of the upper adjacent block has a smaller index. That is, candIntraPredB may be rearranged to MPM[0], and candIntraPredA may be rearranged to MPM[1].

[0418] Alternatively, when the current block and the upper adjacent block are square-shaped, the MPMs may be rearranged so that the intra prediction mode candIntraPredB of the upper adjacent block has a smaller index. That is, candIntraPredB may be rearranged to MPM[0], and candIntraPredA may be rearranged to MPM[1].

[0419] Instead of rearranging the MPMs, when initially allocating candIntraPredX to the MPMs, at least one of the size of the current block, the shape of the current block, the size of the neighboring blocks, and the shape of the neighboring blocks may be considered.

[0420] The MPMs can be rearranged based on the size or shape of the current block. For example, if the current block is a non-square shape with a width greater than a height, the MPMs can be rearranged in descending order. On the other hand, if the current block is a non-square shape with a height greater than a width, the MPMs can be rearranged in ascending order.

[0421] The derived residual image can be derived by subtracting the original image from the predicted image. In this case, when the residual image is changed to the frequency domain, even if the high-frequency components in the frequency components are removed, the subjective image quality of the video will not be significantly reduced. Therefore, if the value of the high-frequency component is converted to a smaller value or the value of the high-frequency component is set to 0, it has the effect of improving the compression efficiency without causing obvious visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into 2-dimensional frequency components. The transformation can be performed using transformation techniques such as discrete cosine transform (DCT) or discrete sine transform (DST).

[0422] DCT uses cosine transform to decompose (or transform) the residual image into 2-dimensional frequency components, while DST uses sine transform to decompose (or transform) the residual image into 2-dimensional frequency components. As a result of the transformation of the residual image, the frequency components can be represented as a basic image. For example, when DCT transform is performed on a block with an N×N size, N 2 The size of each basic pattern component included in the block of size N×N can be obtained by transformation. Depending on the transformation technology used, the size of the basic pattern component can be called a DCT coefficient or a DST coefficient.

[0423] The DCT transform technique is mainly used to transform images with a large number of non-zero low-frequency components. The DST transform technique is mainly used to transform images with a large number of high-frequency components.

[0424] The residual image may also be transformed using transformation techniques other than DCT or DST.

[0425] Hereinafter, the process of transforming the residual image into two-dimensional frequency components is referred to as two-dimensional image transformation. Furthermore, the magnitudes of the basic pattern components obtained through the transformation are referred to as transform coefficients. For example, transform coefficients may refer to DCT coefficients or DST coefficients. When both the primary transform and the secondary transform (described later) are applied simultaneously, the transform coefficients may represent the magnitudes of the basic pattern components generated by the secondary transform.

[0426] The transform technique can be determined on a block-by-block basis. The transform technique can be determined based on at least one of the prediction coding mode of the current block, the size of the current block, or the shape of the current block. For example, when the current block is encoded in intra-frame prediction mode and the size of the current block is less than N×N, the transform technique DST can be used to perform the transform. On the other hand, when the above conditions cannot be met, the transform technique DCT can be used to perform the transform.

[0427] In the residual image, it is also possible to not perform a 2D image transform on some blocks. Not performing a 2D image transform is called a transform skip. When transform skipping is applied, quantization can be applied to the residual values that have not been transformed.

[0428] After the current block is transformed using DCT or DST, the transformed current block can be transformed again. In this case, the transformation based on DCT or DST can be defined as the main transformation, and the process of transforming the block to which the main transformation is applied again is called the secondary transformation.

[0429] The main transform may be performed using any one of a plurality of transform core candidates, for example, DCT2, DCT8, or DCT7.

[0430] Different transform kernels may be used for the horizontal direction and the vertical direction. Information indicating the combination of the transform kernel for the horizontal direction and the transform kernel for the vertical direction may also be signaled via the bitstream.

[0431] The primary and secondary transforms may be performed on different units. For example, the primary transform may be performed on an 8×8 block, and the secondary transform may be performed on 4×4 sub-blocks within the transformed 8×8 block. In this case, the transform coefficients of the remaining area where the secondary transform is not performed may be set to 0.

[0432] Alternatively, the primary transform may be performed on the 4×4 block, and the secondary transform may be performed on an 8×8 sized region including the transformed 4×4 block.

[0433] Information indicating whether to perform secondary transformation may be signaled through a bitstream.

[0434] The inverse transform of the secondary transform (second inverse transform) can be performed in the decoder, and the inverse transform of the primary transform (first inverse transform) can be performed on the result thereof. As a result of the second inverse transform and the first inverse transform, a residual signal of the current block can be obtained.

[0435] Quantization is used to reduce the energy of a block, and the quantization process involves dividing the transform coefficients by a specific constant. The constant can be derived from a quantization parameter, and the quantization parameter can be defined as a value between 1 and 63.

[0436] If the encoder performs transformation and quantization, the decoder can obtain a residual block through inverse quantization and inverse transformation. The decoder can obtain a reconstructed block of the current block by adding the predicted block and the residual block.

[0437] If a reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss during quantization and encoding. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before the in-loop filter is applied is referred to as the first reconstructed block, and the reconstructed block after the in-loop filter is applied is referred to as the second reconstructed block.

[0438] At least one of a deblocking filter, SAO, or ALF may be applied to the first reconstructed block to obtain the second reconstructed block. In this case, SAO or ALF may be applied after the deblocking filter is applied.

[0439] The deblocking filter is used to alleviate image quality degradation (blocking artifact) generated at block boundaries when quantization is performed in block units. To apply the deblocking filter, a block strength (BS) between a first reconstructed block and adjacent reconstructed blocks may be determined.

[0440] Figure 30 is a flow chart illustrating a process for determining block strength.

[0441] like Figure 30 In the example shown, P represents a first reconstructed block, and Q represents a neighboring reconstructed block, wherein the neighboring reconstructed block may be adjacent to the left side or the top of the current block.

[0442] exist Figure 30In the illustrated example, block strength is determined considering prediction coding modes of P and Q, whether non-zero transform coefficients are included, whether inter-frame prediction is performed using the same reference image, and whether the difference in motion vectors is greater than or equal to a threshold.

[0443] Based on the block strength, it can be determined whether a deblocking filter is applied. For example, when the block strength is 0, no filtering may be performed.

[0444] SAO is used to alleviate the ringing artifact generated when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. The offset determination method includes edge offset (EO) or band offset (Band Offset). EO represents a method of determining the offset of the current sample based on the pattern of the surrounding pixels. BO represents a method of applying a common offset to a set of pixels with similar brightness values in an area. Specifically, the pixel brightness can be divided into 32 equal intervals, and pixels with similar brightness values can be set as a set. For example, 4 adjacent bands in the 32 bands can be set as a group, and the same offset can be applied to samples belonging to the 4 bands.

[0445] ALF is a method of applying a filter of a predefined size / shape to a first reconstructed image or a reconstructed image to which a deblocking filter is applied to generate a second reconstructed image. The following Equation 21 shows an application example of ALF.

[0446] Equation 21

[0447]

[0448] Any one of the predefined filter candidates may be selected in units of picture, coding tree unit, coding block, prediction block, or transform block. Any one of the size or shape of each filter candidate may be different.

[0449] Figure 31 is a diagram showing predefined filter candidates.

[0450] As in Figure 31 In the example shown, at least one of 5×5, 7×7, and 9×9 diamond shapes may be selected.

[0451] Only diamonds of size 5×5 can be used for chroma components.

[0452] To encode high-resolution videos such as panoramic videos, 360-degree videos, or 4K / 8K UHD (ultra-high definition) in real time or with low latency, it is possible to divide an image into multiple regions and encode / decode them in parallel. To this end, the image can be divided into tiles (i.e., the basic unit of parallel encoding / decoding), and these tiles can be processed in parallel.

[0453] Tiles can be constrained to have a rectangular shape. When encoding / decoding a tile, data from other tiles is not used. The probability table for context-adaptive binary arithmetic coding (CABAC) contexts can be initialized per tile and can be set to not apply loop filters at tile boundaries.

[0454] Figure 32 An example of dividing an image into multiple tiles is shown.

[0455] The tile includes at least one coding tree unit, and a boundary of the tile coincides with a boundary of the coding tree unit.

[0456] As in Figure 32 In the example shown in , the image may be divided into a plurality of tile sets. Information for dividing the image into the plurality of tile sets may be signaled via a bitstream.

[0457] Depending on the partitioning type of the image, tiles can have the same size in all areas except at the image borders.

[0458] Alternatively, the image may be divided so that tiles adjacent to each other in the horizontal direction have the same height, or the image may be divided so that tiles adjacent to each other in the vertical direction have the same width.

[0459] When the image is divided by using at least one of vertical lines or horizontal lines intersecting the image, each tile belongs to a different column and / or row. In the exemplary embodiments described below, the column to which the tile belongs is referred to as a tile column, and the row to which the tile belongs is referred to as a tile row.

[0460] Information used to determine the shape of the tiles for dividing an image can be signaled via a bitstream. This information can be encoded and signaled via a picture parameter set or a sequence parameter set. This information is used to determine the number of tiles in an image and can include information indicating the number of tile rows and information indicating the number of tile columns. For example, the syntax element num_tile_columns_minus1 indicates a value obtained by subtracting 1 from the number of tile columns, and the syntax element num_tile_rows_minus1 indicates a value obtained by subtracting 1 from the number of tile rows.

[0461] exist Figure 32In the example shown, since the number of tile columns is 4 and the number of tile rows is 3, num_tile_columns_minus1 may be 3 and num_tile_rows_minus1 may be 2.

[0462] When an image is divided into a plurality of tiles, information indicating the tile size may be signaled via the bitstream. For example, when an image is divided into a plurality of tile columns, information indicating the width of each tile column may be signaled via the bitstream, and when an image is divided into a plurality of tile rows, information indicating the height of each tile row may be signaled via the bitstream. For example, for each tile column, a syntax element column_width_minus1 indicating the width of the tile column may be encoded and signaled, and for each tile row, a syntax element row_height_minus1 indicating the height of the tile row may be encoded and signaled.

[0463] column_width_minus1 may indicate a value obtained by subtracting 1 from the width of a tile column, and row_height_minus1 may indicate a value obtained by subtracting 1 from the height of a tile row.

[0464] For the last tile column, the encoding of column_width_minus1 can be omitted, and for the last tile row, the encoding of row_height_minus1 can be omitted. The width of the last tile column and the height of the last row can be derived taking into account the image size.

[0465] The decoder can determine the size of the tile based on column_width_minus1 and row_height_minus1.

[0466] Table 9 shows a syntax table for dividing an image into tiles.

[0467]

Table 9

[0468]

[0469]

[0470] Referring to Table 9, a syntax element num_tile_columns_minus1 indicating the number of tile columns and a syntax element num_tile_rows_minus1 indicating the number of tile rows may be signaled.

[0471] Next, a syntax element uniform_spacing_flag indicating whether the image is divided into tiles of equal size may be signaled. When uniform_spacing_flag is true, tiles in the remaining areas except for the image boundaries may be divided into equal sizes.

[0472] When uniform_spacing_flag is false, a syntax element column_width_minus1 indicating the width of each tile column and a syntax element row_height_minus1 indicating the height of each tile row may be signaled.

[0473] The syntax element loop_filter_across_tiles_enabled_flag indicates whether the use of the loop filter at tile boundaries is enabled.

[0474] The tile column with the smallest width among tile columns may be referred to as a minimum-width tile, and the tile row with the smallest height among tile rows may be referred to as a minimum-height tile. Information indicating the width of the minimum-width tile and information indicating the height of the minimum-height tile may be signaled via the bitstream. For example, the syntax element min_column_width_minus1 indicates a value obtained by subtracting 1 from the width of the minimum-width tile, and the syntax element min_row_height_minus1 indicates a value obtained by subtracting 1 from the height of the minimum-height tile.

[0475] For each tile column, information indicating the difference from the minimum tile width can be signaled. For example, the syntax element diff_column_width indicates the width difference between the current tile column and the minimum tile column. The width difference can be expressed as the difference in the number of coding tree unit columns. The decoder can derive the width of the current tile by adding the width of the minimum width tile derived based on min_column_width_minus1 to the width difference derived based on diff_column_width.

[0476] Additionally, for each tile row, information indicating the difference from the minimum tile height can be signaled. For example, the syntax element diff_row_height indicates the height difference between the current tile row and the minimum tile row. The height difference can be expressed as the difference in the number of coding tree unit rows. The decoder can derive the height of the current tile by adding the minimum height tile height derived based on min_row_height_minus1 to the height difference derived based on diff_row_height.

[0477] Table 10 shows a syntax table including information on size differences.

[0478]

Table 10

[0479]

[0480] The image can be divided so that horizontally adjacent tiles have different heights, or the image can be divided so that vertically adjacent tiles have different widths. The above image division method can be referred to as a flexible tile division method, and the tiles divided by the flexible tile division method can be referred to as flexible tiles.

[0481] Figure 33 is a diagram showing an image partitioning pattern according to the flexible tile technique.

[0482] The search order of the tiles generated by dividing the image can follow a predetermined scanning order. In addition, an index can be assigned to each tile according to the predetermined scanning order.

[0483] The scanning order of the tiles can be any of raster scan, diagonal scan, vertical scan or horizontal scan. Figure 33 (a) to Figure 33 (d) shows an example of assigning an index to each tile according to raster scanning, diagonal scanning, vertical scanning, and horizontal scanning, respectively.

[0484] The next scanning order may be determined based on the size or position of the current tile. For example, when the height of the current tile is different from the height of the tile adjacent to the right of the current tile (e.g., when the height of the right-adjacent tile is greater than the height of the current tile), the leftmost tile on the same vertical line as the vertical line of the tile adjacent to the bottom of the current tile may be determined as the scanning target after the current tile.

[0485] The scanning order of tiles can be determined in units of images or sequences.

[0486] Alternatively, the tile scanning order may be determined based on the size of the first tile in the image. For example, when the width of the first tile is greater than its height, the tile scanning order may be set to horizontal scanning. When the height of the first tile is greater than its width, the tile scanning order may be set to vertical scanning. When the width and height of the first tile are the same, the tile scanning order may be set to raster scanning or diagonal scanning.

[0487] Information indicating the total number of tiles can be signaled through the bitstream. For example, when the flexible tile technique is applied, a syntax element number_of_tiles_in_picture_minus2 derived by subtracting 2 from the total number of tiles in the picture can be signaled. The decoder can identify the number of tiles included in the current picture based on number_of_tiles_in_picture_minus2.

[0488] Table 11 shows a syntax table including information on the number of tiles.

[0489]

Table 11

[0490]

[0491] To reduce the number of bits required to encode the size of a tile, information indicating the size of a sub-tile can be encoded and signaled. A sub-tile is a basic unit that constitutes a tile, and each tile can be configured to include at least one sub-tile. A sub-tile can include one or more coding tree units.

[0492] For example, the syntax element subtile_width_minus1 indicates a value obtained by subtracting 1 from the width of the sub-tile. The syntax element subtile_height_minus1 indicates a value obtained by subtracting 1 from the height of the sub-tile.

[0493] Information indicating whether the remaining tiles except the first tile have the same size as the previous tile can be encoded and signaled. For example, the syntax element use_previous_tile_size_flag indicates whether the size of the current tile is the same as the size of the previous tile. When use_previous_tile_size_flag is true, it indicates that the size of the current tile is the same as the size of the previous tile. When use_previous_tile_size_flag is false, information indicating the size of the current tile can be encoded and signaled. For the first tile, encoding of use_previous_tile_size_flag can be omitted and the value of the flag can be set to false.

[0494] The information indicating the tile size may include a syntax element tile_width_minus1[i] indicating the width of the i-th tile and a syntax element tile_height_minus1[i] indicating the height of the i-th tile.

[0495] The information indicating the tile size can indicate the difference from the size of the sub-tile. Using sub-tile size information can improve encoding / decoding efficiency by reducing the number of bits required to encode the size of each tile. For example, the width tileWidth of the i-th tile can be derived based on the following equation 22, and the height tileHeight of the i-th tile can be derived based on the following equation 23.

[0496] Equation 22

[0497] tileWidth=(subtile_width_minus1+1)*(tile_width_minus1[i]+1)

[0498] Equation 23

[0499] tileHeight=(subtile_height_minus1+1)*(tile_height_minus1[i]+1)

[0500] Alternatively, encoding of the sub-tile size information may be omitted, and the size of the i-th tile may be encoded as the tile size information as is. The sub-tile size information may optionally be encoded. Information indicating whether the sub-tile size information is encoded may be signaled via a video parameter set, a sequence parameter set, or a picture parameter set.

[0501] Information related to the tile size may be encoded to indicate the number of coding tree units and signaled. For example, column_width_minus1, min_column_width_minus1, subtile_width_minus1, tile_width_minus1, etc. may indicate the number of coding tree unit columns included in the tile. In addition, diff_column_width may indicate the difference between the number of coding tree unit columns included in the minimum width tile and the number of coding tree unit columns included in the current tile.

[0502] In addition, row_height_minus1, min_row_height_minus1, subtile_height_minus1, tile_height_minus1, etc. may indicate the number of coding tree unit rows included in the tile. Furthermore, diff_row_height may indicate the difference between the number of coding tree unit rows included in the minimum height tile and the number of coding tree unit rows included in the current tile.

[0503] The decoder can determine the size of the tile based on the number of coding tree unit columns and / or the number of coding tree unit rows derived based on the syntax elements and the size of the coding tree unit. For example, the width of the i-th tile can be set to (tile_width_minus1[i]+1)*(width of the coding tree unit), and the height of the i-th tile can be set to (tile_height_minus1[i]+1)*(height of the coding tree unit).

[0504] Meanwhile, information indicating the size of the coding tree unit may be signaled through a sequence parameter set or a picture parameter set.

[0505] Table 11 describes the use of the syntax element use_previous_tile_size_flag to indicate whether the size of the current tile is the same as the size of the previous tile. As another example, information indicating whether the width of the current tile is the same as the width of the previous tile or information indicating whether the height of the current tile is the same as the height of the previous tile may be encoded and signaled.

[0506] Table 12 shows a syntax table including information indicating whether the width of the current tile is the same as the width of the previous tiles.

[0507]

Table 12

[0508]

[0509] The syntax element use_previous_tile_width_flag indicates whether the width of the current tile is the same as the width of the previous tile. When use_previous_tile_width_flag is true, the width of the current tile can be set equal to the width of the previous tile. In this case, the encoding of the information indicating the width of the current tile can be omitted, and the width of the current tile can be derived from the width of the previous tile.

[0510] When use_previous_tile_width_flag is false, information indicating the width of the current tile may be signaled. For example, tile_width_minus1[i] may indicate a value obtained by subtracting 1 from the width of the i-th tile.

[0511] The syntax element use_previous_tile_width_flag may be encoded and signaled only when it is determined that the size of the current tile is different from the size of the previous tile (eg, when the value of use_previous_tile_size_flag is 0).

[0512] Tile_width_minus1[i] may have a value obtained by subtracting 1 from the number of coding tree unit columns included in the i-th tile. The decoder may derive the number of columns of the coding tree unit belonging to the i-th tile by adding 1 to tile_width_minus1[i] and multiplying the derived value by the width of the coding tree unit to calculate the tile width.

[0513] Table 13 shows a syntax table further including information indicating whether the height of the current tile is the same as the height of the previous tile.

[0514]

Table 13

[0515]

[0516]

[0517] The syntax element use_previous_tile_height_flag indicates whether the height of the current tile is the same as the height of the previous tile. When use_previous_tile_height_flag is true, the height of the current tile can be set equal to the height of the previous tile. In this case, the encoding of the information indicating the height of the current tile can be omitted, and the height of the current tile can be derived from the height of the previous tile.

[0518] When use_previous_tile_height_flag is false, information indicating the height of the current tile may be signaled. For example, tile_height_minus1[i] may indicate a value obtained by subtracting 1 from the height of the i-th tile.

[0519] The syntax element use_previous_tile_height_flag may be encoded and signaled only when the size of the current tile is determined to be different from the size of the previous tile (e.g., when the value of use_previous_tile_size_flag is 0). In addition, the syntax element use_previous_tile_height_flag is only signaled when use_previous_tile_width_flag is false.

[0520] Table 12 shows an example of using use_previous_tile_width_flag, and Table 13 shows an example of using use_previous_tile_width_flag and use_previous_tile_height_flag. Although not shown in the above table, encoding of use_previous_tile_width_flag may be omitted, and only use_previous_tile_height_flag may be used.

[0521] Which of use_previous_tile_height_flag and use_previous_tile_size_flag is used can be determined based on at least one of the tile scanning order, the width and height of the first tile, and the width and height of the previous tile. For example, when the tile scanning order is vertical, use_previous_tile_height_flag can be used, and when the tile scanning order is horizontal, use_previous_tile_width_flag can be used. Alternatively, when the first tile or the previous tile is a non-square shape with a width greater than a height, use_previous_tile_width_flag can be used, and when the first tile or the previous tile is a non-square shape with a height greater than a width, use_previous_tile_height_flag can be used.

[0522] When signaling the number of tiles included in an image, encoding of information related to the tile size may be omitted for the last tile.

[0523] Table 14 shows an example of omitting encoding of tile size information for the last tile.

[0524]

Table 14

[0525]

[0526]

[0527] When a tile size other than the last tile is specified, the remaining area in the image can be set to the last tile.

[0528] Each coding tree unit may be assigned an identifier (hereinafter referred to as a tile ID, TileID) for identifying the tile to which the coding tree unit belongs.

[0529] Figure 34is a diagram showing an example of assigning a tile ID to each coding tree unit.

[0530] The same tile ID may be assigned to the coding tree units belonging to the same tile. Specifically, the Nth TileID may be assigned to the coding tree unit belonging to tile N.

[0531] In order to determine the tile ID assigned to each coding tree unit, variables x and y indicating the position of the coding tree unit in the image can be determined. Here, x represents the value obtained by dividing the x-axis coordinate of the position (x0, y0) of the upper left sample of the coding tree unit by the width of the coding tree unit, and y represents the value obtained by dividing the y-axis coordinate of the position (x0, y0) of the upper left sample of the coding tree unit by the height of the coding tree unit. Specifically, x and y can be derived by the following equations 24 and 25.

[0532] Equation 24

[0533] x = (x0 / (CTU width))

[0534] Equation 25

[0535] Y = (y0 / (CTU height))

[0536] The operation of assigning the tile ID to each coding tree unit may be performed through the process described below.

[0537] i) Initialize tile ID

[0538] The tile ID of each coding tree unit may be initialized to a value obtained by subtracting 1 from the number of tiles in the image.

[0539]

Table 15

[0540]

[0541] ii) Export tile ID

[0542] Table 16

[0543]

[0544] In the above embodiments, a flag indicating whether to allow the application of a loop filter at tile boundaries has been signaled via a picture parameter set. However, if the loop filter is not used at all tile boundaries, subjective image quality may be degraded, and coding efficiency may be reduced.

[0545] Therefore, information indicating whether application of the loop filter is allowed for each tile may be encoded and signaled.

[0546] Figure 35 is a diagram illustrating an example of selectively determining whether to apply a loop filter to each tile.

[0547] As in Figure 35 In the example shown in , whether application of a loop filter (eg, deblocking filter, SAO, and / or ALF) is allowed at a horizontal or vertical boundary may be determined for each tile.

[0548] Table 17 shows an example of encoding information indicating whether application of a loop filter is allowed for each tile.

[0549]

Table 17

[0550]

[0551] In the example of Table 17, the syntax element loop_filter_across_tiles_flag[i] indicates whether the loop filter is allowed to be applied to the i-th tile. A value of 1 for loop_filter_across_tile_flag[i] indicates that the loop filter can be used at the horizontal and vertical boundaries of the tile with tile ID i. A value of 0 for loop_filter_across_tile_flag[i] indicates that the loop filter is not used at the horizontal and vertical boundaries of the tile with tile ID i.

[0552] Information indicating whether application of the loop filter is allowed in each of the horizontal and vertical directions may be encoded.

[0553] Table 18 shows an example of encoding information indicating whether application of the loop filter is permitted in the horizontal direction and the vertical direction respectively.

[0554]

Table 18

[0555]

[0556] In the example of Table 18, the syntax element loop_filter_hor_across_tiles_flag[i] indicates whether the loop filter is allowed to be applied at a position intersecting the i-th tile in the horizontal direction. The syntax element loop_filter_ver_across_tiles_flag[i] indicates whether the loop filter is allowed to be applied at a position intersecting the i-th tile in the vertical direction.

[0557] The value of loop_filter_hor_across_tile_flag[i] is 1, indicating that the loop filter can be used at the horizontal boundary of the tile with tile ID i. The value of loop_filter_hor_across_tile_flag[i] is 0, indicating that the loop filter is not used at the vertical boundary of the tile with tile ID i.

[0558] A value of loop_filter_ver_across_tile_flag[i] of 1 indicates that the loop filter may be used at the vertical boundary of the tile with tile ID i, and a value of loop_filter_ver_across_tile_flag[i] of 0 indicates that the loop filter is not used at the vertical boundary of the tile with tile ID i.

[0559] Alternatively, information indicating whether a tile group including a plurality of tiles allows application of a loop filter may be encoded and signaled, and whether application of a loop filter is allowed for the plurality of tiles included in the tile group may be determined based on the information.

[0560] To determine the tile group, at least one of the number of tiles belonging to the tile group, the size of the tile group, and image partition information may be signaled through a bitstream. Alternatively, a region of a predefined size in the encoder and decoder may be set as a tile group.

[0561] The encoding of the information indicating whether the application of the loop filter is permitted may be omitted, and whether the application of the loop filter is permitted may be determined based on at least one of the number of coding tree units included in the tile, the width of the tile, and the height of the tile. For example, when the tile width is less than a reference value, the application of the loop filter in the horizontal direction may be permitted, and when the tile height is less than the reference value, the application of the loop filter in the vertical direction may be permitted.

[0562] When a loop filter is applied at a tile boundary, reconstruction data outside the tile can be generated based on the data included in the tile. In this case, the reconstructed video outside the tile can be obtained by padding or interpolating the data included in the tile. Thereafter, the loop filter can be applied using the reconstruction data outside the tile.

[0563] It is also within the scope of the present invention to apply the embodiments described with a focus on the decoding process or encoding process to the encoding process or decoding process. It is also within the scope of the present invention to change the multiple embodiments described in the predetermined order to a different order than described.

[0564] The embodiments of the present invention also disclose at least the following solutions:

[0565] Solution 1. A video decoding method comprising the following steps:

[0566] Generate a merge candidate list for the current block; specify any one of a plurality of merge candidates included in the merge candidate list;

[0567] deriving a first affine seed vector and a second affine seed vector of the current block based on the first affine seed vector and the second affine seed vector of the specified merge candidate;

[0568] deriving an affine vector of a sub-block in the current block by using the first affine seed vector and the second affine seed vector of the current block, wherein the sub-block is a region having a size smaller than that of the current block; and

[0569] Motion compensated prediction is performed on the sub-block based on the affine vector, wherein,

[0570] The first affine seed vector and the second affine seed vector of the merge candidate are derived based on motion information of a neighboring block adjacent to the current block.

[0571] Solution 2. The video decoding method according to Solution 1, wherein:

[0572] When the neighboring block is included in a coding tree unit different from that of the current block, the first and second affine seed vectors of the merge candidate are derived based on motion vectors of a lower left subblock and a lower right subblock of the neighboring block.

[0573] Solution 3. The video decoding method according to Solution 2, wherein:

[0574] The lower-left subblock includes a lower-left reference sample located at a lower-left corner of the neighboring block, and the lower-right subblock includes a lower-right reference sample located at a lower-right corner of the neighboring block.

[0575] Solution 4. The video decoding method according to Solution 3, wherein:

[0576] The first affine seed vector and the second affine seed vector of the merge candidate are derived from values obtained by performing a shift operation on a difference value of a motion vector between the lower left sub-block and the lower right sub-block using a scale factor,

[0577] And the scale factor is derived based on a value obtained by adding a horizontal distance between the lower left reference sample and the lower right reference sample to an offset.

[0578] Solution 5. The video decoding method according to Solution 3, wherein:

[0579] The first affine seed vector and the second affine seed vector of the merge candidate are derived using a scaling factor based on a value obtained by performing a shift operation on a difference in motion vectors between the lower-left sub-block and the lower-right sub-block, and the scaling factor is derived based on a distance between an adjacent sample to the right of the lower-right reference sample and the lower-left reference sample.

[0580] Solution 6. The video decoding method according to Solution 1, wherein:

[0581] The merge candidate list includes a first merge candidate and a second merge candidate, the first merge candidate being derived based on an upper-neighboring block that is a first available block among upper-neighboring blocks determined to be located above the current block, and the second merge candidate being derived based on a left-neighboring block that is a first available block among left-neighboring blocks determined to be located to the left of the current block.

[0582] Solution 7. The video decoding method according to Solution 1, wherein:

[0583] When the neighboring block is included in the same coding tree unit as that of the current block,

[0584] The first and second affine seed vectors of the merge candidate are derived based on the first and second affine seed vectors of the neighboring blocks.

[0585] Solution 8. A video encoding method comprising the following steps:

[0586] Generate a merge candidate list for the current block; specify any one of a plurality of merge candidates included in the merge candidate list;

[0587] deriving a first affine seed vector and a second affine seed vector of the current block based on the first affine seed vector and the second affine seed vector of the specified merge candidate;

[0588] deriving an affine vector of a sub-block in the current block by using the first affine seed vector and the second affine seed vector of the current block, wherein the sub-block is a region having a size smaller than that of the current block; and

[0589] Motion compensated prediction is performed on the sub-block based on the affine vector, wherein,

[0590] The first affine seed vector and the second affine seed vector of the merge candidate are derived based on motion information of a neighboring block adjacent to the current block.

[0591] Solution 9. The video encoding method according to Solution 8, wherein:

[0592] When the neighboring block is included in a coding tree unit different from that of the current block, the first and second affine seed vectors of the merge candidate are derived based on motion vectors of a lower left subblock and a lower right subblock of the neighboring block.

[0593] Solution 10. The video encoding method according to Solution 9, wherein:

[0594] The lower-left subblock includes a lower-left reference sample located at a lower-left corner of the neighboring block, and the lower-right subblock includes a lower-right reference sample located at a lower-right corner of the neighboring block.

[0595] Solution 11. The video encoding method according to Solution 10, wherein:

[0596] The first affine seed vector and the second affine seed vector of the merge candidate are derived from values obtained by performing a shift operation on a difference value of a motion vector between the lower left sub-block and the lower right sub-block using a scale factor,

[0597] And the scale factor is derived based on a value obtained by adding a horizontal distance between the lower left reference sample and the lower right reference sample to an offset.

[0598] Solution 12. The video encoding method according to Solution 10, wherein:

[0599] The first affine seed vector and the second affine seed vector of the merge candidate are derived from values obtained by performing a shift operation on a difference value of a motion vector between the lower left sub-block and the lower right sub-block using a scale factor,

[0600] And the scaling factor is derived based on the distance between the adjacent sample to the right of the lower right reference sample and the lower left reference sample.

[0601] Solution 13. The video encoding method according to Solution 8, wherein:

[0602] The merge candidate list includes a first merge candidate and a second merge candidate, the first merge candidate being derived based on an upper-neighboring block that is a first available block among upper-neighboring blocks determined to be located above the current block, and the second merge candidate being derived based on a left-neighboring block that is a first available block among left-neighboring blocks determined to be located to the left of the current block.

[0603] Solution 14. The video encoding method according to Solution 8, wherein:

[0604] When the neighboring block is included in the same coding tree unit as that of the current block,

[0605] The first and second affine seed vectors of the merge candidate are derived based on the first and second affine seed vectors of the neighboring blocks.

[0606] Embodiments have been described based on a series of steps or flow charts, but this does not limit the chronological order of the invention, and can be executed simultaneously or in another order as needed. In addition, in the above-mentioned embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagram can also be implemented as hardware devices or software respectively, and multiple structural elements can be combined and implemented as a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc., individually or in combination. Examples of computer-readable recording media can include magnetic media such as hard disks, floppy disks and tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and ROMs, RAMs, flash memories, etc., which are hardware devices specially configured to store program instructions and execute the instructions. The hardware device can be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0607] Industrial Applicability

[0608] The present invention can be applied to electronic devices that encode / decode videos.

Claims

1. A video decoding method, comprising: In response to an upper boundary of a current block contacting an upper boundary of a coding tree unit of the current block, obtaining a first affine seed vector (nv0) of a lower left corner control point of an upper affine neighboring block of the current block and a second affine seed vector (nv1) of a lower right corner control point of the upper affine neighboring block; Derived a third affine seed vector (sv0) and a fourth affine seed vector (sv1) of the current block based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the upper affine neighboring block; deriving an affine vector for a sub-block in the current block using the third affine seed vector (sv0) and the fourth affine seed vector (sv1) of the current block, wherein the sub-block is a region having a size smaller than a size of the current block; as well as Motion compensated prediction is performed on the sub-block based on the affine vector.

2. The method according to claim 1, wherein The sample to the right of the lower right corner sample of the lower right corner sub-block of the upper affine neighboring block is set as the reference sample of the lower right corner control point, and the lower right corner sub-block is set as the affine sub-block of the lower right corner control point.

3. The method according to claim 1 or 2, wherein: The sample to the left of the lower left corner sample of the lower left corner sub-block of the upper affine neighboring block is set as the reference sample of the lower left corner control point, and the lower left corner sub-block is set as the affine sub-block of the lower left corner control point.

4. A video encoding method, comprising: In response to an upper boundary of a current block contacting an upper boundary of a coding tree unit of the current block, obtaining a first affine seed vector (nv0) of a lower left corner control point of an upper affine neighboring block of the current block and a second affine seed vector (nv1) of a lower right corner control point of the upper affine neighboring block; Derived a third affine seed vector (sv0) and a fourth affine seed vector (sv1) of the current block based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the upper affine neighboring block; deriving an affine vector for a sub-block in the current block using the third affine seed vector (sv0) and the fourth affine seed vector (sv1) of the current block, wherein the sub-block is a region having a size smaller than a size of the current block; as well as Motion compensated prediction is performed on the sub-block based on the affine vector.

5. The method according to claim 4, wherein The sample to the right of the lower right corner sample of the lower right corner sub-block of the upper affine neighboring block is set as the reference sample of the lower right corner control point, and the lower right corner sub-block is set as the affine sub-block of the lower right corner control point.

6. The method according to claim 4 or 5, wherein: The sample to the left of the lower left corner sample of the lower left corner sub-block of the upper affine neighboring block is set as the reference sample of the lower left corner control point, and the lower left corner sub-block is set as the affine sub-block of the lower left corner control point.

7. A video decoder configured to perform the following operations: In response to an upper boundary of a current block contacting an upper boundary of a coding tree unit of the current block, obtaining a first affine seed vector (nv0) of a lower left corner control point of an upper affine neighboring block of the current block and a second affine seed vector (nv1) of a lower right corner control point of the upper affine neighboring block; Derived a third affine seed vector (sv0) and a fourth affine seed vector (sv1) of the current block based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the upper affine neighboring block; deriving an affine vector for a sub-block in the current block using the third affine seed vector (sv0) and the fourth affine seed vector (sv1) of the current block, wherein the sub-block is a region having a size smaller than a size of the current block; as well as Motion compensated prediction is performed on the sub-block based on the affine vector.

8. The video decoder according to claim 7, wherein: The sample to the right of the lower right corner sample of the lower right corner sub-block of the upper affine neighboring block is set as the reference sample of the lower right corner control point, and the lower right corner sub-block is set as the affine sub-block of the lower right corner control point.

9. The video decoder according to claim 7 or 8, wherein: The sample to the left of the lower left corner sample of the lower left corner sub-block of the upper affine neighboring block is set as the reference sample of the lower left corner control point, and the lower left corner sub-block is set as the affine sub-block of the lower left corner control point.

10. A video encoder configured to perform the following operations: In response to an upper boundary of a current block contacting an upper boundary of a coding tree unit of the current block, obtaining a first affine seed vector (nv0) of a lower left corner control point of an upper affine neighboring block of the current block and a second affine seed vector (nv1) of a lower right corner control point of the upper affine neighboring block; Derived a third affine seed vector (sv0) and a fourth affine seed vector (sv1) of the current block based on the first affine seed vector (nv0) and the second affine seed vector (nv1) of the upper affine neighboring block; deriving an affine vector for a sub-block in the current block using the third affine seed vector (sv0) and the fourth affine seed vector (sv1) of the current block, wherein the sub-block is a region having a size smaller than a size of the current block; as well as Motion compensated prediction is performed on the sub-block based on the affine vector. The video encoder according to claim 10 , wherein: The sample to the right of the lower right corner sample of the lower right corner sub-block of the upper affine neighboring block is set as the reference sample of the lower right corner control point, and the lower right corner sub-block is set as the affine sub-block of the lower right corner control point.

12. The video encoder according to claim 10 or 11, wherein: The sample to the left of the lower left corner sample of the lower left corner sub-block of the upper affine neighboring block is set as the reference sample of the lower left corner control point, and the lower left corner sub-block is set as the affine sub-block of the lower left corner control point.

13. A bit stream transmission method, wherein: The bit stream is generated according to the encoding method according to any one of claims 4 to 6.

Images