Method for encoding / decoding a video signal and device for said method

By dividing coded blocks into multiple prediction units and utilizing inter-motion information lists, the method enhances video signal encoding and decoding efficiency, overcoming the limitations of HEVC in high-definition video compression.

JP7577232B2Active Publication Date: 2024-11-01GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD

Patent Information

Application Number
JP2024010339
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2018-11-27
Filing Date
2024-01-26
Publication Date
2024-11-01
Estimated Expiration
2039-11-07

AI Technical Summary

Technical Problem

The increasing data requirements of high-definition video services pose challenges for existing video compression technologies, with HEVC showing limitations in efficiency, necessitating improved methods for encoding and decoding video signals.

Method used

A method for dividing coded blocks into multiple prediction units and deriving motion information for each, using inter-motion information lists to enhance video signal encoding and decoding efficiency.

Benefits of technology

Improves inter prediction efficiency by allowing for more precise motion estimation and compression, addressing the limitations of existing video compression standards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007577232000044
    Figure 0007577232000044
  • Figure 0007577232000045
    Figure 0007577232000045
  • Figure 0007577232000046
    Figure 0007577232000046
Patent Text Reader

Abstract

To provide a method and an apparatus for dividing a coded block into a plurality of predictive blocks when encoding / decoding a video signal.SOLUTION: A dividing method of a coding block into multiple prediction units using diagonal lines includes determining whether to divide the coded block into a first prediction unit and a second prediction unit, determining the partition type of the coded block when determine to partition the coded block, deriving first motion information of the first prediction unit and second motion information of the second prediction unit of the encoded block, and obtaining a predicted sample in the encoded block on the basis of the first motion information and the second motion information.SELECTED DRAWING: Figure 33
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a method for encoding / decoding a video signal and to an apparatus for said method. [Background technology]

[0002] As display panels get larger, video services with higher picture quality are becoming necessary. The biggest problem with high-definition video services is the large increase in data volume. To solve this problem, active research is being conducted to improve video compression rates. As a representative example, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC) in 2009. JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013, but its compression performance is about twice that of H.264 / AVC. With the rapid development of high-definition video services, the performance limitations of HEVC are gradually becoming apparent. Summary of the Invention [Problem to be solved by the invention]

[0003] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide a method for dividing a coding block into a number of prediction blocks when encoding / decoding a video signal, and an apparatus for implementing said method.

[0004] It is an object of the present invention to provide a method for deriving motion information of each of a plurality of predictive blocks when encoding / decoding a video signal, and an apparatus for carrying out said method.

[0005] SUMMARY OF THE PRESENTLY PREFERRED EMBODIMENT An object of the present invention is to provide a method for deriving merge candidates using an inter motion information list in encoding / decoding a video signal and an apparatus for performing said method.

[0006] The technical problems that the present invention aims to achieve are not limited to those mentioned above, and those skilled in the art will be able to clearly understand other technical problems not mentioned from the following description. [Means for solving the problem]

[0007] The video signal encoding / decoding method of the present invention includes: determining whether to divide a coding block into a first prediction unit and a second prediction unit; determining a division type of the coding block when it is determined to divide the coding block; deriving first motion information of the first prediction unit and second motion information of the second prediction unit of the coding block; and obtaining a prediction sample in the coding block based on the first motion information and the second motion information. In this case, the first motion information of the first prediction unit may be obtained based on a first merging candidate, the first merging candidate being selected from a plurality of merging candidates included in a merging candidate list by first index information; and the second motion information of the second prediction unit may be obtained based on a second merging candidate, the second merging candidate being selected from a plurality of merging candidates included in the merging candidate list by second index information.

[0008] In the video signal encoding / decoding method of the present invention, when the value of the second index information is greater than or equal to the value of the first index information, the second merging candidate may have an index value that is 1 greater than the value of the second index information.

[0009] In the video signal encoding / decoding method of the present invention, when a value of the second index information is smaller than a value of the first index information, the value of the second index information may be set as an index of the second merging candidate.

[0010] In the video signal encoding / decoding method of the present invention, when the prediction sample is included in a boundary area between the first prediction unit and the second prediction unit, the prediction sample may be derived by weighting and calculating the first prediction sample derived based on the first motion information and the second prediction sample derived based on the second motion information.

[0011] In the video signal encoding / decoding method of the present invention, a first weighting value to be applied to the first predicted sample may be determined based on an x-axis coordinate and a y-axis coordinate of the predicted sample.

[0012] In the video signal encoding / decoding method of the present invention, a second weighting value applied to the second predicted sample may be derived by subtracting the first weighting value from a constant.

[0013] In the video signal encoding / decoding method of the present invention, the size of the boundary area may be determined based on at least one of the size of the encoding block or the shape of the encoding block.

[0014] The above briefly described features of the present invention are exemplary embodiments of the detailed description of the invention set forth below and are not intended to limit the scope of the present invention. Effect of the Invention

[0015] According to the present invention, by providing a method for dividing a coding block into a plurality of predictive blocks and deriving motion information of each of the plurality of predictive blocks, it is possible to improve inter prediction efficiency.

[0016] According to the present invention, by providing a method for deriving merge candidates using an inter motion information list, the inter prediction efficiency can be improved.

[0017] The effects that can be achieved by the present invention are not limited to the above effects, and those skilled in the art can clearly understand other effects not mentioned in the following description. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a block diagram of a video encoder according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing a number of division types of a coding block. [Diagram 5] FIG. 5 is a diagram showing an example of division of a coding tree unit. [Figure 6] FIG. 6 is a flowchart of an inter prediction method according to an embodiment of the present invention. [Figure 7] FIG. 7 is a diagram illustrating the nonlinear motion of an object. [Figure 8] FIG. 8 is a flowchart illustrating an affine motion-based inter-prediction method according to an embodiment of the present invention. [Figure 9] FIG. 9 is a diagram showing examples of affine seed vectors of each affine motion model. [Figure 10] FIG. 10 is a diagram showing an example of affine vectors of sub-blocks in a four-parameter motion model. [Figure 11] FIG. 11 is a diagram showing neighboring blocks that can be used to derive merging candidates. [Figure 12] FIG. 12 is a diagram illustrating deriving an affine seed vector for a current block based on the affine seed vectors of affine neighboring blocks. [Figure 13]FIG. 13 is a diagram showing an example in which the motion vector of a sub-block is set as the affine seed vector of an affine adjacent block. [Figure 14] FIG. 14 is a diagram showing the positions of the reference samples. [Figure 15] FIG. 15 is a diagram showing the positions of the reference samples. [Figure 16] FIG. 16 is a diagram showing the positions of the reference samples. [Figure 17] FIG. 17 is a diagram showing an application example of the modified affine merge vector derivation method. [Figure 18] FIG. 18 is a diagram showing an example of deriving an affine seed vector of an affine merge candidate based on a plurality of motion vectors of a plurality of sub-blocks belonging to an adjacent block. [Figure 19] FIG. 19 is a diagram illustrating an example of deriving an affine seed vector of an affine merge candidate based on the motion vectors of multiple sub-blocks located to the left of the current block. [Figure 20] FIG. 20 is a diagram illustrating an example of deriving an affine seed vector of an affine merge candidate based on motion information of a non-adjacent block or an adjacent block located to the left of the current block. [Figure 21] FIG. 21 is a diagram showing the positions of blocks for deriving affine seed vectors of affine merge candidates. [Figure 22] FIG. 22 is a diagram for explaining an example of deriving combined merge candidates by combining a plurality of motion vectors of a plurality of adjacent blocks. [Figure 23] FIG. 23 shows that adjacent blocks cannot be used. [Figure 24] FIG. 24 is a flowchart of a process for deriving motion information of a current block in the merge mode. [Diagram 25] FIG. 25 is a diagram for explaining an example of updating the inter-motion information list. [Figure 26] FIG. 26 illustrates an embodiment of updating the inter-merge candidate list. [Figure 27]FIG. 27 is a diagram showing an example in which the indices of pre-stored inter-merge candidates are updated. [Figure 28] FIG. 28 shows the locations of representative sub-blocks. [Figure 29] FIG. 29 is an example illustrating generating inter motion information lists for different inter prediction modes. [Diagram 30] FIG. 30 is a diagram showing an example of adding an inter-merge candidate included in the long-term motion information list to the merge candidate list. [Diagram 31] FIG. 31 is a diagram showing an example in which redundancy check is performed on only some of the merging candidates. [Diagram 32] FIG. 32 is a diagram showing an example in which redundancy checks are omitted for a specific merging candidate. [Diagram 33] FIG. 33 is a diagram showing an example of dividing a coding block into a plurality of prediction units using diagonals. [Diagram 34] FIG. 34 is a diagram showing an example of dividing a coding block into two prediction units. [Diagram 35] FIG. 35 is a diagram showing an example of dividing a coding block into a plurality of prediction blocks of different sizes. [Diagram 36] FIG. 36 is a diagram showing adjacent blocks for deriving triangle merging candidates. [Figure 37] FIG. 37 is a diagram for explaining an example of determining the availability of adjacent blocks for each triangular prediction unit. [Figure 38] FIG. 38 is a diagram showing an example of deriving a predicted sample by weighting and calculating the first predicted sample and the second predicted sample. [Figure 39] FIG. 39 is a diagram showing an example of deriving a predicted sample by weighting and calculating the first predicted sample and the second predicted sample. [Diagram 40] FIG. 40 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention. [Diagram 41] FIG. 41 is a diagram showing intra prediction modes. [Diagram 42]FIG. 42 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Diagram 43] FIG. 43 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Diagram 44] FIG. 44 is a diagram showing angles between angular intra prediction modes and a line parallel to the x-axis. [Diagram 45] FIG. 45 is a diagram showing an example of obtaining predicted samples when the current block is non-square. [Figure 46] FIG. 46 is a diagram showing the wide-angle intra prediction mode. [Figure 47] FIG. 47 is a flow chart showing the process of determining the blocking strength. [Figure 48] FIG. 48 is a diagram showing predefined filter candidates. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0020] Video encoding and decoding is performed on a block-by-block basis. For example, encoding / decoding operations such as transform, quantization, prediction, in-loop filtering, or reconstruction may be performed on a coding block, a transform block, or a prediction block.

[0021] Hereinafter, the block to be coded / decoded is referred to as a “current block.” For example, depending on the current coding / decoding process step, the current block may indicate a coding block, a transformation block, or a prediction block.

[0022] Also, as used herein, the term "unit" may be understood to refer to a basic unit for performing a particular encoding / decoding process, and "block" may be understood to refer to a sample array of a given size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described below, a coding block and a coding unit may be understood to have the same meaning.

[0023] FIG. 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0024] As shown in FIG. 1, the video encoding device 100 may include a picture partitioning unit 110, a prediction unit 120, 125, a transform unit 130, a quantization unit 135, a reordering unit 160, an entropy coding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150 and a memory 155.

[0025] Each component shown in FIG. 1 is shown separately to illustrate various characteristic functions of the video encoding device, without indicating that each component is a separate hardware or a single software component. That is, for convenience of explanation, each component is listed and included as a representative component, and at least two components may be combined into one component or one component may be divided into multiple components to perform a function. As long as it does not deviate from the essence of the present invention, embodiments in which each component is combined and embodiments in which each component is separated are also within the scope of the present invention.

[0026] In addition, some structural elements are not necessary for performing the essential functions of the present invention, but are optional structural elements used only to improve performance. The present invention may be implemented by including only the components necessary to realize the essence of the present invention (not including structural elements for improving performance), and a structure including only necessary structural elements (not including structural elements for improving performance) also falls within the scope of the present invention.

[0027] The picture division unit 110 can divide an input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU, Prediction Unit), a transform unit (TU, Transform Unit), or a coding unit (CU, Coding Unit). The picture division unit 110 can divide one picture into a combination of multiple coding units, prediction units, and transform units, and select a combination of the coding unit, prediction unit, and transform unit based on a predetermined standard (e.g., a cost function) to code the picture.

[0028] For example, a picture may be divided into multiple coding units. A recursive tree structure, such as a quad tree structure, may be used to divide a picture into coding units, and a video or largest coding unit may be used as a root to divide the coding units into other coding units, and the coding unit may have as many child nodes as the number of divided coding units. Due to some restrictions, coding units that are not divided again become leaf nodes. That is, assuming that a coding unit can only realize a square division, a coding unit can be divided into at most four other coding units.

[0029] Hereinafter, in the embodiments of the present invention, the encoding unit may refer to a unit that performs encoding, and may also refer to a unit that performs decoding.

[0030] The prediction units in a coding unit may be divided into at least one square or rectangle of the same size, or one prediction unit in a coding unit may be divided into a different shape and / or size than the other prediction units.

[0031] When a prediction unit for performing intra prediction using a coding unit is not the smallest coding unit, intra prediction may be performed without dividing the prediction unit into a plurality of N×N prediction units.

[0032] The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It may be determined whether to use inter prediction or intra prediction for the prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) may be determined based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific contents. For example, the prediction method and prediction mode may be determined by the prediction unit, and the prediction may be performed by the transformation unit. Residual values ​​(residual blocks) between the generated prediction block and the original block may be input to the transformation unit 130. In addition, prediction mode information, motion vector information, etc. for prediction may be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When a specific encoding mode is used, the prediction units 120 and 125 may not generate a prediction block, but the original block may be directly encoded and transmitted to the decoder.

[0033] The inter prediction unit 120 may predict the prediction unit based on information of at least one of a picture before or after the current picture. In some cases, the prediction unit may be further predicted based on information of a part of an encoded region in the current picture. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0034] The reference picture interpolation unit can receive reference picture information from the memory 155 and generate integer pixel or fractional pixel information based on the reference picture. For luma pixels, a DCT-based 8-dimensional interpolation filter with different filter coefficients may be used to generate fractional pixel information in 1 / 4 pixel units. For chroma signals, a DCT-based 4-dimensional interpolation filter with different filter coefficients may be used to generate fractional pixel information in 1 / 8 pixel units.

[0035] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. A motion vector may be calculated using a number of methods, such as a full search-based block matching algorithm (FBMA), a three step search (TSS), a new three step search algorithm (NTS), etc. The motion vector may have a motion vector value in units of half pixel or quarter pixel based on the interpolated pixel. The motion prediction unit may predict the current prediction unit using various motion prediction methods. The motion prediction method may be a number of methods, such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, etc.

[0036] The intra prediction unit 125 may generate a prediction unit based on reference pixel information surrounding the current block (the reference pixel information is pixel information in the current picture). When a neighboring block of the current prediction unit is a block on which inter prediction has been performed and the reference pixels are pixels on which inter prediction has been performed, the reference pixels included in the block on which inter prediction has been performed may be used as reference pixel information of the neighboring block on which intra prediction has been performed. That is, when a reference pixel is unavailable, at least one of the available reference pixels may replace the unavailable reference pixel information.

[0037] In intra prediction, the prediction mode may include an angular prediction mode that uses reference pixel information based on the prediction direction, and a non-angular mode that does not use direction information when performing prediction. The prediction mode of the luma information and the prediction mode of the chroma information may be different, and the intra prediction mode information used to predict the luma information or the predicted luma signal information may be used to predict the chroma information.

[0038] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction may be performed on the prediction unit based on the pixel located to the left, the pixel located to the top left, and the pixel located above. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction may be performed based on the reference pixel of the transform unit. In addition, intra prediction using N×N division may be applied only to the minimum coding unit.

[0039] The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to the reference pixels based on the prediction mode. The type of the adaptive intra smoothing filter applied to the reference pixels may be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted based on the intra prediction mode of a prediction unit located in the vicinity of the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same may be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, the prediction mode information of the current block may be encoded by performing entropy encoding.

[0040] Also, a residual block including residual information can be generated, the residual information being a difference between a prediction unit that performs prediction using the prediction unit generated in the prediction unit 120, 125 and the original block of the prediction unit. The generated residual block can be input to the conversion unit 130.

[0041] The transform unit 130 may transform the residual block using a transform method such as Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST), and the residual block includes residual information between the original block and the prediction unit generated by the prediction unit 120, 125. The DCT transform core includes at least one of DCT2 or DCT8, and the DST transform core includes DST7. It may be determined whether to apply DCT or DST to transform the residual block based on intra prediction mode information of the prediction unit used to generate the residual block. Furthermore, the transform of the residual block may be skipped. A flag indicating whether to skip the transform of the residual block may be coded. A residual block whose size is equal to or smaller than a threshold, a luma component, or a chroma component (4:4:4 format or smaller) may be allowed to skip the transform.

[0042] The quantization unit 135 may quantize the values ​​transformed into the frequency domain by the transform unit 130. The quantization coefficient may vary depending on the importance of the block or the video. The values ​​calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the reordering unit 160.

[0043] Reordering unit 160 may perform reordering of coefficient values ​​on the quantized residual values.

[0044] The rearrangement unit 160 may convert two-dimensional block shape coefficients into one-dimensional vector format by a coefficient scanning method. For example, the rearrangement unit 160 may scan DC coefficients to high frequency region coefficients using a zig-zag scan method and convert them into one-dimensional vector format. Depending on the size of the transform unit and the intra prediction mode, instead of the zig-zag scan, vertical scanning for scanning two-dimensional block shape coefficients along a column direction and horizontal scanning for scanning two-dimensional block shape coefficients along a row direction may be used. That is, depending on the size of the transform unit and the intra prediction mode, it may be determined which scanning method to use among zig-zag scan, vertical scan, and horizontal scan.

[0045] The entropy coding unit 165 may perform entropy coding based on the values ​​calculated by the reordering unit 160. For example, the entropy coding may use a number of coding methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0046] The entropy coding unit 165 may code multiple pieces of information such as residual value coefficient information of the coding unit from the rearrangement unit 160 and the prediction units 120 and 125, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0047] The entropy coding unit 165 can perform entropy coding on the coefficient values ​​of the coding unit input from the rearrangement unit 160 .

[0048] The inverse quantization unit 140 and the inverse transform unit 145 inverse quantize a plurality of values ​​quantized by the quantization unit 135 and inverse transform the values ​​transformed by the transform unit 130. Residual values ​​generated by the inverse quantization unit 140 and the inverse transform unit 145 can be merged with prediction units predicted by a motion prediction unit, a motion compensation unit, and an intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0049] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0050] The deblocking filter can remove block artifacts generated in a reconstructed picture due to boundaries between blocks. In order to determine whether to perform deblocking, it may be determined whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter may be applied based on a required deblocking filtering strength. In addition, when performing vertical filtering and horizontal filtering in the process of using the deblocking filter, horizontal filtering and vertical filtering may be processed synchronously.

[0051] The offset correction unit may correct the offset between the deblocked picture and the original picture in units of pixels. The offset correction unit may perform offset correction on a designated picture by dividing pixels included in a picture into a predetermined number of regions, determining a region for which offset is to be performed, and applying an offset to the corresponding region, or applying an offset in consideration of edge information of each pixel.

[0052] Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing a reconstructed picture after filtering with an original picture. After dividing pixels included in a picture into predetermined groups, one filter to be used for the corresponding group may be determined, or filtering may be performed for each group separately. Information related to whether adaptive loop filtering is applied according to a luminance signal may be transmitted to each coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may be different for each block. Furthermore, the same type (fixed type) of ALF may be applied regardless of the characteristics of the block to which it is applied.

[0053] The memory 155 may store reconstructed blocks or pictures calculated by the filter unit 150 and may provide the stored reconstructed blocks or pictures to the prediction units 120 and 125 when performing inter prediction.

[0054] FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0055] As shown in FIG. 2, the video decoder 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0056] When a video bitstream is input from a video encoder, the input bitstream may be decoded according to the reverse steps of the video encoder.

[0057] The entropy decoding unit 210 may perform entropy decoding according to the reverse steps of the steps of performing entropy encoding in the entropy encoding unit of the video encoder. For example, in accordance with the method performed in the video encoder, multiple methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc. may be applied.

[0058] The entropy decoder 210 may decode information related to the intra- and inter-prediction performed by the encoder.

[0059] The reordering unit 215 may perform reordering based on the method of reordering the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. The reordering unit 215 may restore a plurality of coefficients represented in a one-dimensional vector format to coefficients in a two-dimensional block shape and perform reordering. The reordering unit 215 may receive information related to coefficient scanning performed by the encoding unit and perform reordering by performing a reverse scan according to the scanning order performed by the corresponding encoding unit.

[0060] The inverse quantization unit 220 may perform inverse quantization based on a quantization parameter provided by the encoder and the coefficient values ​​of the block to be reordered.

[0061] The inverse transform unit 225 may perform an inverse discrete cosine transform and an inverse discrete sine transform on the quantization result performed by the video encoder, and the inverse discrete cosine transform and the inverse discrete sine transform belong to the inverse transform of the transform performed in the transform unit, i.e., the inverse transform of the discrete cosine transform and the discrete sine transform. The DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform unit 225 may not perform the inverse transform. The inverse transform may be performed by a transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder may selectively perform a transform method (e.g., DCT or DST) based on a plurality of information such as a prediction method, a size of a current block, and a prediction direction.

[0062] The prediction units 230 and 235 may generate a prediction block based on information related to the generation of the prediction block provided by the entropy decoding unit 210 and previously decoded block or picture information provided by the memory 245 .

[0063] As described above, when performing intra prediction in the same manner as the operation in a video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixel located to the left of the prediction unit, the pixel located to the upper left, and the pixel located above the prediction unit, and if the size of the prediction unit when performing intra prediction is different from the size of the transform unit, intra prediction may be performed using reference pixels based on the transform unit. In addition, intra prediction using N×N division may be applied only to the minimum coding unit.

[0064] The prediction unit 230, 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives multiple types of information, such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and classifies the prediction unit based on the current encoding unit to determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may use information necessary for performing inter prediction of the current prediction unit provided by the video encoder, and perform inter prediction on the current prediction unit based on information included in at least one picture of the previous picture or the next picture of the current picture to which the current prediction unit belongs. Alternatively, the inter prediction may be further performed based on information of a restored part of the area in the current picture to which the current prediction unit belongs.

[0065] To perform inter prediction, the coding unit may determine which of the following modes is the motion prediction method of the prediction unit included in the corresponding coding unit: skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), or intra block copy mode.

[0066] The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit on which intra prediction has been performed, the intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra smoothing filter is a part that performs filtering on reference pixels of the current block, and may determine whether to apply a filter based on the prediction mode of the current prediction unit. Adaptive intra smoothing filtering may be performed on reference pixels of the current block using the prediction mode of the prediction unit and adaptive intra smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode in which adaptive intra smoothing filtering is not performed, the adaptive intra smoothing filter may not be applied.

[0067] When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values ​​to be interpolated for reference pixels, the reference pixel interpolation unit may generate reference pixels in pixel units of integer or fractional values ​​by interpolating the reference pixels. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block in a manner that does not interpolate reference pixels, the reference pixels may not need to be interpolated. When the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0068] The reconstructed block or picture may be provided to a filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0069] From the video encoder, information related to whether to apply a deblocking filter to a corresponding block or picture and information related to whether to apply a strong filter or a weak filter when applying the deblocking filter can be received. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblock filtering on the corresponding block.

[0070] The offset correction unit can perform offset correction on the reconstructed picture based on the type of offset correction applied to the picture during encoding, offset amount information, and the like.

[0071] ALF may be applied to a coding unit based on encoder-provided information related to whether or not to apply ALF, ALF coefficient information, etc. Such ALF information may be provided by being included in a particular parameter set.

[0072] The memory 245 is capable of storing the reconstructed pictures or blocks, making them available as reference pictures or blocks, and providing the reconstructed pictures to an output.

[0073] FIG. 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0074] A coding block of a maximum size may be defined as a coding tree block. A picture may be divided into multiple coding tree units (CTUs). A coding tree unit is a coding unit of a maximum size, and may be further referred to as a largest coding unit (LCU). Figure 3 shows an example of dividing a picture into multiple coding tree units.

[0075] The size of the coding tree unit may be defined at the picture level or the sequence level, and therefore, information indicating the size of the coding tree unit may be signaled by a picture parameter set or a sequence parameter set.

[0076] For example, the coding tree unit size of the whole pictures in the sequence may be set as 128×128. Alternatively, any one of 128×128 or 256×256 at the picture level may be determined as the coding tree unit size. For example, the coding tree unit size of the first picture may be set as 128×128, and the coding tree unit size of the second picture may be set as 256×256.

[0077] A coding block can be generated by dividing a coding tree unit. A coding block indicates a basic unit for performing an encoding / decoding process. For example, prediction or transformation may be performed by different coding blocks, or a predictive coding mode may be determined by different coding blocks. A predictive coding mode indicates a method for generating a predicted picture. For example, the predictive coding mode may include intra prediction (Intra Prediction), inter prediction (Inter Prediction), current picture referencing (CPR, Current Picture Referencing, or Intra Block Copy (IBC)), or combined prediction. For a coding block, at least one predictive coding mode among intra prediction, inter prediction, current picture referencing, or combined prediction may be used to generate a predictive block related to the coding block.

[0078] Information indicating the predictive coding mode of the current block may be transmitted by a signal via the bitstream. For example, the information may be a one-bit flag indicating whether the predictive coding mode is an intra mode or an inter mode. Only when it is determined that the predictive coding mode of the current block is an inter mode, the current picture reference or combined prediction may be used.

[0079] The current picture reference is used to set the current picture as a reference picture and obtain a prediction block of the current block from a coded / decoded area in the current picture. The current picture means a picture including the current block. Information indicating whether the current picture reference is applied to the current block may be transmitted by a signal via the bitstream. For example, the information may be a 1-bit flag. If the flag is true, the prediction coding mode of the current block may be determined as the current picture reference, and if the flag is false, the prediction mode of the current block may be determined as inter prediction.

[0080] Alternatively, the predictive coding mode of the current block may be determined based on the reference picture index. For example, if the reference picture index points to the current picture, the predictive coding mode of the current block may be determined as current picture reference. If the reference picture index points to another picture other than the current picture, the predictive coding mode of the current block may be determined as inter prediction. That is, the current picture reference is a prediction method that uses information of an area in the current picture after encoding / decoding, and the inter prediction is a prediction method that uses information of another picture after encoding / decoding.

[0081] The combined prediction indicates a coding mode in which two or more of intra prediction, inter prediction, and current picture reference are combined. For example, when the combined prediction is applied, the first predicted block may be generated by one of intra prediction, inter prediction, or current picture reference, and the second predicted block may be generated by another of intra prediction, inter prediction, or current picture reference. When the first predicted block and the second predicted block are generated, a final predicted block may be generated by averaging or weighting and calculating the first predicted block and the second predicted block. Information indicating whether the combined prediction is applied may be transmitted by a signal via a bitstream. The information may be a 1-bit flag.

[0082] FIG. 4 is a diagram showing a number of division types of a coding block.

[0083] A coding block may be divided into a plurality of coding blocks by quadtree division, binary tree division, or ternary tree division. Furthermore, a coding block divided by quadtree division, binary tree division, or ternary tree division may be further divided into a plurality of coding blocks.

[0084] Quadtree partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of the quadtree partitioning, the current block may be divided into four square segments (see “SPLIT_QT” in part (a) of Figure 4).

[0085] Binary tree splitting refers to a splitting technique that splits a current block into two blocks. The process of splitting a current block into two blocks along the vertical direction (i.e., using a vertical line across the current block) may be referred to as vertical binary tree splitting, and the process of splitting a current block into two blocks along the horizontal direction (i.e., using a horizontal line across the current block) may be referred to as horizontal binary tree splitting. As a result of the binary tree splitting, the current block may be split into two non-square segments. "SPLIT_BT_VER" in part (b) of FIG. 4 indicates the vertical binary tree splitting result, and "SPLIT_BT_HOR" in part (c) of FIG. 4 indicates the horizontal binary tree splitting result.

[0086] Ternary tree partitioning refers to a partitioning technique that partitions the current block into three blocks. The process of partitioning the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) may be referred to as vertical ternary tree partitioning, and the process of partitioning the current block into three blocks along the horizontal direction (i.e., using two horizontal lines crossing the current block) may be referred to as horizontal ternary tree partitioning. As a result of the ternary tree partitioning, the current block may be partitioned into three non-square segments. In such a case, the width / height of the segment located at the center of the current block may be twice the width / height of the other segments. "SPLIT_TT_VER" in part (d) of FIG. 4 indicates the vertical ternary tree partitioning result, and "SPLIT_TT_HOR" in part (e) of FIG. 4 indicates the horizontal ternary tree partitioning result.

[0087] The number of times a coding tree unit is divided may be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit may be determined at a sequence or picture level. Therefore, the maximum partitioning depth of a coding tree unit may be different for different sequences or pictures.

[0088] Alternatively, the maximum partitioning depth may be determined independently for each of multiple partitioning techniques, e.g., the maximum partitioning depth allowed for quadtree partitioning may be different from the maximum partitioning depth allowed for binary and / or ternary tree partitioning.

[0089] The encoder may transmit information indicating at least one of the partition type or partition depth of the current block by a signal via the bitstream, and the decoder may determine the partition type and partition depth of the coding tree unit based on the information parsed from the bitstream.

[0090] FIG. 5 is a diagram showing an example of division of a coding tree unit.

[0091] The process of partitioning a coding block using partitioning techniques such as quad-tree partitioning, binary-tree partitioning, and / or ternary-tree partitioning may be referred to as multi-tree partitioning.

[0092] A coding block generated by applying multi-tree partitioning to a coding block may be called a downward coding block. When the partition depth of a coding block is k, the partition depth of multiple downstream coding blocks is set as k+1.

[0093] On the other hand, with respect to a coding block with a division depth of k+1, a coding block with a division depth of k may be referred to as an upstream coding block.

[0094] The partition type of the current coding block may be determined based on at least one of the partition type of the upstream coding block or the partition type of the adjacent coding block. The adjacent coding block may be adjacent to the current coding block and may include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the current coding block. The partition type may include at least one of whether to partition into a quad tree, whether to partition into a binary tree, the partition direction of the binary tree, whether to partition into a ternary tree, or the partition direction of the ternary tree.

[0095] To determine the split type of the coding block, a signal may be sent via the bitstream to indicate whether the coding block is split, the signal being a one-bit flag "split_cu_flag", which indicates that the coding block is split by a multi-tree splitting technique if the flag is true.

[0096] If "split_cu_flag" is true, a signal may be sent via the bitstream indicating whether the coding block has been quadtree split, the information being a 1-bit flag "split_qt_flag", where if the flag is true, the coding block may be split into four blocks.

[0097] 5 shows an example in which a coding tree unit is quadtree-divided to generate four coding blocks with a division depth of 1. It also shows that quadtree division is applied again to the first and fourth coding blocks among the four coding blocks generated as a result of the quadtree division. Finally, four coding blocks with a division depth of 2 can be generated.

[0098] In addition, quadtree division may be applied again to an encoding block with a division depth of 2 to generate an encoding block with a division depth of 3.

[0099] When quad-tree partitioning is not applied to a coding block, it may be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block, taking into consideration at least one of the following: a size of the coding block, whether the coding block is located on a picture boundary, a maximum partition depth, or a partition type of an adjacent block. When it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating a partitioning direction may be transmitted by a signal via a bit stream. The information may be a one-bit flag "mtt_split_cu_vertical_flag". It may be determined whether the partitioning direction is vertical or horizontal based on the flag. Also, it may be transmitted by a signal via a bit stream, which information of binary tree partitioning or ternary tree partitioning to apply to the coding block. The information may be a one-bit flag "mtt_split_cu_binary_flag". It may be determined whether to apply binary tree partitioning or ternary tree partitioning to the coding block based on the flag.

[0100] For example, the example shown in Figure 5 shows that vertical binary tree partitioning is applied to a coding block with a partition depth of 1, vertical ternary tree partitioning is applied to the left coding block among the coding blocks generated as a result of the partitioning, and vertical binary tree partitioning is applied to the right coding block.

[0101] Inter prediction means predicting the predictive coding mode of the current block using information of the previous picture. For example, a block in the previous picture that is located at the same position as the current block (hereinafter referred to as a collocated block) may be set as the prediction block of the current block. Hereinafter, a prediction block generated based on a block that is located at the same position as the current block is referred to as a collocated prediction block.

[0102] On the other hand, if an object in a previous picture has already moved to another position in the current picture, the current block may be effectively predicted according to the motion of the object. For example, if the moving direction and size of an object can be known by comparing the previous picture with the current picture, a predicted block (or predicted picture) of the current block may be generated taking into account the motion information of the object. Hereinafter, the predicted block generated according to the motion information may be referred to as a motion predicted block.

[0103] A residual block can be generated by removing the prediction block from the current block. In this case, if an object moves, the energy of the residual block can be reduced by replacing the collocated prediction block with the motion prediction block, thereby improving the compression performance of the residual block.

[0104] The process of generating a prediction block using motion information as described above may be referred to as motion compensated prediction. In most inter predictions, a prediction block may be generated by motion compensated prediction.

[0105] The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weighting value index. The motion vector indicates the moving direction and size of an object. The reference picture index specifies a reference picture of the current block among multiple reference pictures included in a reference picture list. The prediction direction indicates one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of the motion information in the L0 direction or the motion information in the L1 direction may be used based on the prediction direction of the current block. The bidirectional weighting value index specifies a weighting value applied to the L0 prediction block and a weighting value applied to the L1 prediction block.

[0106] FIG. 6 is a flowchart of an inter prediction method according to an embodiment of the present invention.

[0107] As shown in FIG. 6, the inter prediction method includes determining an inter prediction mode of a current block (S601), obtaining motion information of the current block based on the determined inter prediction mode (S602), and performing motion compensated prediction on the current block based on the obtained motion information (S603).

[0108] The inter prediction mode indicates a number of techniques for determining motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information may include a merge mode and an advanced motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. Depending on the inter prediction mode, the motion information of the current block may be determined based on information analyzed from a neighboring block or a bitstream adjacent to the current block.

[0109] The inter prediction method using affine motion information will now be described in detail.

[0110] FIG. 7 is a diagram illustrating the nonlinear motion of an object.

[0111] In a video, the motion of an object may be nonlinear. For example, in the example shown in FIG. 7, nonlinear motion of an object may occur, such as zoom-in, zoom-out, rotation, or affine transformation of a video camera. When nonlinear motion of an object occurs, the motion of the object cannot be effectively represented by a translational motion vector. Therefore, in the part where nonlinear motion of an object occurs, the translational motion may be replaced by affine motion, thereby improving coding efficiency.

[0112] FIG. 8 is a flowchart illustrating an affine motion-based inter-prediction method according to an embodiment of the present invention.

[0113] It may be determined whether to apply an affine motion-based inter prediction technique to the current block based on information analyzed from the bitstream. Specifically, it may be determined whether to apply an affine motion-based inter prediction technique to the current block based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0114] When applying an affine motion-based inter prediction technique to a current block, an affine motion model of the current block may be determined (S801). The affine motion model may be determined as at least one of a six-parameter affine motion model or a four-parameter affine motion model. The six-parameter affine motion model represents affine motion with six parameters, and the four-parameter affine motion model represents affine motion with four parameters.

[0115] Equation 1 shows the six-parameter affine motion. The affine motion represents the translational motion of a given region determined by an affine seed vector.

[0116] equation 1

number

[0117] When affine motion is represented by six parameters, complex motion can be represented, but the number of bits required to encode each parameter increases, resulting in a decrease in encoding efficiency. Therefore, affine motion may be represented by four more parameters. Equation 2 shows the case where affine motion is represented by four parameters.

[0118] equation 2

number

[0119] Information for determining an affine motion model of a current block may be coded and transmitted by a signal via a bitstream. For example, the information may be a 1-bit flag "affine_type_flag". When the value of the flag is 0, it indicates that a 4-parameter affine motion model is applied, and when the value of the flag is 1, it indicates that a 6-parameter affine motion model is applied. The flag may be coded in units of a slice, a segment, or a block (e.g., a coding block or a coding tree unit). When a flag is transmitted by a signal at a slice level, the affine motion model determined at the slice level may be applied to all blocks to which the slice belongs.

[0120] Alternatively, the affine motion model of the current block may be determined based on the affine inter prediction mode of the current block. For example, when the affine merge mode is applied, the affine motion model of the current block may be determined as a four-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information for determining the affine motion model of the current block may be encoded and transmitted by a signal via a bitstream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block may be determined based on a 1-bit flag "affine_type_flag".

[0121] Next, an affine seed vector for the current block may be derived (S802). If a four-parameter affine motion model is selected, motion vectors on two control points of the current block may be derived. On the other hand, if a six-parameter affine motion model is selected, motion vectors on three control points of the current block may be derived. The motion vectors on the control points may be referred to as affine seed vectors. The control points may include at least one of the top left corner, the top right corner, or the bottom left corner of the current block.

[0122] FIG. 9 is a diagram showing examples of affine seed vectors of each affine motion model.

[0123] In a four-parameter affine motion model, an affine seed vector associated with two of the top-left corner, the top-right corner, or the bottom-left corner may be derived. For example, in the example shown in part (a) of FIG. 9, when a four-parameter affine motion model is selected, an affine vector may be derived using an affine seed vector sv0 associated with the top-left corner of the current block (e.g., the top-left sample (x0, y0)) and an affine seed vector sv1 associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)). The affine seed vector associated with the bottom-left corner may further replace the affine seed vector associated with the top-left corner, or the affine seed vector associated with the bottom-left corner may further replace the affine seed vector associated with the top-right corner.

[0124] In a six-parameter affine motion model, affine seed vectors associated with the top-left corner, the top-right corner, and the bottom-left corner may be derived. For example, in the example shown in part (b) of FIG. 9, when a six-parameter affine motion model is selected, an affine vector may be derived using an affine seed vector sv0 associated with the top-left corner of the current block (e.g., the top-left sample (x0, y0)), an affine seed vector sv1 associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)), and an affine seed vector sv2 associated with the top-left corner of the current block (e.g., the top-left sample (x2, y2)).

[0125] In the embodiment described below, in the four-parameter affine motion model, the affine seed vectors of the top-left and top-right control points are referred to as the first and second affine seed vectors, respectively. In the embodiment described below that uses the first and second affine seed vectors, at least one of the first and second affine seed vectors may be replaced with the affine seed vector of the bottom-left control point (third affine seed vector) or the affine seed vector of the bottom-right control point (fourth affine seed vector).

[0126] In the six-parameter affine motion model, the affine seed vectors of the top-left control point, the top-right control point, and the bottom-left control point are referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector, respectively. In the embodiment described below that uses the first affine seed vector, the second affine seed vector, and the third affine seed vector, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector may be replaced with the affine seed vector of the bottom-right control point (fourth affine seed vector).

[0127] The affine seed vector may be used to derive affine vectors for different sub-blocks (S803). The affine vectors represent translational motion vectors derived based on the affine seed vector. The affine vectors of the sub-blocks may be referred to as affine sub-block motion vectors or sub-block motion vectors.

[0128] FIG. 10 is a diagram showing an example of affine vectors of sub-blocks in a four-parameter motion model.

[0129] An affine vector for a subblock may be derived based on the control point positions, the subblock positions, and the affine seed vector. For example, Equation 3 shows an example of deriving an affine subblock vector.

[0130] equation 3

number

[0131] In Equation 3, (x, y) indicates the position of the subblock. The position of the subblock indicates the position of the reference sample contained in the subblock. The reference sample may be a sample located in the upper left corner of the subblock, or a sample whose at least one of the x-axis or y-axis coordinates is located in the center position. (x0, y0) indicates the position of the first control point, and (sv 0x ,sv 0y ) indicates the first affine seed vector. Also, (x1, y1) indicates the position of the second control point, and (sv 1x ,sv 1y ) denotes the second affine seed vector.

[0132] If the first and second control points correspond to the upper left and upper right corners of the current block, respectively, x1-x0 may be set to be equal to the width of the current block.

[0133] Then, motion compensated prediction may be performed on each subblock using the affine vector of each subblock (S804). After performing motion compensated prediction, a prediction block associated with each subblock may be generated. The prediction block of the subblock may be set as the prediction block of the current block.

[0134] An affine seed vector of the current block may be derived based on an affine seed vector of a neighboring block adjacent to the current block. If the inter prediction mode of the current block is an affine merge mode, an affine seed vector of a merge candidate included in the merge candidate list may be determined as an affine seed vector of the current block. Also, if the inter prediction mode of the current block is an affine merge mode, at least one motion information of a reference picture index including the current block, a specific direction prediction flag, or a bidirectional weighting value may be set to be the same as that of a merge candidate.

[0135] The merging candidates may be derived based on neighboring blocks of the current block, which may include at least one of spatial neighboring blocks that are spatially neighboring to the current block and temporal neighboring blocks included in a picture other than the current picture.

[0136] FIG. 11 is a diagram showing neighboring blocks that can be used to derive merging candidates.

[0137] The neighboring blocks of the current block may include at least one of the neighboring block (A) adjacent to the left of the current block, the neighboring block (B) adjacent to the top of the current block, the neighboring block (C) adjacent to the top right corner of the current block, the neighboring block (D) adjacent to the bottom left corner of the current block, or the neighboring block adjacent to the top left corner of the current block. If the coordinates of the top left sample of the current block are (x0, y0), the left neighboring block A includes a sample on the (x0-1, y0+H-1) position, and the top neighboring block B includes a sample on the (x0+W-1, y0-1) position. W and H indicate the width and height of the current block, respectively. The top right neighboring block C includes a sample on the (x0+W, y0-1) position, and the bottom left neighboring block D includes a sample on the (x0-1, y0+H) position. The top left neighboring block E includes a sample on the (x0-1, y0-1) position.

[0138] When encoding a neighboring block in an affine inter prediction mode, the affine seed vector of a merge candidate may be derived based on the affine seed vector of the corresponding neighboring block. Hereinafter, the neighboring block encoded in the affine inter prediction mode is referred to as an affine neighboring block, and the merge candidate derived from the affine neighboring block is referred to as an affine merge candidate.

[0139] Affine merge candidates related to the current block may be generated by searching neighboring blocks according to a predefined scanning order. The scanning order may be predefined in the encoder and the decoder. For example, neighboring blocks may be searched according to the order of A, B, C, D, E. Merge candidates may be derived in order from the searched affine neighboring blocks. Alternatively, the scanning order may be adaptively determined based on at least one of the size, shape, or affine motion model of the current block. That is, the scanning orders of blocks having different sizes, shapes, or affine motion models are different.

[0140] Alternatively, the multiple blocks located above the current block are searched in order to derive one affine merge candidate from the previously found affine adjacent blocks, and the multiple blocks located to the left of the current block are searched in order to derive one affine merge candidate from the previously found affine adjacent blocks. The multiple adjacent blocks located above the current block may include at least one of adjacent block E, adjacent block B, or adjacent block C, and the multiple blocks located to the left of the current block may include at least one of block A or block D. In this case, adjacent block E may be further classified as a block located to the left of the current block.

[0141] Although not shown, affine merge candidates may be derived from temporal neighboring blocks of the current block. The temporal neighboring blocks may include blocks located at the same position as the current block in a co-located picture or blocks neighboring the current block. Specifically, if the temporal neighboring blocks of the current block are encoded in an affine inter prediction mode, affine merge candidates may be derived based on affine seed vectors of the temporal affine merge candidates.

[0142] A merge candidate list including affine merge candidates may be generated, and an affine seed vector of one of the merge candidates included in the merge candidate list may be determined as an affine seed vector of the current block. For this purpose, index information indicating any one of the multiple merge candidates may be encoded and transmitted via a bitstream.

[0143] As another example, multiple neighboring blocks may be searched according to the scan order, and the affine seed vector of the current block may be derived from the affine seed vectors of previously searched affine neighboring blocks.

[0144] As can be seen, in affine merge mode, the affine seed vector of the neighboring blocks may be used to derive the affine seed vector of the current block.

[0145] When the inter prediction mode of the current block is an affine motion vector prediction mode, an affine seed vector of a motion vector prediction candidate included in the motion vector prediction candidate list may be determined as an affine seed vector prediction value of the current block, and an affine seed vector difference may be added to the affine seed vector prediction value to derive an affine seed vector of the current block.

[0146] An affine seed vector prediction candidate may be derived based on adjacent blocks of the current block. Specifically, a plurality of adjacent blocks located above the current block may be searched according to a predetermined scanning order, and a first affine seed vector prediction candidate may be derived from the first searched affine adjacent block. Alternatively, a plurality of adjacent blocks located to the left of the current block may be searched according to a predetermined scanning order, and a second affine seed vector prediction candidate may be derived from the first searched affine adjacent block.

[0147] Information for determining the affine seed vector difference may be encoded and transmitted via the bitstream. The information may include magnitude information indicating the magnitude of the affine seed vector difference and a sign signal indicating the sign of the affine seed vector difference. The affine seed vector difference associated with each control point may be set to be the same. Alternatively, a different affine seed vector difference may be set for each control point.

[0148] As described above, the affine seed vector of the affine merge candidate or the affine seed vector prediction candidate may be derived from the affine seed vector of the affine neighboring block, and the affine seed vector of the current block may be derived using the affine seed vector of the derived affine merge candidate or the affine seed vector prediction candidate. Alternatively, after searching for a plurality of affine neighboring blocks according to a predetermined scanning order, the affine seed vector of the current block may be derived from the affine seed vector of the previously searched affine neighboring block.

[0149] A method for deriving an affine seed vector, an affine merge candidate, or an affine seed vector prediction candidate of a current block from an affine seed vector of an affine neighboring block will be described in detail below. In the embodiments described below, deriving an affine seed vector of a current block may be understood as deriving an affine seed vector of an affine merge candidate, or may further be understood as deriving an affine seed vector of an affine seed vector prediction candidate.

[0150] FIG. 12 is a diagram illustrating deriving an affine seed vector for a current block based on the affine seed vectors of affine neighboring blocks.

[0151] When a first affine seed vector nv0 associated with the top-left control point and a second affine seed vector nv1 associated with the top-right control point are stored in the affine neighboring block, a third affine seed vector nv2 associated with the bottom-left control point of the affine neighboring block may be derived based on the first and second affine seed vectors. Equation 4 shows an example of deriving the third affine seed vector.

[0152] Equation 4

number

[0153] In equation 4, (nv 0x ,nv 0y ) denotes the first affine seed vector nv0, and (nv 1x ,nv 1y ) denotes the second affine seed vector nv1, and (nv 2x ,nv 2y ) denotes the third affine seed vector nv2. Also, (x n0 ,x n0 ) indicates the position of the first control point, and (x n1 ,x n1 ) indicates the position of the second control point, and (x n2 ,x n2 ) indicates the location of the third control point.

[0154] An affine seed vector for the current block may then be derived using the first affine seed vector, the second affine seed vector, and the third affine seed vector. Equation 5 shows an example of deriving the first affine seed vector v0 for the current block, and Equation 6 shows an example of deriving the second affine seed vector v1 for the current block.

[0155] equation 5

number

[0156] equation 6

number

[0157] In Equation 5 and Equation 6, (v 0x ,v 0y ) denotes the first affine seed vector sv0 of the current block, and (v 1x ,v 1y ) indicates the second affine seed vector sv1 of the current block. Also, (x0, y0) indicates the position of the first control point, and (x1, y1) indicates the position of the second control point. For example, the first control point indicates the upper left corner of the current block, and the second control point indicates the upper right corner of the current block.

[0158] In the above example, three affine seed vectors associated with the affine neighboring blocks are used to derive the multiple affine seed vectors of the current block. In another example, two of the multiple affine seed vectors of the affine neighboring blocks may be used to derive the affine seed vector of the current block.

[0159] Alternatively, multiple affine seed vectors for the current block may be derived using a fourth affine seed vector associated with the lower right corner, without using the first affine seed vector in the upper left corner, the second affine seed vector in the upper right corner, or the third affine seed vector in the lower left corner associated with the affine neighboring blocks.

[0160] In particular, when the upper boundary of the current block touches the upper boundary of the coding tree unit and the affine seed vector of the upper control point (e.g., the upper left corner or the upper right corner) of the affine neighboring block (hereinafter referred to as the upper affine neighboring block) adjacent to the upper side of the current block is to be used, it is necessary to store them in memory in advance. This may cause a problem of increasing the number of row buffer areas. Therefore, when the upper boundary of the current block touches the upper boundary of the coding tree unit, it may be set to use the affine seed vector of the lower control point (e.g., the lower left corner or the lower right corner) instead of using the affine seed vector of the upper control point for the upper affine neighboring block. For example, the third affine seed vector associated with the lower left corner and the fourth affine seed vector associated with the lower right corner of the upper affine neighboring block may be used to derive multiple affine seed vectors of the current block. In such a case, the affine seed vector associated with the lower corner may be derived by copying the affine seed vector associated with the upper corner, or may be derived from the affine seed vectors associated with multiple upper corners. For example, the first affine seed vector, the second affine seed vector or the third affine seed vector may be transformed / permuted with a fourth affine seed vector associated with the bottom right corner.

[0161] Equation 7 and Equation 8 show an example of deriving the first and second affine seed vectors of the current block using the third affine seed vector associated with the bottom-left control point and the fourth affine seed vector associated with the bottom-right control point of the neighboring affine vectors.

[0162] Equation 7

number

[0163] equation 8

number

[0164] In Equation 7 and Equation 8, (x n2 ,y n2 ) denotes the coordinates of the lower-left control point of the affine neighbor block, and (x n3 ,y n3 ) indicates the coordinates of the bottom right control point of the affine neighboring block. (x0,y0) indicates the coordinates of the top left control point of the current block, and (x1,y1) indicates the coordinates of the top right control point of the current block. (nv 2x ,nv 2y ) denotes the affine seed vector of the lower-left control point of the affine neighboring block (i.e., the third affine seed vector), and (nv 3x ,nv 3y ) denotes the affine seed vector of the lower-right control point of the affine neighboring block (i.e., the fourth affine seed vector). 0x ,v 0y ) denotes the affine seed vector of the top-left control point of the current block (i.e., the first affine seed vector), and (v 1x ,v 1y ) denotes the affine seed vector of the top-right control point of the current block (i.e., the second affine seed vector).

[0165] Furthermore, the division operations in Equation 7 and Equation 8 may be changed to shift operations. The shift operation is the width between the lower-left control point and the lower-right control point (i.e., (x n3 -x n2 )) may be performed based on values ​​derived by

[0166] In the above example, a plurality of affine seed vectors of the current block may be derived based on a plurality of affine seed vectors of the affine neighboring blocks after encoding / decoding. For this purpose, a plurality of affine seed vectors of the affine neighboring blocks after encoding / decoding must be stored in memory. However, in addition to a plurality of translational motion vectors (i.e., a plurality of affine vectors) of a plurality of sub-blocks included in the affine neighboring blocks, a plurality of affine seed vectors of the affine neighboring blocks are further stored in memory, which causes a problem of increased memory usage. To eliminate such a problem, an affine seed vector of the current block may be derived using a motion vector of a sub-block adjacent to a control point of the affine neighboring block, to replace the affine seed vector of the affine neighboring block. That is, the motion vector of the sub-block adjacent to the control point of the affine neighboring block may be set as the affine seed vector of the affine neighboring block. The sub-block may be a block having a size / shape predefined in the encoder and the decoder, and may be a block that stores a basic size / shape of the motion vector. For example, the sub-block may be a square block of size 4×4. Alternatively, a motion vector specifying the sample position may be set as the affine seed vector of the affine neighboring block.

[0167] FIG. 13 is a diagram showing an example in which the motion vector of a sub-block is set as the affine seed vector of an affine adjacent block.

[0168] The motion vector of a sub-block adjacent to a control point may be set as the affine seed vector of the corresponding control point. For example, in the example shown in FIG. 13, the motion vector (nv 4x ,nv 4y ) to the affine seed vector (nv 2x ,nv 2y ), and the motion vector (nv 5x ,nv 5y) to the affine seed vector (nv 3x ,nv 3y ) of the adjacent affine block. n2 ,y n2 ) adjacent samples (e.g., (x n2 ,y n2 −1) of the adjacent affine block, and the lower-right sub-block is the sub-block that contains the lower-right control point (x n3 ,y n3 ) adjacent samples (e.g., (x n3 -1,y n3 When deriving the affine seed vector of the current block based on Equation 7 and Equation 8, the third affine seed vector of the affine neighboring block may be replaced with the motion vector of the lower-left sub-block, and the fourth affine seed vector of the affine neighboring block may be replaced with the motion vector of the lower-right sub-block.

[0169] Hereinafter, in the embodiments described below, the sub-blocks of an affine seed vector that are used as affine neighboring blocks are referred to as affine sub-blocks.

[0170] According to an embodiment of the present invention, an affine sub-block may be determined based on a sample at a specific position. For example, a sub-block including a sample at a specific position may be set as an affine sub-block. Hereinafter, the sample at the specific position is referred to as an affine reference sample. In addition, the reference sample of the affine sub-block for determining the bottom-left control point is referred to as the bottom-left reference sample, and the reference sample of the affine sub-block for determining the bottom-right control point is referred to as the bottom-right reference sample.

[0171] The lower-left reference sample and the lower-right reference sample may be selected from a plurality of samples included in the affine neighboring blocks. For example, at least one of the upper-left sample, the lower-left sample, the upper-right sample, or the lower-left sample of the lower-left sub-block may be set as the lower-left reference sample, and at least one of the upper-left sample, the lower-left sample, the upper-right sample, or the lower-left sample of the lower-right sub-block may be set as the lower-right reference sample. Thus, the motion vectors of the lower-left sub-block including the lower-left reference sample and the lower-right sub-block including the lower-right reference sample may be set as the affine seed vector associated with the lower-left control point and the affine seed vector associated with the lower-right control point, respectively.

[0172] As another example, at least one of the lower-left reference sample and the lower-right reference sample may be set as a sample located outside the affine adjacent block, which will be described in detail with reference to FIGS.

[0173] 14 to 16 are diagrams showing the positions of the reference samples.

[0174] In the example shown in part (a) of FIG. 14, for the bottom-left control point, the top-left sample of the bottom-left sub-block is used as the reference sample (x n4 ,y n4 ) can be set as the reference sample (x n4 ,y n4 ) may be set as the affine sub-block associated with the lower-left control point.

[0175] For the bottom-right control point, the sample located to the right of the top-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be set as the reference sample (x n5 ,y n5 ) may be set as the affine sub-block associated with the lower-right control point.

[0176] Alternatively, in the example shown in part (b) of FIG. 14, for the bottom-left control point, the sample located to the left of the top-left sample of the bottom-left sub-block is used as the reference sample (x n4 ,y n4 ) can be set as the reference sample (x n4 ,y n4 ) may be set as the affine sub-block associated with the lower-left control point.

[0177] For the bottom-right control point, the top-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be set as the reference sample (x n5 ,y n5 ) may be set as the affine sub-block associated with the lower-right control point.

[0178] Alternatively, in the example shown in part (a) of FIG. 15, for the bottom-left control point, the bottom-left sample of the bottom-left sub-block is used as the reference sample (x n4 ,y n4 ) can be set as the reference sample (x n4 ,y n4 ) may be set as the affine sub-block associated with the lower-left control point.

[0179] For the bottom-right control point, the sample located to the right of the bottom-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be set as the reference sample (x n5 ,y n5 ) may be set as the affine sub-block associated with the lower-right control point.

[0180] Alternatively, in the example shown in part (b) of FIG. 15, for the bottom-left control point, the sample located to the left of the bottom-left sample of the bottom-left sub-block is used as the reference sample (x n4 ,y n4) can be set as the reference sample (x n4 ,y n4 ) may be set as the affine sub-block associated with the lower-left control point.

[0181] For the bottom-right control point, the bottom-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be set as the reference sample (x n5 ,y n5 ) may be set as the affine sub-block associated with the lower-right control point.

[0182] Alternatively, in the example shown in part (a) of FIG. 16, for the bottom-left control point, a sample located between the top-left sample and the bottom-left sample of the bottom-left sub-block (e.g., the left middle sample) is used as the reference sample (x n4 ,y n4 ) can be set as the reference sample (x n4 ,y n4 ) may be set as the affine sub-block associated with the lower-left control point.

[0183] For the lower-right control point, the sample to the right of the sample located between the upper-right sample and the lower-right sample of the lower-right sub-block (e.g., the right middle sample) is set as the reference sample (x n5 ,y n5 ) can be set as the reference sample (x n5 ,y n5 ) may be set as the affine sub-block associated with the lower-right control point.

[0184] Alternatively, in the example shown in part (b) of FIG. 16, for the bottom-left control point, the sample to the left of the sample located between the top-left sample and the bottom-left sample of the bottom-left sub-block is set as the reference sample (x n4 ,y n4 ) can be set as the reference sample (x n4 ,yn4 ) may be set as the affine sub-block associated with the lower-left control point.

[0185] For the bottom-right control point, the sample located between the top-right sample and the bottom-right sample of the bottom-right sub-block is the reference sample (x n5 ,y n5 ) can be set as the reference sample (x n5 ,y n5 ) may be set as the affine sub-block associated with the lower-right control point.

[0186] When deriving a plurality of affine seed vectors for the current block based on Equation 7 and Equation 8, the third affine seed vector of the affine neighboring block may be replaced with the motion vector of the affine sub-block associated with the bottom-left control point, the fourth affine seed vector may be replaced with the motion vector of the affine sub-block associated with the bottom-right control point, the position of the bottom-left control point may be replaced with the position of the bottom-left reference sample, and the position of the bottom-right control point may be replaced with the position of the bottom-right reference sample.

[0187] Unlike the contents described in FIG. 14 to FIG. 16, a sub-block including a sample adjacent to the reference sample may be set as an affine sub-block. Specifically, a sample located outside the affine adjacent sub-block may be set as a reference sample, or a sub-block included in the affine adjacent block may be set as an affine sub-block. For example, in the example shown in FIG. 14(a), a sample located to the right of the upper right sample of the lower right sub-block is set as the reference sample (x n5 ,y n5 ) and the bottom right sub-block may be set as the affine sub-block associated with the bottom right corner. Alternatively, in the example shown in part (b) of FIG. 14, the sample located to the left of the top left sample of the bottom left sub-block may be set as the reference sample (x n4 ,y n4), and the bottom-left sub-block may be set as the affine sub-block associated with the bottom-left corner.

[0188] The embodiments illustrated in Figures 15 and 16 may be applied in a similar manner. That is, in the example shown in part (a) of Figure 15 or part (a) of Figure 16, the sample located to the right of the lower right sample of the lower right sub-block or the right middle sample of the lower right sub-block is used as the reference sample (x n5 ,y n5 ) and the bottom right sub-block may be set as the affine sub-block associated with the bottom right corner. Alternatively, in the example shown in part (b) of FIG. 15 or part (b) of FIG. 16, the bottom left sample of the bottom left sub-block or the sample located to the left of the left middle sample may be set as the reference sample (x n4 ,y n4 ), and the bottom-left sub-block may be set as the affine sub-block associated with the bottom-left corner.

[0189] In the above example, the motion vectors of the affine sub-blocks may be used to derive the affine seed vectors of the affine neighboring blocks, and therefore the motion vectors may be stored on a sub-block basis for the encoded / decoded block.

[0190] As another example, after a minimum number of affine seed vectors are stored for an affine neighboring block, the stored affine seed vectors may be used to derive motion vectors for the affine sub-blocks.

[0191] Equation 9 and Equation 10 show an example of deriving the motion vector of an affine sub-block using the affine seed vectors of the affine neighboring blocks.

[0192] equation 9

number

[0193] Equation 10

number

[0194] In Equation 9 and Equation 10, (nv 4x ,nv 4y ) denotes the motion vector of the affine subblock relative to the lower-left control point, and (nv 5x ,nv 5y ) denotes the motion vector of the affine subblock associated with the bottom-right control point. The motion vector of the affine subblock and the affine seed vector of the control point are set to be the same, so that (nv 4x ,nv 4y ) is the affine seed vector (nv 2x ,nv 2y ) or (nv 5x ,nv 5y ) is the affine seed vector (nv 3x ,nv 3y ) may be substituted.

[0195] (x n4 ,y n4 ) indicates the position of the reference sample of the lower left sub-block. Alternatively, the position may be replaced with the center position of the lower left sub-block or the position of the lower left control point. (x n5 ,y n5 ) indicates the position of the reference sample of the lower right sub-block. Alternatively, the position may be further replaced with the center position of the lower right sub-block or the position of the lower right control point.

[0196] If the current block does not touch the boundary of the coding tree unit, Equation 9 and Equation 10 may be applied. If the current block touches the upper boundary of the coding tree unit, instead of using Equation 9 and Equation 10, the translational motion vector of the affine sub-block determined based on the lower-left reference sample may be set as the third affine seed vector, and the translational motion vector of the affine sub-block determined based on the lower-right reference sample may be set as the fourth affine seed vector.

[0197] In Equation 7 and Equation 8, (xn3 -x n2 ) indicates the width between the bottom left control point and the bottom right control point. n3 is the position of the lower right reference sample x n5 may be substituted with x n2 is the position of the lower left reference sample x n4 In the following, (x n3 -x n2 ) or the value obtained by substituting the position of the above equation with the position of the reference sample (e.g., (x n5 -x n4 )) to variable W seed where the variable is called the subseed vector width.

[0198] Depending on the position of the reference sample, the sub-seed vector width may not be a power series of 2 (e.g., 2n). For example, when the lower-left sample of the lower-left sub-block is set as the lower-left reference sample and the lower-right sample of the lower-right sub-block is set as the lower-right reference sample, the width of the sub-seed vector is not a multiple of 2. As described above, if the sub-seed vector width is not a power series of 2, the sub-seed vector width may be converted to a power series of 2. The conversion may include adding an offset amount to the sub-seed vector width, subtracting an offset amount from the sub-seed vector width, or substituting the position of the reference sample with the position of a sample adjacent to the reference sample. For example, the converted sub-seed vector width may be derived by adding 1 to the width between the lower-left reference sample and the lower-right reference sample. Alternatively, the width between the adjacent reference sample adjacent to the right of the lower-right reference sample and the lower-left reference sample may be set as the converted sub-seed vector width. The converted sub-seed vector width may then be substituted into Equation 7 and Equation 8 to derive the affine seed vector of the current block.

[0199] Additionally, the division operations included in Equation 7 and Equation 8 may be changed to shift operations. The shift operations may be performed based on a value derived from the transformed subseed vector width (i.e., a value expressed as a power series of 2).

[0200] When the reference sample for determining the affine sub-block does not belong to the affine adjacent block, the affine seed vector of the affine adjacent block may be derived based on a sample adjacent to the reference sample among the multiple samples included in the affine adjacent block. Specifically, the translational motion vector of a sub-block including a sample adjacent to the reference sample in the affine adjacent block (hereinafter referred to as an adjacent reference sample) may be set as the affine seed vector of the affine adjacent block. As described above, the method of deriving the affine seed vector using the adjacent reference sample may be defined as a modified affine merge vector derivation method.

[0201] FIG. 17 is a diagram showing an application example of the modified affine merge vector derivation method.

[0202] The lower-right reference sample (x n5 ,y n5 ) does not belong to the affine neighboring block, the sample (x n5 -1,y n5 ) may be used to derive an affine seed vector. Specifically, the neighboring reference samples (x n5 -1,y n5 ) may be set as the affine seed vector of the bottom-right control point.

[0203] In the example shown in Figure 17, the right-neighboring sample of the top-right sample of the bottom-right sub-block is indicated as the bottom-right reference sample. When the right-neighboring sample of the bottom-right sample of the bottom-right sub-block or the right-neighboring sample of the right-middle sample of the bottom-right sub-block is set as the bottom-right reference sample, an affine seed vector may be derived based on the left-neighboring sample of the neighboring reference sample.

[0204] Also, if the bottom-left reference sample does not belong to an affine neighboring block, the affine seed vector may be derived based on the neighboring sample to the right of the bottom-left reference sample according to the embodiment described.

[0205] By setting the positions of the reference samples and the sub-blocks for deriving the affine seed vectors in different manners, the sub-seed vector width may be set as a power series of two.

[0206] Alternatively, if the motion vectors of the bottom-left and bottom-right sub-blocks are available, multiple affine seed vectors may be derived based on the widths of the neighboring blocks.

[0207] FIG. 18 is a diagram showing an example of deriving an affine seed vector of an affine merge candidate based on a plurality of motion vectors of a plurality of sub-blocks belonging to an adjacent block.

[0208] If the upper boundary of the current block touches the boundary of a coding tree unit, the affine seed vector of the affine merge candidate may be derived based on the motion vectors of the lower-left and lower-right sub-blocks of the upper neighboring block located above the current block. Assuming that the position of the upper-left sample of the upper neighboring block is (xNb, yNb), and the width and height of the upper neighboring block are NbW and NbH, respectively, the lower-left sub-block may include a sample at the (xNb, yNb+yNbH-1) position, and the lower-right sub-block may include a sample at the (xNb+NbW-1, yNb+NbH-1) position.

[0209] A plurality of affine seed vectors of the affine merge candidate may be derived based on the width of the adjacent block and the coordinate difference between the adjacent block and the current block. For example, the plurality of affine seed vectors of the affine merge candidate may be derived based on the following equations 11 to 13.

[0210] Equation 11

number

[0211] Equation 12

number

[0212] Equation 13

number

[0213] In the above equations 11 to 13, (v 0x ,v 0y ) denotes the first affine seed vector, and (v 1x ,v 1y ) denotes the second affine seed vector, and (v 2x ,v 2y ) denotes the third affine seed vector. Vlbx denotes the horizontal component motion vector of the lower left sub-block, and Vlby denotes the vertical component motion vector of the lower left sub-block. Vrbx denotes the horizontal component motion vector of the lower right sub-block, and Vrby denotes the vertical component motion vector of the lower right sub-block. NbW denotes the width of the neighboring block, and xCbW and xCbH denote the width and height of the current block, respectively. xCb denotes the x-coordinate of the top left sample of the current block, and xNb denotes the x-coordinate of the top left sample of the neighboring block.

[0214] Further, the affine merge candidate may be generated by combining the multiple affine seed vectors derived from the first neighboring block with the motion vector of the second neighboring block. For example, in the example shown in Figure 18, instead of deriving the third affine seed vector of the affine merge candidate based on the motion vectors of the lower left sub-block and the lower right sub-block, the third affine seed vector of the affine merge candidate may be further derived based on the motion vector of the sub-block located to the left of the current block.

[0215] FIG. 19 is a diagram illustrating an example of deriving an affine seed vector of an affine merge candidate based on the motion vectors of multiple sub-blocks located to the left of the current block.

[0216] An affine seed vector of the affine merge candidate may be derived based on the motion vectors of the lower left and lower right sub-blocks of an upper neighboring block located above the current block. Specifically, a first affine seed vector and a second affine seed vector of the affine merge candidate may be derived based on the motion vectors of the lower left and lower right sub-blocks. For example, the first affine seed vector and the second affine seed vector of the affine merge candidate may be derived based on Equation 11 and Equation 12.

[0217] A third affine seed vector of the affine merge candidate may be derived based on the motion vector of the neighboring block located to the left of the current block. For example, the motion vector of the neighboring block adjacent to the left of the current block or the neighboring block adjacent to the bottom left corner of the current block may be set as the third affine seed vector. The neighboring block adjacent to the left of the current block may include a sample at the (xCb-1, yCb+CbH-1) position, and the neighboring block adjacent to the bottom left corner of the current block may include a sample at the (xCb-1, yCb+CbH) position. Equation 14 and Equation 15 show an example of setting the motion vector of the neighboring block located to the left of the current block as the third affine seed vector.

[0218] Equation 14

number

[0219] Equation 15

number

[0220] In Equation 14, V Leftxdenotes the horizontal motion vector of the neighboring block on the left side of the current block, and V Lefty V denotes the vertical motion vector of the neighboring block on the left side of the current block. In Equation 15, V LBx denotes the horizontal motion vector of the neighboring block adjacent to the lower-left corner of the current block, and V LBy denotes the vertical motion vector of the neighboring block adjacent to the lower left corner of the current block.

[0221] Only if the motion vector of the neighboring block adjacent to the left side of the current block is unavailable, the motion vector of the neighboring block adjacent to the bottom left corner of the current block can be used to derive the third affine seed vector.

[0222] Furthermore, the method of deriving the third affine seed vector of different affine merge candidates may be set according to whether the motion vector of the neighboring block located to the left of the current block is available. For example, if the motion vector of the neighboring block located to the left of the current block (e.g., the sub-block adjacent to the left of the current block or the sub-block adjacent to the lower left corner of the current block) is available, the third affine seed vector may be derived using Equation 14 or Equation 15. On the other hand, if the motion vector of the neighboring block located to the left of the current block is unavailable, the third affine seed vector may be derived using Equation 13.

[0223] A third affine seed vector may further be derived based on a motion vector of a non-adjacent block that is not adjacent to the current block in the reference sample. To derive the third affine seed vector, it may be determined whether to use the motion vector of the neighboring block that is adjacent to the current block in the reference sample or the motion vector of the non-adjacent block that is not adjacent to the current block in the reference sample based on the position of the upper neighboring block used in deriving the affine merge candidate.

[0224] FIG. 20 is a diagram illustrating an example of deriving an affine seed vector of an affine merge candidate based on motion information of a non-adjacent block or an adjacent block located to the left of the current block.

[0225] The position of the block for deriving the third affine seed vector may be determined by comparing the position of the top left sample of the upper neighboring block for deriving the affine merge candidate with the position of the top left sample of the current block. For example, in the example shown in part (a) of FIG. 20, if the x coordinate (xNb) of the top left sample of the upper neighboring block is smaller than the x coordinate (xCb) of the top left sample of the current block, the third affine seed vector may be derived based on the motion vector of a non-adjacent block that is not adjacent to the left side of the current block. Specifically, the third affine seed vector may be derived based on the motion vector of a left non-adjacent block including a sample (xNb, yCb+CbH-1) having the same x coordinate as the top left sample of the upper neighboring block or a bottom left non-adjacent block including a sample at the (xNb, yCb+CbH) position. Equation 16 and Equation 17 show an example of deriving the third affine seed vector based on the motion vector of a non-adjacent block.

[0226] Equation 16

number

[0227] Equation 17

number

[0228] In Equation 16, V Left2x denotes the horizontal motion vector of the left non-adjacent block, and V Left2y In Equation 17, V LB2x denotes the horizontal motion vector of the lower-left non-adjacent block, and V LB2y denotes the vertical motion vector of the lower-left non-adjacent block.

[0229] Only if the motion vector of the left non-adjacent block is unavailable can the motion vector of the lower-left non-adjacent block be used to derive the third affine seed vector.

[0230] In the example shown in part (b) of Figure 20, if the x coordinate (xNb) of the top left sample of the upper neighboring block is the same as the x coordinate (xCb) of the top left sample of the current block, or if the x coordinate (xNb) of the top left sample of the upper neighboring block is greater than the x coordinate (xCb) of the top left sample of the current block, the third affine seed vector may be derived based on the motion vector of the neighboring block adjacent to the left or bottom left corner of the current block. Specifically, the third affine seed vector may be derived based on the motion vector of the left neighboring block including the sample (xCb, yCb+CbH-1) adjacent to the left of the current block, or the motion vector of the bottom left neighboring block including the sample (xCb, yCb+CbH) adjacent to the bottom left corner of the current block.

[0231] Furthermore, the positions of the reference samples of the left adjacent block or the left non-adjacent block may be set in a manner different from that shown in Figures 18 to 20. For example, a block including a sample (xCb-1, yCb+subH-1) adjacent to the current block may be set as the left adjacent block, and a block including a sample (xNb, yCb+subH-1) not adjacent to the current block may be set as the left non-adjacent block. subH indicates the minimum height of the block storing the motion information and may be an integer such as 2, 4, or 8.

[0232] FIG. 21 is a diagram showing the positions of blocks for deriving affine seed vectors of affine merge candidates.

[0233] For example, in the example shown in part (a) of FIG. 21, if the x-coordinate (xNb) of the top left sample of the upper neighboring block is smaller than the x-coordinate (xCb) of the top left sample of the current block, the third affine seed vector may be derived based on the motion vector of a non-neighboring block that is not adjacent to the left side of the current block. Specifically, the third affine seed vector may be derived based on the motion vector of a left non-neighboring block that includes a sample (xNb, yCb+subH-1) that has the same x-coordinate as the top left sample of the upper neighboring block and is a predetermined distance (e.g., subH) away from the bottom left sample of the upper neighboring block. Equation 18 shows an example of deriving the third affine seed vector based on the motion vector of a non-neighboring block.

[0234] Equation 18

number

[0235] In Equation 18, V LeftT2x V denotes the horizontal motion vector of the left non-adjacent block containing a sample that is a given distance away from the bottom-left sample of the upper adjacent block. LeftT2y denotes the vertical motion vector of the left non-adjacent block containing a sample that is a given distance away from the bottom-left sample of the upper adjacent block.

[0236] Alternatively, in the example shown in part (b) of Figure 21, if the x-coordinate (xNb) of the top-left sample of the upper neighboring block is the same as the x-coordinate (xCb) of the top-left sample of the current block, or if the x-coordinate (xNb) of the top-left sample of the upper neighboring block is greater than the x-coordinate (xCb) of the top-left sample of the current block, the third affine seed vector may be derived based on the motion vector of the left neighboring block including a sample (xCb-1, yCb+subH-1) that is adjacent to the left of the current block and is a predetermined distance (e.g., subH-1) away from the top-left sample of the current block along the vertical direction. Equation 19 shows an example of deriving the third affine seed vector based on the motion vector of the neighboring block.

[0237] Equation 19

number

[0238] In Equation 19, V LeftTx V denotes the horizontal motion vector of the left neighboring block, which is adjacent to the left of the current block and includes a sample that is a predetermined distance away from the top-left sample of the current block along the vertical direction. LeftTy denotes the vertical motion vector of a left neighboring block that is adjacent to the left of the current block and includes a sample that is a predetermined distance away from the top-left sample of the current block along the vertical direction.

[0239] Alternatively, the reference sample may be set at a position different from the positions shown in Figures 18 to 21. For example, a sample (xCb-1, yCb) adjacent to the left side of the current block and having the same y coordinate as the top left sample of the current block, or a sample (xNb, yCb) not adjacent to the left side of the current block and having the same y coordinate as the top left sample of the current block, may be set as the reference sample.

[0240] In the illustrated example, the first and second affine seed vectors of the affine merge candidate may be derived from an adjacent block located above the current block, and the third affine seed vector of the affine merge candidate may be derived from an adjacent block located to the left of the current block. The first and third affine seed vectors of the affine merge candidate may be derived from an adjacent block located to the left of the current block, and the second affine seed vector of the affine merge candidate may be derived from an adjacent block located above the current block.

[0241] In the example shown in Equation 11 to Equation 13, the motion vector V LB and the motion vector V of the lower right subblock RBIn such a case, instead of the motion vector of the lower-left sub-block or the motion vector of the lower-right sub-block, the affine seed vector of the affine merge candidate may be derived using the motion vector of another block. For example, the motion vector V of the lower-left sub-block of the upper neighboring block may be derived using the motion vector of the lower-left sub-block of the upper neighboring block. LB and the motion vector V of the lower right sub-block RB If the values ​​of are the same, then the motion vector of the block of samples located to the right or below the lower right sub-block is V RB For example, the motion vector V of the bottom right sub-block may be RB Alternatively, the motion vector of the block containing the sample at the (xNb+NbW, yNb+NbH-1) location may be used to derive the motion vector of the affine merge candidate.

[0242] Alternatively, the motion vector V of the lower left sub-block of the upper neighboring block LB and the motion vector V of the lower right sub-block RB If the values ​​of are the same, the motion vector of the block containing the sample located to the left or below the lower-left subblock is V LB For example, the motion vector V of the bottom left sub-block may be LB Alternatively, the motion vector of the block containing the sample at the (xNb, yNb+NbH-1+offset) position may be used to derive the motion vector of the affine merge candidate. The offset amount may be an integer equal to or greater than 1.

[0243] Alternatively, the VRB may be replaced by a motion vector of a block containing samples located to the left or below the lower-left sub-block, or the VLB may be replaced by a motion vector of a block containing samples located to the right or below the lower-right sub-block.

[0244] Furthermore, a merge candidate may be derived by combining multiple motion vectors of multiple adjacent blocks adjacent to the current block. A merge candidate obtained by combining multiple motion vectors of multiple adjacent blocks may be referred to as a combined merge candidate. When deriving a combined merge candidate, an affine seed vector of the control point may be derived from one of the multiple adjacent blocks adjacent to the control point.

[0245] FIG. 22 is a diagram for explaining an example of deriving combined merge candidates by combining a plurality of motion vectors of a plurality of adjacent blocks.

[0246] The neighboring blocks may include an upper neighboring block located above the current block and a left neighboring block located to the left of the current block. The upper neighboring block may be at least one of a block B0 including a sample on the (xCb+CbW, yCb-1) position, a block B1 including a sample on the (xCb+CbW-1, yCb-1) position, a block B2 including a sample on the (xCb-1, yCb-1) position, or a block B3 including a sample on the (xCb, yCb-1) position. (xCb, yCb) indicates the position of the top left sample of the current block, and CbW indicates the width of the current block. The left neighboring block may be at least one of a block A0 including a sample on the (xCb-1, yCb+CbH) position, a block A1 including a sample on the (xCb-1, yCb+CbH-1) position, or a block A2 including a sample on the (xCb-1, yCb) position. CbH indicates the height of the current block.

[0247] Additionally, the neighboring blocks may further include time neighboring blocks of the current block.

[0248] The motion information of two or more of the adjacent blocks may be combined to derive the combined merge candidate. The motion information of two or three adjacent blocks may be combined to derive the combined merge candidate. Specifically, the combined merge candidate may be further derived by combining two or more of the motion information of the adjacent block adjacent to the first control point, the motion information of the adjacent block adjacent to the second control point, or the motion information of the adjacent block adjacent to the third control point.

[0249] For example, the motion vectors of two or more of the adjacent blocks A2, B2, or B3 adjacent to the top-left control point CP0, the adjacent blocks B1 or B0 adjacent to the top-right control point CP1, or the temporal adjacent block T associated with the bottom-left control point CP2 may be combined to derive a combined merge candidate.

[0250] In such a case, when scanning a plurality of neighboring blocks adjacent to a control point according to a predefined scanning order, a motion vector of an available neighboring block retrieved first may be set as an affine seed vector of the control point. For example, a first affine seed vector of a combined merge candidate associated with CP0 may be set as a motion vector of an available neighboring block retrieved first when searching a plurality of neighboring blocks according to the order of B2, B3, and A2. A second affine seed vector of a combined merge candidate associated with CP1 may be set as a motion vector of an available neighboring block retrieved first when searching a plurality of neighboring blocks according to the order of B1 and B0. A third affine seed vector of a combined merge candidate associated with CP2 may be set as a motion vector of an available neighboring block retrieved first when searching a plurality of neighboring blocks according to the order of A1 and A0.

[0251] When deriving combined merge candidates, neighboring blocks coded with an affine motion model or neighboring blocks derived by affine merge candidates may be set to be unusable.

[0252] FIG. 23 shows that adjacent blocks cannot be used.

[0253] When deriving an affine merge candidate from a neighboring block B1 located above the current block, it may be determined that the neighboring block B1 cannot be used to derive a combined merge candidate, and therefore, the motion vector of B1 may not be used when deriving an affine seed vector for a combined merge candidate associated with CP1.

[0254] Alternatively, when encoding a neighboring block with an affine motion model, the combined merge candidate may be derived based on the affine vector of the sub-block containing the reference sample. For example, when encoding a neighboring block containing a sample at the B1 position with an affine motion model, the combined merge candidate may be derived using the affine vector of the sub-block containing the sample at the B1 position of the neighboring block.

[0255] Alternatively, the scanning order may be determined after considering whether the neighboring block is coded with an affine motion model or whether the neighboring block is derived by an affine merge candidate. For example, the scanning order of the neighboring block coded with an affine motion model or the neighboring block derived by an affine merge candidate may be set to last. For example, when the neighboring block on the B1 position is coded with an affine motion model, the second affine seed vector of the combined merge candidate associated with CP2 may be derived by searching multiple neighboring blocks according to the order of B0 and B1.

[0256] If the reference picture indexes between multiple neighboring blocks are different, the motion vector may be scaled based on the reference picture with the smallest index or the reference picture with the largest index. The scaling may be performed based on the picture order count (POC) difference between the reference picture and the current picture.

[0257] Alternatively, the combination of the multiple neighboring blocks may be performed using only the neighboring blocks having the same reference picture index among the multiple neighboring blocks. For example, if at least one reference picture index among the multiple neighboring blocks is different, the combination may be set to be unavailable as a combination merge candidate. Also, the availability of the combination may be determined regardless of whether the motion information or motion vectors of the neighboring blocks are the same. For example, if the motion vectors (e.g., the CP0 affine seed vector and the CP1 affine seed vector) of the multiple neighboring blocks are the same, the multiple neighboring blocks having the same reference picture index may be combined to derive a combination merge candidate. Alternatively, the scanning order of the multiple neighboring blocks may be determined after considering the reference picture indexes of the multiple neighboring blocks. Alternatively, the combination may be performed using only the multiple neighboring blocks having the same prediction direction among the multiple neighboring blocks.

[0258] Merge candidates may be generated according to a set combination order. For example, the combination order may be 1. {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector} 2. {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector} 3. {CP0 affine seed vector, CP2 affine seed vector, CP3 affine seed vector} 4. {CP1 affine seed vector, CP2 affine seed vector, CP3 affine seed vector} 5. {CP0 affine seed vector, CP1 affine seed vector} 6. May be set to {CP0 affine seed vector, CP2 affine seed vector}.

[0259] Although six example combinations are shown, fewer or more example combinations may be used.

[0260] A combined merge candidate generated by combining three affine seed vectors may be referred to as a six-parameter set merge candidate, and a combined merge candidate generated by combining two affine seed vectors may be referred to as a four-parameter set merge candidate.

[0261] The combination order may be predefined in the encoder and the decoder. Alternatively, the combination order may be determined based on at least one of the size, shape, partition type, affine motion model, position of the current block in the coding tree unit, or output order of reference pictures of the current block. For example, when a four-parameter affine motion model is applied to the current block, the combination examples used for the four-parameter set merging candidates may be set to have a higher priority than the combination examples used for the six-parameter set merging candidates.

[0262] Although the combined merge candidate may be generated according to the combination order, it may be determined that only the merge candidate generated by combining the multiple neighboring blocks with the same reference picture may be used. Alternatively, if at least one reference picture of the multiple neighboring blocks is different, the merge candidate may be derived by scaling the motion vector based on the reference picture with the largest index or the reference picture with the smallest index. The scaling may be performed according to the difference in the output order between the current picture and the reference picture. Alternatively, if two reference pictures in the multiple neighboring blocks are the same but other reference pictures are different, the combined merge candidate may be derived by scaling the motion vector of the other neighboring block based on the reference picture jointly applied to the two neighboring blocks.

[0263] The number of combined merge candidates added to the merge candidate list may be determined based on at least one of the number of merge candidates included in the merge candidate list or the maximum number of merge candidates. Alternatively, only 6 parameter set merge candidates may be added to the merge candidate list, or only 4 parameter set merge candidates may be added to the combined merge candidate list based on the affine motion model of the current block.

[0264] For example, when the number of combination merge candidates to be added to the merge candidate list is 1 and a 6-parameter motion model is applied to the current block, one 6-parameter set merge candidate based on the set combination order may be added to the merge candidate list. Specifically, when the CP0 affine seed vector, the CP1 affine seed vector, and the CP2 affine seed vector are all available, the combination merge candidate {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector} may be added to the merge candidate list. Alternatively, when the CP2 affine seed vector is unavailable and the CP3 affine seed vector is available, the combination merge candidate {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector} may be added to the merge candidate list.

[0265] Alternatively, if the number of combination merge candidates to be added to the merge candidate list is one and a six-parameter motion model is applied to the current block, one four-parameter set merge candidate based on the set combination order may be added to the merge candidate list.

[0266] As another example, the number of combination merge candidates to be added to the merge candidate list may be set to be different based on the affine motion model of the current block. For example, if a six-parameter motion model is applied to the current block, one combination merge candidate may be added to the merge candidate list. On the other hand, if a four-parameter motion model is applied to the current block, two combination merge candidates may be added to the merge candidate list.

[0267] A combination merge candidate to be added to the merge candidate list may be selected based on the number of available affine merge candidates. For example, if the number of available affine merge candidates is two or more, only four parameter set merge candidates may be added to the merge candidate list. On the other hand, if the number of available affine merge candidates is one or less, only six parameter set merge candidates may be added to the merge candidate list. Alternatively, if the number of available affine merge candidates is one or less, N six parameter set merge candidates and M four parameter set merge candidates may be added to the merge candidate list. N and M may be integers, and M may be derived based on the maximum number of merge candidates minus N. Alternatively, if the number of available affine merge candidates is one or less, only four parameter set merge candidates may be added to the merge candidate list.

[0268] Alternatively, the combination order may be determined based on the availability of multiple affine seed vectors. For example, taking into account the availability of multiple affine seed vectors, 1. If the CP0 affine seed vector, the CP1 affine seed vector and the CP2 affine seed vector are available {CP0 affine seed vector, CP1 affine seed vector, CP2 affine seed vector} 2. When the CP0 affine seed vector, the CP1 affine seed vector and the CP3 affine seed vector are available {CP0 affine seed vector, CP1 affine seed vector, CP3 affine seed vector} 3. When CP0 affine seed vector, CP2 affine seed vector and CP3 affine seed vector are available {CP0 affine seed vector, CP2 affine seed vector, CP3 affine seed vector} 4. When CP0 affine seed vector, CP2 affine seed vector and CP3 affine seed vector are available {CP1 affine seed vector, CP2 affine seed vector, CP3 affine seed vector} 5. When CP0 and CP1 affine seed vectors are available {CP0 affine seed vector, CP1 affine seed vector} 6. When CP0 affine seed vector and CP2 affine seed vector are available The combined merge candidates may be added to the merge candidate list according to the order {CP0 affine seed vector, CP2 affine seed vector}.

[0269] When the number of combination merge candidates that can be added to the merge candidate list is one, the combination merge candidate for which the condition among the above conditions 1 to 6 is satisfied first may be added to the merge candidate list. When the conditions 1 to 6 are not satisfied, the combination merge candidate may not be added to the merge candidate list.

[0270] As another example, the maximum number of combined merge candidates that can be added to the merge candidate list may be determined further based on the number of available affine merge candidates.

[0271] If the number of merge candidates in the merge candidate list is less than the maximum number, a zero merge candidate with a motion vector of 0 may be added to the merge candidate list. Thus, in affine merge mode, 1. Derive affine merge candidates 2. Combination Merge Candidates 3. Merge candidates may be derived according to the order of zero merge candidates.

[0272] Next, the inter prediction method using translational motion information will be described in detail.

[0273] The motion information of the current block may be derived from the motion information of another block of the current block. The other block may be a block that has been encoded / decoded by inter prediction before the current block. When the motion information of the current block is set to be the same as the motion information of another block, this may be defined as a merge mode. Also, when the motion vector of the other block is set as a predicted value of the motion vector of the current block, this may be defined as a motion vector prediction mode.

[0274] FIG. 24 is a flowchart of a process for deriving motion information of a current block in the merge mode.

[0275] Merge candidates for a current block may be derived (S2401). The merging candidates for the current block may be derived from blocks that have been encoded / decoded by inter prediction before the current block.

[0276] The candidate blocks used to derive merging candidates may include adjacent blocks, i.e., samples adjacent to the current block. For example, if the coordinates of the top-left sample of the current block are (0,0), then at least one of the block containing a reference sample on the (-1,H-1) position, the block containing a reference sample on the (W-1,-1) position, the block containing a reference sample on the (W,-1) position, the block containing a reference sample on the (-1,H) position, or the block containing a reference sample on the (-1,-1) position may be used as a candidate block.

[0277] Alternatively, a candidate block that does not belong to the same coding tree unit as the current block may be set as ineligible for use as a merging candidate, for example, a candidate block containing a reference sample may be set as ineligible for use as a merging candidate if the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs.

[0278] The merging candidates may further be derived from temporally adjacent blocks included in a picture different from the current block. For example, the merging candidates may further be derived from a co-located block included in the co-located picture. Any one of the multiple reference pictures included in the reference picture list may be set as the co-located picture. Index information for indicating the co-located picture among the multiple reference pictures may be transmitted by a signal via the bitstream. Alternatively, a reference picture having a predefined index among the multiple reference pictures may be determined as the co-located picture.

[0279] The motion information of the merging candidate may be set to be the same as the motion information of the candidate block. For example, at least one of the motion vector, the reference picture index, the prediction direction, or the bidirectional weighting value index of the candidate block may be set as the motion information of the merging candidate.

[0280] A merge candidate list may be generated that includes the merge candidates (S2402).

[0281] The indices of the multiple merging candidates in the merging candidate list may be assigned according to a predetermined order, for example, in the following order: merging candidates derived from the left neighboring block, merging candidates derived from the upper neighboring block, merging candidates derived from the upper right neighboring block, merging candidates derived from the lower left neighboring block, merging candidates derived from the upper left neighboring block, and merging candidates derived from the temporal neighboring blocks.

[0282] When the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates may be selected (S2403). Specifically, information for specifying one of the multiple merge candidates specified by a signal may be transmitted via the bit stream. For example, index information merge_idx indicating one of the multiple merge candidates included in the merge candidate list may be transmitted via the bit stream by a signal.

[0283] If the number of merging candidates included in the merging candidate list is less than a threshold, merging candidates included in the inter motion information list may be added to the merging candidate list. The threshold may be the maximum number of merging candidates that can be included in the merging candidate list or the maximum number of merging candidates minus an offset amount. The offset amount may be an integer such as 1 or 2. The inter motion information list may include merging candidates derived based on blocks that are encoded / decoded before the current block.

[0284] The inter motion information list includes merge candidates derived from blocks in the current picture that are coded / decoded by inter prediction. For example, the motion information of the merge candidates included in the inter motion information list may be set to be the same as the motion information of the blocks that are coded / decoded by inter prediction. The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weighting value index.

[0285] For convenience of explanation, the merge candidates included in the inter motion information list are referred to as inter merge candidates.

[0286] The maximum number of merging candidates that can be included in the inter motion information list in the encoder and decoder may be predefined, for example, the maximum number of merging candidates that can be included in the inter motion information list may be 1, 2, 3, 4, 5, 6, 7, 8 or more (e.g., 16).

[0287] Alternatively, information indicating the maximum number of merging candidates in the inter motion information list may be signaled via the bitstream, said information being signaled at a sequence level, a picture level or a slice level.

[0288] Alternatively, the maximum number of merge candidates in the inter motion information list may be determined based on the size of the picture, the size of the slice, or the size of the coding tree unit.

[0289] The inter motion information list may be initialized in units of pictures, slices, segments, blocks, coding tree units, or coding tree unit rows (rows or columns). For example, when a slice is initialized, the inter motion information list is also initialized, and the inter motion information list may not include any merge candidates.

[0290] Alternatively, information indicating whether to initialize the inter motion information list may be further transmitted by a signal via the bitstream, and the information may be transmitted by a signal at a slice level, a segment level, a block level, or a block level. Before the information indicates to initialize the inter motion information list, the set inter motion information list may be used.

[0291] Alternatively, information related to inter merge candidates may be sent by signaling via the picture parameter set or slice header. When a slice is initialized, the inter motion information list may also include the initial inter merge candidates. Thus, the inter merge candidates may be used for the first coded / decoded blocks in the slice.

[0292] It is also possible to encode / decode blocks according to the encoding / decoding order, and to set a plurality of blocks encoded / decoded by inter prediction according to the encoding / decoding order as inter merge candidates in order.

[0293] FIG. 25 is a diagram for explaining an example of updating the inter-motion information list.

[0294] When performing inter prediction on a current block (S2501), an inter merge candidate may be derived based on the current block (S2502). Motion information of the inter merge candidate may be set to be the same as the motion information of the current block.

[0295] If the inter motion information list is empty (S2503), an inter merge candidate derived based on the current block may be added to the inter motion information list (S2504).

[0296] If the inter-motion information list includes the inter-merge candidate (S2503), a redundancy check may be performed on the motion information of the current block (or the inter-merge candidate derived based on the current block) (S2505). The redundancy check is used to determine whether the motion information of the inter-merge candidate stored in the inter-motion information list is the same as the motion information of the current block. The redundancy check may be performed with all inter-merge candidates stored in the inter-motion information list as objects. Alternatively, the redundancy check may be performed with an inter-merge candidate whose index is greater than or less than a threshold value among the inter-merge candidates stored in the inter-motion information list as an object.

[0297] If the current block does not include an inter merge candidate having the same motion information as the current block, the inter merge candidate derived based on the current block may be added to the inter motion information list (S2508). Whether the inter merge candidates are the same may be determined based on whether their motion information (e.g., motion vectors and / or reference picture indexes, etc.) is the same.

[0298] In this case, if the maximum number of inter merge candidates is stored in the inter motion information list (S2506), the first inter merge candidate may be deleted (S2507), and an inter merge candidate derived based on the current block may be added to the inter motion information list (S2508).

[0299] A number of inter merge candidates may be identified based on each index. When an inter merge candidate derived from the current block is added to the inter motion information list, the inter merge candidate may be assigned the smallest index (e.g., 0), and the index of each stored inter merge candidate may be incremented by 1. In this case, if the maximum number of inter merge candidates is stored in the inter motion information list, the inter merge candidate with the highest index is removed.

[0300] Alternatively, when an inter merge candidate derived from the current block is added to the inter motion information list, the inter merge candidate may be assigned the largest index. For example, if the number of inter merge candidates stored in the inter motion information list is less than the maximum value, the inter merge candidate may be assigned an index whose value is equal to the number of stored inter merge candidates. Alternatively, if the number of inter merge candidates stored in the inter motion information list is equal to the maximum value, the inter merge candidate may be assigned an index whose value is the maximum value minus 1. Also, the inter merge candidate with the smallest index is removed, and 1 is subtracted from each of the indexes of the remaining stored inter merge candidates.

[0301] FIG. 26 illustrates an embodiment of updating the inter-merge candidate list.

[0302] Assume that an intermerge candidate derived from the current block is added to the intermerge candidate list and assigned the maximum index, and also assume that the maximum number of intermerge candidates is stored in the intermerge candidate list.

[0303] When adding the inter merge candidate HmvpCand[n+1] derived from the current block to the inter merge candidate list HmvpCandList, the inter merge candidate HmvpCand[0] with the smallest index is deleted from the stored inter merge candidates, and 1 is subtracted from the indexes of each of the remaining inter merge candidates. Also, the index of the inter merge candidate HmvpCand[n+1] derived from the current block may be set as the maximum value (n in the example shown in FIG. 26).

[0304] If an inter merge candidate that is the same as the inter merge candidate derived based on the current block is stored (S2505), the inter merge candidate derived based on the current block may not be added to the inter motion information list (S2509).

[0305] Alternatively, as an inter merge candidate derived based on the current block is added to the inter motion information list, stored inter merge candidates that are the same as the inter merge candidate may be removed, causing the indexes of the stored inter merge candidates to be updated.

[0306] FIG. 27 is a diagram showing an example in which the indices of pre-stored inter-merge candidates are updated.

[0307] If the index of a stored inter merge candidate that is the same as the inter merge candidate mvCand derived based on the current block is hIdx, the stored inter merge candidate may be deleted, and 1 may be subtracted from each of the indexes of the inter merge candidates whose indexes are greater than hIdx. For example, the example shown in Figure 27 shows that HmvpCand[2] that is the same as mvCand is deleted from the inter motion information list HvmpCandList, and 1 is subtracted from each of the indexes of HmvpCand[3] to HmvpCand[n].

[0308] In addition, the inter merge candidate mvCand derived based on the current block may be added to the end of the inter motion information list.

[0309] Alternatively, the index assigned to a stored inter merge candidate that is the same as the inter merge candidate derived based on the current block may be updated, for example, the index of the stored inter merge candidate may be changed to a minimum or maximum value.

[0310] The motion information of the block included in a specific region may be set not to be added to the inter motion information list. For example, the inter merge candidate derived based on the motion information of the block included in the merge processing region cannot be added to the inter motion information list. Because the coding / decoding order of the blocks included in the merge processing region is not defined, it is not appropriate to use the motion information of any one of these blocks for the inter prediction of other blocks. Therefore, the inter merge candidate derived based on the block included in the merge processing region may not be added to the inter motion information list.

[0311] When motion compensated prediction is performed based on sub-blocks, inter-merge candidates may be derived based on motion information of a representative sub-block among a plurality of sub-blocks included in the current block. For example, when a sub-block merging candidate is used for the current block, the inter-merge candidate may be derived based on motion information of a representative sub-block among the sub-blocks.

[0312] The motion vector of the sub-block may be derived in the following order: First, any one of the merging candidates included in the merging candidate list of the current block may be selected, and an initial shift vector (shVector) may be derived based on the motion vector of the selected merging candidate. In addition, a shift sub-block at a reference sample position (xColSb, yColSb) may be derived by adding the initial shift vector to a position (xSb, ySb) of a reference sample (e.g., a top-left sample or a middle-position sample) of each sub-block of the coding block. The following equation 20 shows an equation for deriving the shift sub-block.

[0313] Equation 20

number

[0314] Next, the motion vector of the collocated block corresponding to the center position of the sub-block containing (xColSb, yColSb) is set as the motion vector of the sub-block containing (xSb, ySb).

[0315] The representative sub-block may refer to the sub-block that includes the top-left sample or the center sample of the current block.

[0316] FIG. 28 shows the locations of representative sub-blocks.

[0317] 28(a) shows an example in which a sub-block located at the upper left of a current block is set as a representative sub-block, and FIG. 28(b) shows an example in which a sub-block located at the center of the current block is set as a representative sub-block. When motion compensated prediction is performed on a sub-block basis, an inter-merge candidate for the current block may be derived based on a motion vector of a sub-block including an upper left sample of the current block or a sub-block including a center sample of the current block.

[0318] Furthermore, it may be determined whether to use the current block as an inter merge candidate based on the inter prediction mode of the current block. For example, it may be set so that a block encoded / decoded based on an affine motion model cannot be used as an inter merge candidate. Therefore, even if the current block is encoded / decoded by inter prediction, if the inter prediction mode of the current block is an affine prediction mode, the inter prediction motion information list is not updated based on the current block.

[0319] Alternatively, the inter-merge candidate may be derived based on at least one sub-block vector of the sub-blocks included in the block encoded / decoded based on the affine motion model. For example, the inter-merge candidate may be derived using the sub-block located at the top left, the sub-block located at the center, or the sub-block located at the top right of the current block. Alternatively, the average value of the sub-block vectors of multiple sub-blocks may be set as the motion vector of the inter-merge candidate.

[0320] Alternatively, the inter-merge candidate may be derived based on an average value of the affine seed vectors of the blocks that are further coded / decoded based on the affine motion model, for example, an average value of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block may be set as the motion vector of the inter-merge candidate.

[0321] Alternatively, the inter motion information list may be set for different inter prediction modes. For example, at least one of the inter motion information list of the block for coding / decoding by intra block copy, the inter motion information list of the block for coding / decoding based on a translational motion model, or the inter motion information list of the block for coding / decoding based on an affine motion model may be defined. Any one of the multiple inter motion information lists may be selected based on the inter prediction mode of the current block.

[0322] FIG. 29 shows an example of generating inter motion information lists for different inter prediction modes.

[0323] When a block is coded / decoded based on a non-affine motion model, the inter-merge candidate mvCand derived based on the block may be added to the inter non-affine motion information list HmvpCandList, whereas when a block is coded / decoded based on an affine motion model, the inter-merge candidate mvAfCand derived based on the block may be added to the inter-affine motion information list HmvpAfCandList.

[0324] The affine seed vector of the block may be stored in an inter merge candidate derived from the encoded / decoded block based on the affine motion model, and the inter merge candidate may then be used as a merge candidate for deriving the affine seed vector of the current block.

[0325] In addition to the inter-motion information list described above, another inter-motion information list may be defined. In addition to the inter-motion information list described above (hereinafter referred to as the first inter-motion information list), a long-term motion information list may be defined (hereinafter referred to as the second inter-motion information list). The long-term motion information list includes long-term merge candidates.

[0326] When the first inter-motion information list and the second inter-motion information list are both empty, the inter-merge candidate may be added to the second inter-motion information list first. After the number of available inter-merge candidates in the second inter-motion information list reaches a maximum number, the inter-merge candidate may be added to the first inter-motion information list.

[0327] Alternatively, one more inter merge candidate may be added to both the second inter motion information list and the first inter motion information list.

[0328] In this case, the set second inter motion information list may not be updated again. Alternatively, when the decoding area is equal to or larger than a predetermined ratio of the slice, the second inter motion information list may be updated. Alternatively, the second inter motion information list may be updated every N coding tree unit rows.

[0329] On the other hand, the first inter motion information list may be updated every time a block encoded / decoded by inter prediction occurs. However, the inter merge candidate added to the second inter motion information list may be set not to be used for updating the first inter motion information list.

[0330] Information for selecting either the first inter motion information list or the second inter motion information list may be transmitted by a signal via the bit stream, and if the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter motion information list indicated by the information may be added to the merge candidate list.

[0331] Alternatively, the inter motion information list may be selected further according to the size, shape, and inter prediction mode of the current block, whether bidirectional prediction is performed, whether the motion vector is subdivided, or whether triangulation is performed.

[0332] Alternatively, if the number of merge candidates included in the merge candidate list is less than the maximum number of merges even after adding the inter merge candidates included in the first inter motion information list, the inter merge candidates included in the second inter motion information list may be added to the merge candidate list.

[0333] FIG. 30 is a diagram showing an example of adding an inter-merge candidate included in the long-term motion information list to the merge candidate list.

[0334] If the number of merge candidates included in the merge candidate list is less than the maximum number, the inter-merge candidates included in the first inter-motion information list HmvpCandList may be added to the merge candidate list.If the inter-merge candidates included in the first inter-motion information list are added to the merge candidate list, but the number of merge candidates included in the merge candidate list is also less than the maximum number, the inter-merge candidates included in the long-term motion information list HmvpLTCandList may be added to the merge candidate list.

[0335] Table 1 shows the process of adding inter-merge candidates included in the long-term motion information list to the merge candidate list.

[0336] [Table 1]

[0337] The intermerge candidates may be set to include additional information other than the motion information. For example, the size, shape, or block division information of the storage block may be added to the intermerge candidates. When constructing a merge candidate list for the current block, only intermerge candidates whose size, shape, or division information is the same or similar to the current block may be used in the intermerge candidates, or intermerge candidates whose size, shape, or division information is the same or similar to the current block may be preferentially added to the merge candidate list.

[0338] Alternatively, the inter-motion information list may be generated for different block sizes, shapes, or partition information. The merge candidate list for the current block may be generated using an inter-motion information list corresponding to the shape, size, or partition information of the current block among the multiple inter-motion information lists.

[0339] If the number of merge candidates included in the merge candidate list of the current block is less than a threshold, the inter merge candidates included in the inter motion information list may be added to the merge candidate list. The addition process may be performed in ascending or descending order of indexes. For example, the inter merge candidate with the largest index may be added to the merge candidate list.

[0340] When an inter merge candidate included in the inter motion information list is to be added to the merge candidate list, a redundancy check of the inter merge candidate and the multiple merge candidates stored in the merge candidate list may be performed.

[0341] For example, Table 2 shows the process of adding an intermerge candidate to the merge candidate list.

[0342] [Table 2]

[0343] Further, the redundancy check may be performed only on some of the inter merge candidates included in the inter motion information list. For example, the redundancy check may be performed only on the inter merge candidates whose indices are greater than or equal to a threshold value or less than or equal to a threshold value. Alternatively, the redundancy check may be performed only on the N merge candidates with the highest indexes or the N merge candidates with the lowest indexes.

[0344] Alternatively, the redundancy check may be performed on only some of the merge candidates stored in the merge candidate list, for example, on only merge candidates whose index is greater than or equal to a threshold or less than or equal to a threshold, or on merge candidates derived from a block at a specific location, which may include at least one of the left neighboring block, the upper neighboring block, the upper right neighboring block, or the lower left neighboring block of the current block.

[0345] FIG. 31 is a diagram showing an example in which redundancy check is performed on only some of the merging candidates.

[0346] When trying to add an inter merge candidate HmvpCand[j] to the merge candidate list, a redundancy check may be performed on the inter merge candidate and the two merge candidates with the highest indices mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1]. NumMerge may indicate the number of available spatial and temporal merge candidates.

[0347] What is different from the illustrated example is that when adding the inter merge candidate HmvpCand[j] to the merge candidate list, a redundancy check may be further performed on the inter merge candidate and the two merge candidates with the smallest index. For example, it may be confirmed whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, a redundancy check may be performed only on the merge candidates derived from a specific position. For example, a redundancy check may be performed only on at least one of the merge candidates derived from the adjacent block located to the left of the current block or the merge candidates derived from the adjacent block located above the current block. If there is no merge candidate derived from a specific position in the merge candidate list, the inter merge candidate may be added to the merge candidate list without performing a redundancy check.

[0348] If a merge candidate that is the same as the first intermerge candidate is found, then when performing a redundancy check on a second intermerge candidate, the redundancy check of the merge candidate that is the same as the first intermerge candidate may be omitted.

[0349] FIG. 32 is a diagram showing an example in which redundancy checks for a specific merging candidate are omitted.

[0350] When an inter merge candidate HmvpCand[i] with index i is to be added to the merge candidate list, a redundancy check may be performed on the inter merge candidate and the merge candidates stored in the merge candidate list. In such a case, if a merge candidate mergeCandList[j] identical to the inter merge candidate HmvpCand[i] is found, the inter merge candidate HmvpCand[i] may not be added to the merge candidate list, and a redundancy check may be performed on the inter merge candidate HmvpCand[i-1] with index i-1 and the merge candidate. In such a case, the redundancy check on the inter merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] may be omitted.

[0351] For example, in the example shown in Figure 32, it is determined that HmvpCand[i] is the same as mergeCandList[2]. Therefore, HmvpCand[i] is not added to the merge candidate list, and a redundancy check may be performed on HmvpCand[i-1]. In such a case, the redundancy check of HmvpCand[i-1] and mergeCandList[2] may be omitted.

[0352] If the number of merge candidates included in the merge candidate list of the current block is less than a threshold, in addition to the inter merge candidates, the list may further include at least one of paired merge candidates or zero merge candidates. A paired merge candidate refers to a merge candidate whose motion vector is the average value of the motion vectors of two or more merge candidates, and a zero merge candidate refers to a merge candidate whose motion vector is 0.

[0353] The current list of block merge candidates is: Merge candidates may be added in the following order: spatial merge candidate-temporal merge candidate-inter merge candidate-(inter affine merge candidate)-paired merge candidate-zero merge candidate.

[0354] The spatial merge candidates refer to merge candidates derived from at least one of adjacent or non-adjacent blocks, the temporal merge candidates refer to merge candidates derived from previous reference pictures, and the inter-affine merge candidate list refers to inter-merge candidates derived from blocks coded / decoded by an affine motion model.

[0355] In the advanced motion vector prediction mode, an inter-motion information list may be used. For example, if the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold, an inter-merge candidate included in the inter-motion information list is set as a motion vector prediction candidate related to the current block. Specifically, a motion vector of the inter-merge candidate is set as a motion vector prediction candidate.

[0356] When any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as a motion vector prediction value of the current block. Then, after decoding the motion vector residual value of the current block, the motion vector prediction value and the motion vector residual value may be added to obtain the motion vector of the current block.

[0357] The motion vector prediction candidate list for the current block is The candidate candidates may be constructed in the following order: spatial motion vector prediction candidate--temporal motion vector prediction candidate--inter-decoding region merge candidate--(inter-decoding region affine merge candidate)--zero motion vector prediction candidate.

[0358] The spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one of adjacent blocks or non-adjacent blocks, and the temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from a previous reference picture. The inter-affine merge candidate list indicates inter motion vector prediction candidates derived from blocks encoded / decoded by an affine motion model. The zero motion vector prediction candidate indicates a candidate whose motion vector value is 0.

[0359] A coding block may be divided into multiple prediction units, and prediction may be performed for each divided prediction unit. A prediction unit refers to a basic unit for prediction.

[0360] The coding block may be divided using at least one of a vertical line, a horizontal line, a diagonal line, or a diagonal line. Information for determining at least one of the number, angle, or position of lines dividing the coding block by a signal may be transmitted via a bit stream. For example, information indicating any one of candidates for division type of the coding block may be transmitted via a signal via a bit stream, or information specifying any one of a plurality of line candidates dividing the coding block may be transmitted via a signal via a bit stream. Alternatively, information for determining the number or type of line candidates dividing the coding block by a signal may be transmitted via a bit stream. For example, a 1-bit flag may be used to determine whether a diagonal line with an angle larger than a diagonal line and / or a diagonal line with an angle smaller than a diagonal line can be used as a line candidate.

[0361] Alternatively, at least one of the number, angle or position of lines dividing the coding block may be adaptively determined based on at least one of the intra prediction mode, inter prediction mode, position of available merging candidates or division type of adjacent blocks of the coding block.

[0362] When a coding block is divided into multiple prediction units, intra prediction or inter prediction may be performed on each divided prediction unit.

[0363] FIG. 33 is a diagram showing an example of dividing a coding block into a plurality of prediction units using diagonals.

[0364] In the example shown in parts (a) and (b) of Figure 33, a diagonal may be used to divide a coding block into two triangular prediction units.

[0365] Parts (a) and (b) of Figure 33 show that a coding block is divided into two prediction units using a diagonal line connecting two vertices of the coding block. However, a coding block may be divided into two prediction units using a diagonal line where at least one end of the line does not pass through a vertex of the coding block.

[0366] FIG. 34 is a diagram showing an example of dividing a coding block into two prediction units.

[0367] In the example shown in parts (a) and (b) of Figure 34, the coding block may be divided into two prediction units using a diagonal line whose ends touch the upper and lower boundaries of the coding block, respectively.

[0368] Alternatively, in the example shown in parts (c) and (d) of Figure 34, the coding block may be divided into two prediction units using a diagonal line whose ends touch the left and right boundaries of the coding block, respectively.

[0369] Alternatively, the coding block may be divided into two prediction blocks of different sizes, for example by setting the diagonal line dividing the coding block to touch two boundary surfaces that form a vertex, thereby dividing the coding block into two prediction units of different sizes.

[0370] FIG. 35 is a diagram showing an example of dividing a coding block into a plurality of prediction blocks of different sizes.

[0371] In the examples shown in Figures 35(a) and 35(b), the coding block can be divided into two prediction units of different sizes by setting the diagonal line connecting the upper left or lower right corner of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block without passing through the upper left or lower right corner of the coding block.

[0372] Alternatively, in the examples shown in Figures 35(c) and 35(d), the coding block can be divided into two prediction units of different sizes by setting the diagonal line connecting at the upper right or lower left corner of the coding block to pass through the left, right, upper or lower boundary of the coding block without passing through the upper left or lower right corner of the coding block.

[0373] Each prediction unit generated by dividing the coding block is called an "Nth prediction unit". For example, in the examples shown in Figures 33 to 35, PU1 may be defined as the first prediction unit, and PU2 may be defined as the second prediction unit. The first prediction unit may refer to a prediction unit including a lower left sample or an upper left sample in the coding block, and the second prediction unit may refer to a prediction unit including an upper right sample or a lower right sample in the coding block.

[0374] Conversely, a prediction unit including the top right sample or the bottom right sample in a coding block may be defined as the first prediction unit, and a prediction unit including the bottom left sample or the top left sample in a coding block may be defined as the second prediction unit.

[0375] The following embodiment mainly describes a division example using a diagonal line. In particular, the process of dividing a coding block into two prediction units using a diagonal line is called diagonal division or triangular division, and the prediction unit generated by diagonal division is called a triangular prediction unit. However, the following embodiment may of course use a division example using a vertical line, a horizontal line, or a diagonal line with an angle different from the diagonal line.

[0376] Whether to apply diagonal splitting to a coding block may be determined based on at least one of the slice type, the maximum number of merge candidates that may be included in a merge candidate list, the size of the coding block, the shape of the coding block, the predictive coding mode of the coding block, or the split type of the parent node.

[0377] For example, whether or not to apply diagonal partitioning to a coding block may be determined depending on whether the current slice is of type B. Only if the current slice is of type B, diagonal partitioning is allowed.

[0378] Alternatively, whether to apply diagonal splitting to a coding block may be determined based on whether the maximum number of merging candidates included in the merging candidate list is greater than or equal to 1. Diagonal splitting may be allowed only if the maximum number of merging candidates included in the merging candidate list is greater than or equal to 1.

[0379] Alternatively, if at least one of the width or height of the hardware is greater than 64, a data processing unit of 64×64 size will be accessed redundantly. Therefore, if at least one of the width or height of the coding block is greater than a threshold, it may not be permitted to divide the coding block into multiple prediction blocks. For example, if at least one of the height and width of the coding block is greater than 64, diagonal division may not be used.

[0380] Alternatively, whether to apply diagonal division to the coding block may be determined based on at least one of whether the number of samples included in the coding block is equal to or less than a first threshold or whether the number of samples included in the coding block is equal to or more than a second threshold. For example, if the number of samples included in the coding block is equal to or less than the first threshold or the number of samples included in the coding block is equal to or more than a second threshold, it is set not to apply diagonal division to the coding block. Alternatively, whether to apply diagonal division to the coding block may be determined based on whether the ratio of the width to the height of the coding block is smaller than a first threshold or larger than a second threshold. The ratio whRatio of the width to the height of the coding block may be determined as the ratio of the width CbW and the height CbH of the coding block, and is shown in the following equation 21.

[0381] Equation 21

number

[0382] The second threshold may be the reciprocal of the first threshold, for example, if the first threshold is k, the second threshold may be 1 / k.

[0383] Diagonal partitioning can be applied to a coding block only if the ratio of the width to the height of the coding block is between a first threshold and a second threshold.

[0384] Alternatively, triangular partitioning can be used only if the width-to-height ratio of the coding block is less than a first threshold or greater than a second threshold, e.g., if the first threshold is 16, then diagonal partitioning may not be allowed for coding blocks of size 64x4 or 4x64.

[0385] Alternatively, it may be determined whether diagonal splitting is allowed based on the split type of the parent node. For example, if the coding block as the parent node is split by quadtree splitting, diagonal splitting can be applied to the coding block as the leaf node. On the other hand, if the coding block as the parent node is split by binary or ternary tree splitting, the coding block as the leaf node may be set to not allow diagonal splitting.

[0386] Alternatively, it may be determined whether diagonal splitting is allowed based on the predictive coding mode of the coding block. For example, if the coding block is coded by intra prediction, diagonal splitting may be allowed only if the coding block is coded by inter prediction or if the coding block is coded in a predefined inter prediction mode. The predefined inter prediction mode may indicate at least one of a merge mode, an advanced motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0387] Alternatively, it may be determined whether diagonal splitting is allowed based on the size of the parallel processing region, for example, if the size of the coding block is larger than the size of the parallel processing region, diagonal splitting may not be used.

[0388] Furthermore, it may be possible to determine whether to apply diagonal division to a coding block by taking into consideration two or more of the conditions listed above.

[0389] Furthermore, information indicating whether diagonal partitioning is allowed may be signaled via the bitstream, and the information may be signaled at a sequence level, a picture level, a slice level, or a block level, where the information indicates that diagonal partitioning is allowed and diagonal partitioning can be applied to a coding block only if it satisfies at least one of the listed conditions.

[0390] If it is decided to apply a diagonal division to the coding block, information may be transmitted via a signal via the bitstream indicating the number of lines dividing the coding block or the position of the lines.

[0391] For example, when the coding block is divided by a diagonal, information indicating the direction of the diagonal for dividing the coding block may be transmitted by a signal via the bit stream. For example, a flag triangle_partition_type_flag indicating the direction of the diagonal may be transmitted by a signal via the bit stream. The flag indicates whether the coding block is divided by a diagonal connecting the upper left and lower right, or by a diagonal connecting the upper right and lower left. Dividing the coding block by a diagonal connecting the upper left and lower right may be referred to as a left triangular partition type, while dividing the coding block by a diagonal connecting the upper right and lower left may be referred to as a right triangular partition type. For example, when the value of the flag is 0, it indicates that the division type of the coding block is a left triangular partition type, and when the value of the flag is 1, it indicates that the division type of the coding block is a right triangular partition type.

[0392] Information indicating the partition type of a coding block may be signaled at the coding block level, and thus the partition type may be determined for different coding blocks that apply diagonal partitioning.

[0393] As another example, information indicating the partition type may be transmitted by a signal for a sequence, a picture, a slice, a segment, or a coding tree unit. In such a case, the partition type of a coding block to which diagonal partitioning is applied may be set to be the same in the sequence, the picture, the slice, the segment, or the coding tree unit.

[0394] Alternatively, for the first coding unit that applies diagonal partitioning in the coding tree unit, information for determining the partition type is coded and transmitted by a signal, and the second and subsequent coding units that apply diagonal partitioning are configured to use the same partition type as the first coding unit.

[0395] As another example, the partition type of the coding block may be determined based on the partition type of the neighboring blocks. The neighboring blocks may include at least one of the neighboring blocks adjacent to the upper left corner of the coding block, the neighboring block adjacent to the upper right corner, the neighboring block adjacent to the lower left corner, the neighboring block located above, or the neighboring block located to the left. For example, the partition type of the current block may be set to be the same as the partition type of the neighboring blocks. Alternatively, the partition type of the current block may be determined depending on whether the upper left neighboring block applies a left triangular partition type and whether the upper right neighboring block or the lower left neighboring block applies a right triangular partition type.

[0396] In order to perform motion prediction compensation for the first triangular prediction unit and the second triangular prediction unit, motion information of each of the first triangular prediction unit and the second triangular prediction unit may be derived. In this case, the motion information of the first triangular prediction unit and the second triangular prediction unit may be derived from merge candidates included in the merge candidate list. In order to distinguish between a general merge candidate list and a merge candidate list used in deriving motion information of a triangular prediction unit, the merge candidate list for deriving motion information of a triangular prediction unit is referred to as a triangle merge candidate list, and the merge candidates included in the triangle merge candidate list are referred to as triangle merge candidates. However, the scope of the present invention also includes a list construction method using the above merge candidate derivation method and merge candidate list construction method for triangle merge candidates and triangle merge candidates.

[0397] The signal may transmit information via the bitstream to determine the maximum number of triangle merge candidates that may be included in the triangle merge candidate list, and the information may indicate a difference between the maximum number of merge candidates that may be included in the triangle merge candidate list and the maximum number of triangle merge candidates that may be included in the triangle merge candidate list.

[0398] Triangle merging candidates may be derived from the spatial and temporal neighboring blocks of a coding block.

[0399] FIG. 36 is a diagram showing adjacent blocks for deriving triangle merging candidates.

[0400] At least one of a neighboring block located above the coding block, a neighboring block located to the left of the coding block, or a co-located block included in a different picture from the coding block may be used to derive triangle merging candidates. The upper neighboring block may be at least one of a block including samples (xCb+CbW-1, yCb-1) located above the coding block, a block including samples (xCb+CbW, yCb-1) located above the coding block, or a block including samples (xCb-1, yCb-1) located above the coding block. The left neighboring block may be at least one of a block including samples (xCb-1, yCb+CbH-1) located to the left of the coding block, or a block including samples (xCb-1, yCb+CbH) located to the left of the coding block. The collocated block may be determined as either the block containing samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coding block in the collocated picture, or the block containing samples (xCb / 2, yCb / 2) located at the center of the coding block.

[0401] The neighboring blocks may be searched according to a predefined order, and the triangle merge candidates may be constructed into a triangle merge candidate list according to a predefined order, for example, the triangle merge candidates may be searched according to the order of B1, A1, B0, A0, C0, B2, and C1 to construct the triangle merge candidate list.

[0402] The motion information of the triangular prediction units may be derived based on the triangle merging candidate list, i.e., the triangular prediction units may share one triangle merging candidate list.

[0403] To derive the motion information of the triangle merging unit, at least one of the pieces of information for specifying a triangle merging candidate included in the triangle merging candidate list may be transmitted by a signal via the bitstream, for example, at least one of the pieces of index information merge_triangle_idx for specifying a triangle merging candidate may be transmitted by a signal via the bitstream.

[0404] The index information may specify a combination of a merging candidate of the first triangular prediction unit and a merging candidate of the second triangular prediction unit. For example, Table 3 below shows an example of a combination of merging candidates based on the index information merge_triangle_idx.

[0405] [Table 3]

[0406] When the value of the index information merge_triangle_idx is 1, it indicates that the motion information of the first triangular prediction unit is derived from the merge candidate with index 1, and the motion information of the second triangular prediction unit is derived from the merge candidate with index 0. The index information merge_triangle_idx may determine the triangle merge candidate for deriving the motion information of the first triangular prediction unit and the triangle merge candidate for deriving the motion information of the second triangular prediction unit.

[0407] Furthermore, the partition type of the coding block to which diagonal partitioning is applied may be determined based on index information. That is, the index information may specify a combination of merge candidates of the first triangular prediction unit, merge candidates of the second triangular prediction unit, and a partition direction of the coding block. When the partition type of the coding block is determined based on index information, it is not necessary to code information triangle_partition_type_flag indicating the direction of the diagonal line that divides the coding block. Table 4 shows the partition types of the coding block with respect to index information merge_triangle_idx.

[0408] [Table 4]

[0409] When the variable TriangleDir is 0, it indicates that the coding block applies a left triangular division type, and when the variable TriangleDir is 1, it indicates that the coding block applies a right triangular division type. By combining Table 3 and Table 4, it may be set so that a combination of merging candidates for the first triangular prediction unit, merging candidates for the second triangular prediction unit, and division direction of the coding block is specified based on the index information merge_triangle_idx.

[0410] As another example, index information used for only one of the first triangular prediction unit and the second triangular prediction unit may be transmitted by a signal, and an index of a triangle merge candidate used for the other of the first triangular prediction unit and the second triangular prediction unit may be determined based on the index information. For example, a triangle merge candidate of the first triangular prediction unit may be determined based on index information merge_triangle_idx indicating an index of any one of the triangle merge candidates. Also, a triangle merge candidate of the second triangular prediction unit may be specified based on the merge_triangle_idx. For example, a triangle merge candidate of the second triangular prediction unit may be derived by adding an offset amount to the index information merge_triangle_idx or subtracting an offset amount from the index information merge_triangle_idx. The offset amount may be an integer such as 1 or 2. For example, a triangle merge candidate of the second triangular prediction unit may be determined as a triangle merge candidate having an index obtained by adding 1 to merge_traingle_idx. When merge_triangle_idx indicates the triangle merging candidate with the largest index value among the triangle merging candidates, motion information of the second triangular prediction unit may be derived from the triangle merging candidate with index 0 or the triangle merging candidate whose index is the value obtained by subtracting 1 from merge_triangle_idx.

[0411] Alternatively, the motion information of the second triangular prediction unit may be derived from a triangle merge candidate having the same reference picture as the triangle merge candidate of the first triangular prediction unit specified based on the index information. The triangle merge candidate having the same reference picture as the triangle merge candidate of the first triangular prediction unit may indicate at least one of an L0 reference picture or an L1 reference picture having the same triangle merge candidate as the triangle merge candidate of the first triangular prediction unit. When there are multiple triangle merge candidates having the same reference picture and the triangle merge candidate of the first triangular prediction unit, one of them may be selected depending on whether the merge candidate includes at least one of bidirectional motion information or a difference between an index of the merge candidate and the index information.

[0412] As another example, index information may be transmitted by a signal to each of the first triangular prediction unit and the second triangular prediction unit. For example, first index information 1st_merge_idx for determining a triangle merge candidate for the first triangular prediction unit and second index information 2nd_merge_idx for determining a triangle merge candidate for the second triangular prediction unit may be transmitted by a signal via a bitstream. Motion information of the first triangular prediction unit may be derived from the triangle merge candidate determined based on the first index information 1st_merge_idx, and motion information of the second triangular prediction unit may be derived from the triangle merge candidate determined based on the second index information 2nd_merge_idx.

[0413] The first index information 1st_merge_idx may indicate an index of any one of the triangle merge candidates included in the triangle merge candidate list. The triangle merge candidate of the first triangular prediction unit may be determined as the triangle merge candidate pointed to by the first index information 1st_merge_idx.

[0414] The triangle merge candidate pointed to by the first index information 1st_merge_idx may be set not to be used as a triangle merge candidate of the second triangular prediction unit. Thus, the second index information 2nd_merge_idx of the second triangular prediction unit may indicate an index of any one of the remaining triangle merge candidates other than the triangle merge candidate pointed to by the first index information. If the value of the second index information 2nd_merge_idx is smaller than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangular prediction unit may be determined as the triangle merge candidate having index information indicated by the second index information 2nd_merge_idx. On the other hand, if the value of the second index information 2nd_merge_idx is the same as or larger than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangular prediction unit may be determined as the triangle merge candidate having an index value obtained by adding 1 to the value of the second index information 2nd_merge_idx.

[0415] Alternatively, it may be determined whether to transmit the second index information by a signal based on the number of triangle merge candidates included in the triangle merge candidate list. For example, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is 2 or less, transmitting the second index information by a signal may be omitted. When omitting the signal transmission of the second index information or omitting transmitting the second index information by a signal, the second triangle merge candidate may be derived by adding an offset amount to the first index information or subtracting an offset amount from the first index information. For example, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is 2 and the first index information is index 0, the second triangle merge candidate may be derived by adding 1 to the first index information. Alternatively, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is 2 and the first index information is index 1, the second triangle merge candidate may be derived by subtracting 1 from the first index information.

[0416] Alternatively, when the second index information is omitted from being transmitted by the signal, the second index information may be set as a default value. The default value may be 0. The second triangle merging candidate may be derived by comparing the first index information with the second index information. For example, if the second index information is smaller than the first index information, the merging candidate with index 0 is set as the second triangle merging candidate, and if the second index information is equal to or larger than the first index information, the merging candidate with index 1 is set as the second triangle merging candidate.

[0417] If the triangle merge candidate has unidirectional motion information, the unidirectional motion information of the triangle merge candidate is set as the motion information of the triangle prediction unit. On the other hand, if the triangle merge candidate has bidirectional motion information, only one of the L0 motion information or the L1 motion information is set as the motion information of the triangle prediction unit. It may be determined which of the L0 motion information or the L1 motion information to obtain based on the index of the triangle merge candidate or the motion information of other triangle prediction units.

[0418] For example, if the index of the triangle merge candidate is an even number, the L0 motion information of the triangle prediction unit is set as 0, and the L1 motion information of the triangle merge candidate is set as the L1 motion information of the triangle prediction unit. On the other hand, if the index of the triangle merge candidate is an odd number, the L1 motion information of the triangle prediction unit is set as 0, and the L0 motion information of the triangle merge candidate is set as 0. Conversely, if the index of the triangle merge candidate is an even number, the L0 motion information of the triangle merge candidate may be set as the L0 motion information of the triangle prediction unit, and if the index of the triangle merge candidate is an odd number, the L1 motion information of the triangle merge candidate may be set as the L1 motion information of the triangle prediction unit. Alternatively, for a first triangle prediction unit, if the triangle merge candidate is an even number, the L0 motion information of the triangle merge candidate may be set as the L0 motion information of the first triangle prediction unit, while for a second triangle prediction unit, if the triangle merge candidate is an odd number, the L1 motion information of the triangle merge candidate may be set as the L1 motion information of the second triangle prediction unit.

[0419] Alternatively, if the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit may be set as 0, and the L1 motion information of the triangle merging candidate may be set as the L1 information of the second triangle prediction unit. On the other hand, if the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit may be set as 0, and the L0 motion information of the triangle merging candidate may be set as the L0 motion signal of the second triangle prediction unit.

[0420] Furthermore, the triangle merging candidate list for deriving motion information of the first triangular prediction unit and the triangle merging candidate list for deriving motion information of the second triangular prediction unit may be set to be different from each other.

[0421] For example, when a triangle merge candidate for deriving motion information of the first triangular prediction unit in a triangle merge candidate list is specified based on index information related to the first triangular prediction unit, the motion information of the second triangular prediction unit may be derived using a triangle merge list including remaining triangle merge candidates other than the triangle merge candidate indicated by the index information. Specifically, the motion information of the second triangular prediction unit may be derived from any one of the remaining triangle merge candidates.

[0422] Therefore, the maximum number of triangle merge candidates included in the triangle merge candidate list of the first triangular prediction unit is different from the maximum number of triangle merge candidates included in the triangle merge candidate list of the second triangular prediction unit. For example, if the triangle merge candidate list of the first triangular prediction unit includes M merge candidates, the triangle merge candidate list of the second triangular prediction unit may include M-1 merge candidates other than the triangle merge candidate pointed to by the index information of the first triangular prediction unit.

[0423] As another example, merging candidates for each triangular prediction unit may be derived based on neighboring blocks adjacent to the coding block, and the availability of neighboring blocks may be determined taking into account the shape or position of the triangular prediction unit.

[0424] FIG. 37 is a diagram for explaining an example of determining the availability of adjacent blocks for each triangular prediction unit.

[0425] An adjacent block that is not adjacent to the first triangular prediction unit may be set as an unavailable adjacent block for the first triangular prediction unit, and an adjacent block that is not adjacent to the second triangular prediction unit may be set as an unavailable adjacent block for the second triangular prediction unit.

[0426] For example, in the example shown in Figure 37(a), when the left triangulation type is applied to the coding block, it can be determined that blocks A1, A0, and A2 adjacent to the first triangular prediction unit among the adjacent blocks adjacent to the coding block can be used for the first triangular prediction unit, but blocks B0 and B1 cannot be used for the first triangular prediction unit. Therefore, the triangle merge candidate list related to the first triangular prediction unit includes triangle merge candidates derived from blocks A1, A0, and A2, but does not include triangle merge candidates derived from blocks B0 and B1.

[0427] In the example shown in Figure 37(b), when the left triangulation type is applied to the coding block, it can be determined that blocks B0 and B1 adjacent to the second triangular prediction unit can be used for the second triangular prediction unit, but blocks A1, A0, and A2 cannot be used for the second triangular prediction unit. Therefore, the triangle merge candidate list related to the second triangular prediction unit includes triangle merge candidates derived from blocks B0 and B1, but does not include triangle merge candidates derived from blocks A1, A0, and A2.

[0428] Therefore, the number of triangle merging candidates or the range of triangle merging candidates that the triangular prediction unit can use may be determined based on at least one of the position of the triangular prediction unit or the division type of the coding block.

[0429] As another example, only the merge mode may be applied to one of the first and second triangular prediction units, and motion information of the other of the first and second triangular prediction units may be set to be the same as the motion information of the triangular prediction unit to which the merge mode is applied, or the motion information of the triangular prediction unit to which the merge mode is applied may be derived by subdividing it.

[0430] For example, the motion vector and reference picture index of the first triangular prediction unit may be derived based on the triangle merging candidate, and the motion vector of the second triangular prediction unit may be derived by subdividing the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding the fine motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit, or by subtracting the fine motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0431] The information for determining a fine motion vector indicating a difference between a motion vector of the first triangular prediction unit and a motion vector of the second triangular prediction unit may be transmitted by a signal via a bitstream, and the information may include at least one of information indicating a size of the fine motion vector or information indicating a symbol of the fine motion vector.

[0432] Alternatively, the symbols of the fine motion vector may be derived based on at least one of the position, index, or partition type applied to the coding block of the triangular prediction unit.

[0433] As another example, a motion vector and a reference picture index of one of the first triangular prediction unit and the second triangular prediction unit may be transmitted by a signal, and the motion vector of the other of the first triangular prediction unit and the second triangular prediction unit may be derived by subdividing the motion vector transmitted by the signal.

[0434] For example, the motion vector and the reference picture index of the first triangular prediction unit may be determined based on information signaled from a bitstream. Also, the motion vector of the second triangular prediction unit may be derived by subdividing the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding the fine motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit, or by subtracting the fine motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0435] Based on the motion information of the first triangular prediction unit and the motion information of the second triangular prediction unit, motion prediction and compensation prediction may be performed on the coding blocks, respectively. In this case, image quality may be degraded at the boundary between the first triangular prediction unit and the second triangular prediction unit. For example, image quality may be continuously degraded near the edge on the boundary between the first triangular prediction unit and the second triangular prediction unit. In order to reduce image quality degradation at the boundary, predicted samples may be derived by smoothing filtering or weighted prediction.

[0436] A prediction sample using diagonal division in a coding block may be derived by weighting and calculating a first prediction sample obtained based on motion information of a first triangular prediction unit and a second prediction sample obtained based on motion information of a second triangular prediction unit. Alternatively, a prediction sample of a first triangular prediction unit may be derived from a first prediction block determined based on motion information of a first triangular prediction unit, a prediction sample of a second triangular prediction unit may be derived from a second prediction block determined based on motion information of a second triangular prediction unit, and a prediction sample located in a boundary area between the first triangular prediction unit and the second triangular prediction unit may be derived by weighting and calculating the first prediction sample included in the first prediction block and the second prediction sample included in the second prediction block. For example, the following equation 22 shows an example of deriving prediction samples of a first triangular prediction unit and a second triangular prediction unit.

[0437] Equation 22

number

[0438] In Equation 22, P1 denotes the first predicted sample, P2 denotes the second predicted sample, w1 denotes the weighting value applied to the first predicted sample, and (1-w1) denotes the weighting value applied to the second predicted sample. In the example shown in Equation 22, the weighting value applied to the second predicted sample can be derived by subtracting the weighting value applied to the first predicted sample from a constant.

[0439] When a left triangulation type is applied to a coding block, the boundary region may include prediction samples having the same x-axis and y-axis coordinates, whereas when a right triangulation type is applied to a coding block, the boundary region may include prediction samples whose sum of the x-axis and y-axis coordinates is greater than or equal to a first threshold and less than or equal to a second threshold.

[0440] The size of the boundary area may be determined based on at least one of the size of the coding block, the shape of the coding block, the motion information of the triangular prediction unit, the motion vector difference of the triangular prediction unit, the output order of the reference picture, or the difference between the first prediction sample and the second prediction sample on the diagonal boundary.

[0441] Figures 38 and 39 are diagrams showing an example of deriving a predicted sample by weighting and calculating a first predicted sample and a second predicted sample. Figure 38 shows a case where a coding block applies a left triangular division type, and Figure 39 shows a case where a coding block applies a right triangular division type. Also, Figures 38(a) and 39(a) are diagrams showing prediction states related to the luma component, and Figures 38(b) and 39(b) are diagrams showing prediction states related to the chroma component.

[0442] In the drawing, a number written in a prediction sample near the boundary between the first prediction unit and the second prediction unit indicates a weighting value applied to the first prediction sample. For example, if the number written in a prediction sample is N, the first prediction sample may be derived by applying a weighting value of N / 8 to the first prediction sample and a weighting value of (1-(N / 8)) to the second prediction sample.

[0443] In a non-boundary region, the first prediction sample or the second prediction sample may be determined as a prediction sample. In the example of Fig. 38, in a region belonging to the first triangular prediction unit among regions in which the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, the first prediction sample derived based on the motion information of the first triangular prediction unit may be determined as a prediction sample. On the other hand, in a region belonging to the second triangular prediction unit among regions in which the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, the second prediction sample derived based on the motion information of the second triangular prediction unit may be determined as a prediction sample.

[0444] 39, in a region where the sum of the x-axis coordinate and the y-axis coordinate is smaller than a first threshold, a first prediction sample derived based on the motion information of a first triangular prediction unit may be determined as a prediction sample, while in a region where the sum of the x-axis coordinate and the y-axis coordinate is greater than a second threshold, a second prediction sample derived based on the motion information of a second triangular prediction unit may be determined as a prediction sample.

[0445] The threshold for determining the non-border region may be determined based on at least one of the size of the coding block, the shape of the coding block, or the color component. For example, if the threshold associated with the luma component is set as N, the threshold associated with the chroma component may be set as N / 2.

[0446] A predicted sample included in the boundary region may be derived by weighting and computing the first predicted sample and the second predicted sample, and in such a case, the weighting value applied to the first predicted sample and the second predicted sample may be determined based on at least one of the position of the predicted sample, the size of the coding block, the shape of the coding block, or the color component.

[0447] For example, in the example shown in Fig. 38(a), by applying the same weighting value to the first predicted sample and the second predicted sample, predicted samples at positions having the same x-axis coordinate and y-axis coordinate can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (3:1) or (1:3), predicted samples whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived. Also, by setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), predicted samples whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 2 can be derived.

[0448] Alternatively, in the example shown in Figure 38(b), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample at a position having the same x-axis coordinate and y-axis coordinate can be derived, and by setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived.

[0449] For example, in the example shown in Fig. 39(a), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (3:1) or (1:3), a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as the width or height of the coding block or is 2 less than the width or height of the coding block can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 more than the width or height of the coding block or is 3 less than the width or height of the coding block can be derived.

[0450] Alternatively, in the example shown in Figure 39(b), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as the width or height of the coding block or 2 less than the width or height of the coding block can be derived.

[0451] As another example, the weighting value may be determined taking into account the position of the prediction sample or the shape of the coding block. Equation 23 to Equation 25 show an example of deriving the weighting value when a left triangulation type is applied to the coding block. Equation 23 shows an example of deriving the weighting value applied to the first prediction sample when the coding block is square.

[0452] Equation 23

number

[0453] In Equation 23, x and y denote the location of the predicted sample. If the coding block is non-square, the weighting value applied to the first predicted sample may be derived as shown in Equation 24 or 25. Equation 24 illustrates the case where the width of the coding block is greater than the height, and Equation 25 illustrates the case where the width of the coding block is less than the height.

[0454] Equation 24

number

[0455] Equation 25

number

[0456] When a right triangular decomposition type is applied to the coding block, the weighting values ​​applied to the first predicted samples may be determined as shown in Equations 26 to 28. Equation 26 shows an example of deriving the weighting values ​​applied to the first predicted samples when the coding block is square.

[0457] Equation 26

number

[0458] In equation 26, CbW denotes the width of the coding block. If the coding block is non-square, the weighting value applied to the first predicted sample may be derived as shown in equation 27 or 28. Equation 27 illustrates the case where the coding block width is greater than the height, and equation 28 illustrates the case where the coding block width is less than the height.

[0459] Equation 27

number

[0460] Equation 28

number

[0461] In equation 27, CbH denotes the height of the coding block.

[0462] In the illustrated example, for prediction samples in boundary regions, a sample included in a first triangular prediction unit can be derived by giving a higher weighting value to a first prediction sample than to a second prediction sample, and a sample included in a second triangular prediction unit can be derived by giving a higher weighting value to a second prediction sample than to the first prediction sample.

[0463] When diagonal division is applied to a coding block, the coding block may be set not to apply a combined prediction mode that combines an intra prediction mode and a merge mode.

[0464] Intra prediction predicts the current block using reconstructed samples after encoding / decoding of the surrounding area of ​​the current block. In this case, the intra prediction of the current block may use reconstructed samples before applying the in-loop filter.

[0465] The intra prediction technique includes intra prediction based on a matrix and general intra prediction considering the directionality of surrounding reconstructed samples. Information indicating the intra prediction technique of the current block may be transmitted by a signal through the bitstream. The information may be a 1-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, shape, or intra prediction techniques of the current block or neighboring blocks. For example, if the current block exists across a picture boundary, the current block may be set not to apply matrix-based intra prediction.

[0466] Matrix-based intra prediction is a method of obtaining a prediction block of a current block by matrix multiplication of a matrix stored in an encoder and a decoder and reconstructed samples around the current block. Information for specifying one of a plurality of matrices stored by a signal may be transmitted via a bitstream. The decoder may determine a matrix for intra prediction of the current block based on the information and the size of the current block.

[0467] General intra prediction is a method of obtaining a predicted block related to a current block based on a non-angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in more detail with reference to the accompanying drawings.

[0468] FIG. 40 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention.

[0469] A reference sample line of a current block may be determined (S4001). The reference sample line refers to a set of reference samples included in the Kth line from the top and / or left side of the current block. The reference samples may be derived from reconstructed samples after encoding / decoding around the current block.

[0470] Index information indicating a reference sample line of a current block among a plurality of reference sample lines may be transmitted by a signal via a bitstream. For example, index information intra_luma_ref_idx for specifying a reference sample line of the current block may be transmitted by a signal via a bitstream. The index information may be transmitted by a signal for each coding block.

[0471] The plurality of reference sample lines may include at least one of a first line, a second line, a third line, or a fourth line located above and / or to the left of the current block. Of the plurality of reference sample lines, a reference sample line consisting of a row adjacent to the upper side of the current block and a column adjacent to the left side of the current block may be referred to as an adjacent reference sample line, and the other reference sample lines may be further referred to as non-adjacent reference sample lines.

[0472] Some of the multiple reference sample lines may be selected as the reference sample lines of the current block. For example, the remaining reference sample lines other than the third non-adjacent reference sample line among the multiple reference sample lines may be set as candidate reference sample lines. Table 5 shows the indexes assigned to each candidate reference sample line.

[0473] [Table 5]

[0474] More or fewer candidate reference sample lines may be set than the described sample lines. Also, the number or positions of the non-adjacent reference sample lines set as the candidate reference sample lines are not limited to the described examples. For example, the first non-adjacent reference sample line and the third non-adjacent reference sample line may be set as the candidate reference sample lines, or the second non-adjacent reference sample line and the third non-adjacent reference sample line may be set as the candidate reference sample lines. Alternatively, the first non-adjacent reference sample line, the second non-adjacent reference sample line, and the third non-adjacent reference sample line may all be set as the candidate reference sample lines.

[0475] The number or type of candidate reference sample lines may be determined based on at least one of the size, shape, position of the current block, whether a sub-block is split, or the intra-prediction mode.

[0476] The reference sample line of the current block may further be determined based on at least one of the position, size, shape, or predictive coding mode of the neighboring block of the current block. For example, if the current block contacts the boundary of a picture, a segment, a slice, or a coding tree unit, the first reference sample line may be determined as the reference sample line of the current block.

[0477] Alternatively, if the current block is non-square, the adjacent reference sample line may be set as the reference sample line of the current block. Alternatively, if the width-to-height ratio of the current block is greater than or equal to a threshold, the adjacent reference sample line may be determined as the reference sample line of the current block.

[0478] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper reference sample and the left reference sample may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before applying an in-loop filter.

[0479] Next, an intra prediction mode of the current block may be determined (S4002). For the intra prediction mode of the current block, at least one of a non-angular intra prediction mode or an angular intra prediction mode may be determined as the intra prediction mode of the current block. The non-angular intra prediction modes include planar and DC, and the angular intra prediction modes include 33 or 65 modes from the lower left diagonal to the upper right diagonal.

[0480] FIG. 41 is a diagram showing intra prediction modes.

[0481] FIG. 41(a) shows 35 intra prediction modes, and FIG. 41(b) shows 67 intra prediction modes.

[0482] Additionally, more or less intra-prediction modes than those shown in FIG. 41 may be defined.

[0483] A Most Probable Mode (MPM) may be set based on intra-prediction modes of neighboring blocks adjacent to the current block. The neighboring blocks may include a left neighboring block adjacent to the left side of the current block and an upper neighboring block adjacent to the upper side of the current block.

[0484] The number of MPMs included in the MPM list in the encoder and decoder may be predefined. For example, the number of MPMs may be three, four, five, or six. Alternatively, information indicating the number of MPMs may be transmitted by a signal through the bitstream. Alternatively, the number of MPMs may be determined based on at least one of the predictive coding mode of the neighboring block, the size or shape of the current block. For example, when determining a neighboring reference sample line as the reference sample line of the current block, N MPMs may be used, but when determining a non-neighboring reference sample line as the reference sample line of the current block, M MPMs may be used. M is an integer smaller than N, for example, N may be 6, and M may be 5, 4, or 3. Thus, if the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra prediction mode of the current block may be determined as any one of six candidate intra prediction modes, but if the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra prediction mode of the current block may be determined as any one of five candidate intra prediction modes.

[0485] Alternatively, a fixed number of MPM candidates (eg, six or five) may also be used regardless of the index of the reference sample line of the current block.

[0486] An MPM list including a plurality of MPMs may be generated, and information indicating whether the same MPM as the intra prediction mode of the current block is included in the MPM list may be transmitted by a signal via a bitstream. The information may be a 1-bit flag and may be referred to as an MPM flag. If the MPM flag indicates that the same MPM as the current block is included in the MPM list, index information indicating one of the MPMs may be transmitted by a signal via a bitstream. For example, index information mpm_idx specifying any one of the plurality of MPMs may be transmitted by a signal via a bitstream. The MPM specified by the index information may be set as the intra prediction mode of the current block. If the MPM flag indicates that the same MPM as the current block is not included in the MPM list, remaining mode information indicating any one of the remaining intra prediction modes other than the MPM may be transmitted by a signal via a bitstream. The remaining mode information indicates an index value corresponding to the intra prediction mode of the current block when reassigning an index to the remaining intra prediction modes other than the MPM. The decoder may arrange the MPMs in ascending order and determine the intra prediction mode of the current block by comparing the remaining mode information with the MPM. For example, if the residual mode information is equal to or less than the MPM, 1 may be added to the residual mode information to derive the intra prediction mode of the current block.

[0487] When deriving the intra prediction mode of the current block, comparison of some of the MPMs with the remaining mode information may be omitted. For example, the MPM of the non-angular intra prediction mode may be excluded from the comparison object. When the non-angular intra prediction mode is set as the MPM, the remaining mode information clearly indicates the angular intra prediction mode, so that the intra prediction mode of the current block may be derived by comparing the remaining MPMs other than the non-angular intra prediction mode with the remaining mode information. Instead of excluding the non-angular intra prediction mode from the comparison object, a result value obtained by adding the number of non-angular intra prediction modes to the remaining mode information may be compared with the MPM.

[0488] Instead of setting the default mode as the MPM, information indicating whether the intra prediction mode of the current block is the default mode may be transmitted by a signal via the bitstream. The information may be a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be transmitted by a signal only if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode may include at least one of planar, DC, vertical mode, or horizontal mode. For example, if planar is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is planar. If the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

[0489] When the default mode flag is used, the intra prediction mode that is the same as the default mode may not be set as the MPM. For example, if the default mode flag indicates whether the intra prediction mode of the current block is planar, the intra prediction mode of the current block may be derived using five MPMs other than the MPM corresponding to planar.

[0490] When a plurality of intra prediction modes are set as a default mode, index information indicating one of the default modes may be further transmitted by a signal, and the intra prediction mode of the current block may be set as the default mode indicated by the index information.

[0491] If the index of the reference sample line of the current block is not 0, the default mode is set not to be used. For example, if a non-adjacent reference sample line is determined as the reference sample line of the current block, a non-angular intra prediction mode such as DC mode or planar mode may be set not to be used. Therefore, if the index of the reference sample line is not 0, the default mode flag may not be sent by a signal, and the value of the default mode flag may be set to a predefined value (i.e., false).

[0492] When the intra-prediction mode of the current block is determined, a prediction sample related to the current block may be obtained based on the determined intra-prediction mode (S4003).

[0493] When the DC mode is selected, a predicted sample related to the current block may be generated based on an average value of reference samples. Specifically, values ​​of all samples of the predicted block may be generated based on the average value of reference samples. The average value may be derived using at least one of an upper reference sample located above the current block and a left reference sample located to the left of the current block.

[0494] The number or range of reference samples for deriving the average value may vary depending on the shape of the current block. For example, if the current block is a non-square block with a width greater than its height, the average value may be calculated using only the upper reference samples. On the other hand, if the current block is a non-square block with a width less than its height, the average value may be calculated using only the left reference samples. That is, if the width and height of the current block are different, the average value may be calculated using only the adjacent reference samples on the longer side. Alternatively, it may be determined whether to calculate the average value using only the upper reference samples or only the left reference samples based on the ratio of the width and height of the current block.

[0495] When the planar mode is selected, the prediction sample may be obtained using the horizontal prediction sample and the vertical prediction sample. The horizontal prediction sample is obtained based on the left reference sample and the right reference sample located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on the upper reference sample and the lower reference sample located on the same vertical line as the prediction sample. The right reference sample may be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample may be generated by copying the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample may be obtained by weighting and computing the left reference sample and the right reference sample, and the vertical prediction sample may be obtained by weighting and computing the upper reference sample and the lower reference sample. In this case, the weighting value given to each reference sample may be determined based on the position of the prediction sample. The prediction sample may be obtained by averaging or weighting and computing the horizontal prediction sample and the vertical prediction sample. When performing the weighting and computing, the weighting value given to the horizontal prediction sample and the vertical prediction sample may be determined based on the position of the prediction sample.

[0496] When an angular prediction mode is selected, a parameter indicating a prediction direction (or a prediction angle) of the selected angular prediction mode may be determined. Table 6 below shows the intra prediction parameter intraPredAng for each intra prediction mode.

[0497] [Table 6]

[0498] Table 6 shows, when 35 intra prediction modes are defined, the intra direction parameters of each intra prediction mode having one of indexes 2 to 34. When 33 or more angular intra prediction modes are defined, Table 6 is further subdivided to set the intra direction parameters of each angular intra prediction mode.

[0499] The upper reference sample and the left reference sample of the current block may be arranged in a row, and then a predicted sample may be obtained based on the value of the intra direction parameter. In this case, if the value of the intra direction parameter is a negative number, the left reference sample and the upper reference sample may be arranged in a row.

[0500] 42 and 43 are diagrams showing examples of a one-dimensional array in which reference samples are arranged in one row.

[0501] Figure 42 shows an example of a vertical one-dimensional array in which reference samples are arranged in the vertical direction, and Figure 43 shows an example of a horizontal one-dimensional array in which reference samples are arranged in the horizontal direction. The embodiments of Figures 42 and 43 will be described assuming that 35 intra prediction modes are defined.

[0502] A horizontal one-dimensional array in which the upper reference sample is rotated counterclockwise may be applied when the intra prediction mode index is one of 11 to 18, and a vertical one-dimensional array in which the left reference sample is rotated clockwise may be applied when the intra prediction mode index is one of 19 to 25. When arranging the reference samples in a row, the intra prediction mode angle may be taken into consideration.

[0503] Reference sample decision parameters may be determined based on the intra direction parameters, and may include a reference sample index for specifying the reference sample and a weighting value parameter for determining a weighting value to be applied to the reference sample.

[0504] The reference sample index iIdx and the weighting parameter ifact may be obtained by the following Equations 30 and 31, respectively.

[0505] equation 30

number

[0506] Equation 31

number

[0507] In equations 30 and 31, P ang indicates an intra direction parameter. The reference sample specified based on the reference sample index iIdx corresponds to an integer pel.

[0508] One or more reference samples may be specified to derive a prediction sample. Specifically, the position of the reference sample to derive the prediction sample may be specified in consideration of the gradient of the prediction mode. For example, the reference sample to derive the prediction sample may be specified using a reference sample index iIdx.

[0509] In this case, if the gradient of the intra prediction mode is not represented by one reference sample, a prediction sample may be generated by interpolating multiple reference samples. For example, if the gradient of the intra prediction mode is a value between the gradient between the prediction sample and a first reference sample and the gradient between the prediction sample and a second reference sample, the prediction sample may be obtained by interpolating the first reference sample and the second reference sample. That is, if an angular line according to the intra prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample may be obtained by interpolating adjacent reference samples to the left and right or above and below the position where the angular line passes.

[0510] Equation 32 below shows an example of obtaining a predicted sample based on a reference sample.

[0511] Equation 32

number

[0512] In Equation 32, P denotes a predicted sample, and Ref_1D denotes any one of the reference samples in the one-dimensional array. In such a case, the position of the reference sample may be determined based on the position (x, y) of the predicted sample and the reference sample index iIdx.

[0513] If the gradient of the intra prediction mode may be represented as one reference sample, the weighting value parameter ifact may be set as 0. Thus, Equation 32 may be simplified to the following Equation 33.

[0514] Equation 33

number

[0515] Furthermore, intra prediction may be performed on the current block based on multiple intra prediction modes, for example, intra prediction modes may be derived for different prediction samples, and prediction samples may be derived based on the intra prediction modes assigned to each prediction sample.

[0516] Alternatively, intra prediction modes may be derived for different regions, and intra prediction may be performed for each region based on the intra prediction mode assigned to each region. The regions may include at least one sample. At least one of the sizes or shapes of the regions may be adaptively determined based on at least one of the size, shape, or intra prediction mode of the current block. Alternatively, at the encoder and decoder, at least one of the sizes or shapes of the regions may be predefined regardless of the size or shape of the current block.

[0517] Alternatively, intra prediction may be performed based on multiple intra predictions, and a final prediction sample may be derived by averaging or weighting and computing multiple prediction samples obtained by multiple intra predictions. For example, intra prediction may be performed based on a first intra prediction mode to obtain a first prediction sample, and intra prediction may be performed based on a second intra prediction mode to obtain a second prediction sample. Then, a final prediction sample may be obtained by averaging or weighting and computing the first and second prediction samples. In this case, the weighting values ​​assigned to the first and second prediction samples may be determined by considering whether the first intra prediction mode is a non-angle / angle prediction mode and whether the second intra prediction mode is at least one of a non-angle / angle prediction mode or an intra prediction mode of a neighboring block.

[0518] The multiple intra prediction modes may be a combination of non-angular intra prediction modes and angular prediction modes, a combination of angular prediction modes, or a combination of non-angular prediction modes.

[0519] FIG. 44 is a diagram showing angles between angular intra prediction modes and a line parallel to the x-axis.

[0520] In the example shown in Figure 44, the angular prediction modes may exist between the bottom left diagonal direction and the top right diagonal direction. When described as the angle between the x-axis and the angular prediction modes, the angular prediction modes may exist between 45 degrees (bottom left diagonal direction) and -135 degrees (top right diagonal direction).

[0521] If the current block is non-square in shape, a prediction sample is derived based on the intra prediction mode of the current block using a reference sample that is away from the prediction sample among reference samples located on a corner line according to the intra prediction angle, rather than a reference sample that is close to the prediction sample.

[0522] FIG. 45 is a diagram showing an example of obtaining predicted samples when the current block is non-square.

[0523] For example, in the example shown in Figure 45(a), it is assumed that the current block is a non-square block whose width is greater than its height, and the intra prediction mode of the current block is an angular intra prediction mode whose angle is 0 to 45. In this case, when deriving a predicted sample A near the right column of the current block, a situation occurs in which a left reference sample L that is far from the predicted sample among reference samples in the angular mode located at the angle is substituted for an upper reference sample T that is close to the predicted sample.

[0524] As another example, in the example shown in Figure 45(b), it is assumed that the current block is a non-square block with its height greater than its width, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between -90 degrees and -135 degrees. In this case, when deriving a prediction sample A in the vicinity of the lower row of the current block, a situation occurs in which an upper reference sample T away from the prediction sample among reference samples in the angular mode located at the angle is substituted for a left reference sample L close to the prediction sample.

[0525] To solve the above problem, if the current block is non-square, the intra prediction mode of the current block may be replaced with an intra prediction mode of the opposite direction. Thus, for a non-square block, an angular prediction mode having an angle larger or smaller than the angle of the angular prediction mode shown in FIG. 41 may be used. Such an angular intra prediction mode may be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode refers to an angular intra prediction mode that exceeds the range of 45 degrees to -135 degrees.

[0526] FIG. 46 is a diagram showing the wide-angle intra prediction mode.

[0527] In the example shown in FIG. 46, the intra prediction modes of indexes -1 to -14 and the intra prediction modes of indexes 67 to 80 indicate wide-angle intra prediction modes.

[0528] Figure 46 shows 14 wide-angle intra prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) with angles less than -135 degrees, however, a greater or lesser number of wide-angle intra prediction modes may be defined.

[0529] When using a wide-angle intra prediction mode, the length of the upper reference sample is set as 2W+1, and the length of the left reference sample is set as 2H+1.

[0530] When using a wide-angle intra prediction mode, reference sample T may be used to predict sample A shown in Figure 45(a), and reference sample L may be used to predict sample A shown in Figure 45(b).

[0531] By adding the existing intra prediction modes and the N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 7 shows the intra direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0532] [Table 7]

[0533] If the current block is non-square and the intra prediction mode of the current block obtained in step S4002 is within a transformation range, the intra prediction mode of the current block may be transformed to a wide-angle intra prediction mode. The transformation range may be determined based on at least one of the size, shape, or ratio of the current block. The ratio may indicate a ratio between the width and height of the current block.

[0534] If the current block is a non-square block with width greater than height, a transform range may be set to the right upper diagonal intra prediction mode index (e.g., 66)~(right upper diagonal intra prediction mode index-N). N may be determined based on the ratio of the current block. If the intra prediction mode of the current block is within the transform range, the intra prediction mode may be converted to a wide-angle intra prediction mode. The transform may be performed by subtracting a predefined value from the intra prediction mode, and the predefined value may be the total number of intra prediction modes other than the wide-angle intra prediction mode (e.g., 67).

[0535] Based on the above embodiment, the 66th to 53rd intra prediction modes may be converted into the −1st to −14th wide-angle intra prediction modes, respectively.

[0536] If the current block is a non-square block with height greater than width, a transform range may be set to the lower left diagonal intra prediction mode index (e.g., 2) to (lower left diagonal intra prediction mode index + M). M may be determined based on the ratio of the current block. If the intra prediction mode of the current block is within the transform range, the intra prediction mode may be transformed to a wide-angle intra prediction mode. The transform may be performed by adding a predefined value to the intra prediction mode, and the predefined value may be the total number of angle intra prediction modes other than the wide-angle intra prediction mode (e.g., 65).

[0537] Based on the above embodiment, the 2nd to 15th intra prediction modes may be converted into the 67th to 80th wide-angle intra prediction modes, respectively.

[0538] Hereinafter, the intra prediction modes that are within the transform range are referred to as wide-angle intra substitution prediction modes.

[0539] The transform range may be determined based on the ratio of the current block. For example, Tables 8 and 9 show transform ranges when 35 intra prediction modes and 67 intra prediction modes other than the wide-angle intra prediction mode are defined, respectively.

[0540] [Table 8]

[0541] [Table 9]

[0542] In the examples shown in Tables 8 and 9, the number of wide-angle intra-permutation prediction modes in the transform range may vary according to the proportion of the current block.

[0543] By subdividing the ratio of the current block, the conversion range can be set as shown in Table 10.

[0544] [Table 10]

[0545] When using a multi-line intra prediction coding method for determining a non-adjacent reference sample line as a reference sample line of a current block or selecting one of a plurality of reference sample lines, it may be set not to use a wide-angle intra prediction mode. That is, even if the current block is non-square and the intra prediction mode of the current block is within the conversion range, the intra prediction mode of the current block may not be converted to the wide-angle intra prediction mode.

[0546] Alternatively, when the intra prediction mode of the current block is determined as the wide-angle intra prediction mode, the non-adjacent reference sample line may be set not to be used as the reference sample line of the current block, or the multi-line intra prediction coding method for selecting one of the multiple reference sample lines may be set not to be used. When the multi-line intra prediction coding method is not used, the adjacent reference sample line may be set as the reference sample line of the current block.

[0547] When the wide-angle intra prediction mode is not used, refW and refH are set as the sum of nTbW and nTbH. Therefore, in addition to the upper left reference sample, non-adjacent reference samples having a distance i from the current block may include (nTbW+nTbH+offsetX[i]) upper reference samples and (nTbW+nTbH+offsetY[i]) left reference samples. That is, non-adjacent reference samples having a distance i from the current block may include (2nTbW+2nTbH+offsetX[i]+offsetY[i]+1) reference samples. For example, if the value of whRatio is greater than 1, the value of offsetX is set to be greater than the value of offsetY. For example, the value of offsetX is set to be 1, and the value of offsetY is set to be 0. On the other hand, if the value of whRatio is less than 1, the value of offsetY is set to be greater than the value of offsetX. For example, set the value of offsetX as 0 and the value of offsetY as 1.

[0548] As the wide-angle intra-prediction mode is used in addition to the existing intra-prediction modes, the resources required for encoding the wide-angle intra-prediction mode increase, which may reduce the coding efficiency. Therefore, instead of directly encoding the wide-angle intra-prediction mode, a substitute intra-prediction mode related to the wide-angle intra-prediction mode is encoded to improve the coding efficiency.

[0549] For example, when the 67th wide-angle intra prediction mode is used to encode the current block, the intra prediction mode of the current block may be coded as the 67th wide-angle replacement intra prediction mode, i.e., number 2. When the current block is encoded using the -1th wide-angle intra prediction mode, the intra prediction mode of the current block may be coded as the -1th wide-angle replacement intra prediction mode, i.e., number 66.

[0550] The decoder may decode the intra-prediction mode of the current block to determine whether the decoded intra-prediction mode is within a transform range. If the decoded intra-prediction mode is a wide-angle substitute intra-prediction mode, the decoder may transform the intra-prediction mode to a wide-angle intra-prediction mode.

[0551] Alternatively, when encoding the current block in a wide-angle intra-prediction mode, a wider-angle intra-prediction mode may be directly encoded.

[0552] Encoding of the intra prediction mode may be realized based on the MPM list. Specifically, when encoding a neighboring block in a wide-angle intra prediction mode, the MPM may be set based on a wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode.

[0553] A residual picture may be derived by subtracting the predicted picture from the original picture. In this case, when the residual picture is transformed into the frequency domain, removing high frequency components in the frequency domain does not significantly degrade the subjective quality of the video. Therefore, transforming the values ​​of the high frequency components to relatively small values ​​or setting the values ​​of the high frequency components to zero has the effect of improving compression efficiency without causing noticeable visual distortion. To reflect the above characteristics, the current block may be transformed to decompose the residual picture into two-dimensional frequency components. The transformation may be performed using a transformation technique such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST).

[0554] While the DCT uses a cosine transform to decompose (or transform) the residual picture into two-dimensional frequency components, the DST uses a sine transform to decompose (or transform) the residual picture into two-dimensional frequency components. As a result of the transformation of the residual picture, the frequency components may be denoted as base pictures. For example, when performing a DCT transform on a block of size N×N, N 2 basic pattern components may be obtained. The size of each basic pattern component in a block of size N×N may be obtained by transformation. Depending on the transformation technique used, the size of the basic pattern component may be referred to as a DCT coefficient or a DST coefficient.

[0555] The DCT transform technique is mainly used for transforming pictures with a higher distribution of non-zero low-frequency components, while the DST transform technique is mainly used for pictures with a higher distribution of high-frequency components.

[0556] Furthermore, the residual picture may be transformed using transform techniques other than the DCT or DST.

[0557] Hereinafter, the process of transforming the residual picture into two-dimensional frequency components is referred to as two-dimensional picture transform. Also, the size of the basic pattern components obtained by the transform result is referred to as transform coefficients. For example, the transform coefficients may refer to DCT coefficients or DST coefficients. When the main transform and secondary transform to be described later are applied simultaneously, the transform coefficients may indicate the size of the basic pattern components generated by the secondary transform result.

[0558] A transform technique may be determined on a block-by-block basis. The transform technique may be determined based on at least one of a predictive coding mode of the current block, a size of the current block, or a shape of the current block. For example, if the current block is coded in an intra prediction mode and the size of the current block is smaller than N×N, the transform technique DST may be used to perform the transform. On the other hand, if the above conditions cannot be satisfied, the transform technique DCT may be used to perform the transform.

[0559] In the residual picture, the 2D picture transform may not be performed on some blocks. Not performing the 2D picture transform may be referred to as a transform skip. When the transform skip is applied, quantization may be applied to the residual values ​​that are not transformed.

[0560] After transforming the current block using a DCT or DST, the transformed current block may be transformed again. In this case, the transformation based on the DCT or DST may be defined as a primary transformation, and the process of transforming the block again using the primary transformation is called a secondary transformation.

[0561] The main transform may be performed using any one of a number of possible transform cores, for example, any one of DCT2, DCT8 or DCT7 may be used to perform the main transform.

[0562] Different transform cores may also be used for the horizontal and vertical directions, and information indicating the combination of horizontal and vertical transform cores may be transmitted by a signal via the bitstream.

[0563] The execution units of the primary transformation and the secondary transformation are different. For example, the primary transformation may be performed on an 8×8 block, and the secondary transformation may be performed on a sub-block of size 4×4 in the transformed 8×8 block. In such a case, the transformation coefficients of the spare area where the secondary transformation is not performed may be set to 0.

[0564] Alternatively, a primary transform may be performed on the 4x4 block, and a secondary transform may be performed on a region of size 8x8 that contains the transformed 4x4 block.

[0565] Information indicating whether or not to perform a secondary transformation may be transmitted by a signal via the bitstream.

[0566] In the decoder, an inverse transform (second inverse transform) of the secondary transform may be performed, and an inverse transform (first inverse transform) of the primary transform may be performed on the result. As a result of performing the second inverse transform and the first inverse transform, a residual signal of the current block may be obtained.

[0567] Quantization is used to reduce the energy of a block, and the quantization process includes dividing the transform coefficients by a specific constant, which may be derived from a quantization parameter, and the quantization parameter may be defined as a value between 1 and 63.

[0568] If the encoder performs transform and quantization, the decoder may obtain a residual block by inverse quantization and inverse transform. The decoder may add the prediction block and the residual block to obtain a reconstructed block of the current block.

[0569] When a reconstructed block of a current block is obtained, information loss occurring during quantization and encoding processes can be reduced by in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO) or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter is referred to as a first reconstructed block, and the reconstructed block after applying the in-loop filter is referred to as a second reconstructed block.

[0570] At least one of a deblocking filter, SAO, or ALF may be applied to the first reconstructed block to obtain a second reconstructed block. In this case, SAO or ALF may be applied after applying the deblocking filter.

[0571] The deblocking filter is used to reduce image quality degradation (blocking artifacts) that occurs at block boundaries when quantization is performed on a block-by-block basis. To apply the deblocking filter, a blocking strength (BS) between a first restored block and an adjacent restored block may be determined.

[0572] FIG. 47 is a flow chart showing the process of determining the blocking strength.

[0573] 47, P denotes the first reconstruction block, and Q denotes the adjacent reconstruction block, which may be adjacent to the left or above the current block.

[0574] The example shown in Figure 47 illustrates determining blocking strength based on the predictive coding mode taking into account P and Q, whether it contains non-zero transform coefficients, whether inter prediction is performed using the same reference picture, and whether the difference in motion vectors is greater than or equal to a threshold.

[0575] Based on the blocking strength, it may be decided whether to apply a deblocking filter. For example, if the blocking strength is 0, no filtering may be performed.

[0576] SAO is used to reduce ringing artifacts that occur when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset amount determined in consideration of the pattern of the first restored picture. Methods for determining the offset amount include edge offset (EO) or band offset. EO indicates a method for determining an offset of a current sample based on a pattern of surrounding pixels. BO indicates a method for applying the same offset to a pixel set having similar luminance values ​​in a region. Specifically, pixel luminance may be divided into 32 equal intervals, and pixels having similar luminance values ​​may be set as one set. For example, four adjacent bands out of the 32 bands may be set as one set, and the same offset amount may be applied to samples belonging to the four bands.

[0577] ALF is a method of generating a second restored picture by applying a filter of a predefined size / shape to a first restored picture or a restored picture to which a deblocking filter is applied. Equation 34 below shows an example of applying ALF.

[0578] Equation 34

number

[0579] Any one of predefined filter candidates may be selected in units of a picture, a coding tree unit, a coding block, a prediction block, or a transform block. Any one of the sizes or shapes of each filter candidate may be different.

[0580] FIG. 48 is a diagram showing predefined filter candidates.

[0581] In the example shown in FIG. 48, at least one of a 5×5, 7×7 and 9×9 diamond may be selected.

[0582] Only diamonds of size 5x5 can be used for the chromaticity components.

[0583] It is within the scope of the present invention to use the embodiments described with a focus on a decoding process or an encoding process for an encoding process or a decoding process. It is also within the scope of the present invention to modify the embodiments described according to a predetermined order in an order different from that in which they are described.

[0584] Although the embodiments are described based on a series of steps or flow charts, the embodiments do not limit the time sequence order of the present invention, and the steps may be executed simultaneously or according to another order as necessary. In addition, in the above embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagrams may be further realized as hardware devices or software, and multiple structural elements may be implemented in combination as a single hardware device or software. The embodiments may be implemented in the form of program instructions, which may be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., independently or in combination with the program instructions, data files, data structures, etc. Examples of computer-readable recording media may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. The hardware devices may be configured to operate as one or more software modules to perform the processes of the present invention, or vice versa. [Industrial Applicability]

[0585] The present invention may be applied to electronic devices that encode / decode video.

Claims

1. 1. A video decoding method, comprising: determining a partition type of a coding block when a first prediction and a second prediction corresponding to the coding block exist; deriving first motion information of the first prediction and second motion information of the second prediction of the coding block; obtaining a prediction sample in the coding block based on the first motion information and the second motion information; obtaining the first motion information of the first prediction based on a first merging candidate, the first merging candidate being selected from a plurality of merging candidates included in a merging candidate list by first index information; obtaining the second motion information of the second prediction based on a second merging candidate, the second merging candidate being selected from a plurality of merging candidates included in the merging candidate list by second index information; When a value of the second index information is greater than or equal to a value of the first index information, an index of the second merging candidate is equal to a value of the second index information plus 1.

2. The video decoding method of claim 1 , wherein if a value of the second index information is smaller than a value of the first index information, the index of the second merging candidate is equal to a value of the second index information.

3. 2. The video decoding method of claim 1, wherein when the prediction sample is included in a boundary area between the first prediction and the second prediction, the prediction sample is derived by weighting and calculating the first prediction sample derived based on the first motion information and the second prediction sample derived based on the second motion information.

4. 4. The method of claim 3, further comprising determining a first weighting value to be applied to the first predicted sample based on an x-axis coordinate and a y-axis coordinate of the predicted sample.

5. 5. The method of claim 4, further comprising: deriving a second weighting value to be applied to the second predicted sample by subtracting the first weighting value from a constant.

6. 1. A video encoding method, comprising: determining a partition type of a coding block when a first prediction and a second prediction corresponding to the coding block exist; deriving first motion information of the first prediction and second motion information of the second prediction of the coding block; obtaining a prediction sample in the coding block based on the first motion information and the second motion information; obtaining the first motion information of the first prediction based on a first merging candidate among a plurality of merging candidates included in a merging candidate list; obtaining the second motion information of the second prediction based on a second merging candidate among a plurality of merging candidates included in the merging candidate list; encoding first index information for specifying the first merging candidate and second index information for specifying the second merging candidate, A video encoding method, wherein if a value of the second index information is equal to or greater than a value of the first index information, an index of the second merging candidate is equal to a value of the second index information plus 1.

7. The video encoding method of claim 6 , wherein if the value of the second index information is smaller than the value of the first index information, the index of the second merging candidate is equal to the value of the second index information.

8. A video decoding device comprising an inter prediction unit, the inter prediction unit is used for determining a division type of the coding block when a first prediction and a second prediction corresponding to the coding block exist, deriving first motion information of the first prediction and second motion information of the second prediction of the coding block, and obtaining a prediction sample in the coding block based on the first motion information and the second motion information; obtaining the first motion information of the first prediction based on a first merging candidate, the first merging candidate being selected from a plurality of merging candidates included in a merging candidate list by first index information; obtaining the second motion information of the second prediction based on a second merging candidate, the second merging candidate being selected from a plurality of merging candidates included in the merging candidate list by second index information; When a value of the second index information is equal to or greater than a value of the first index information, an index of the second merging candidate is equal to a value that is equal to the value of the second index information plus 1.

9. The video decoding apparatus of claim 8 , wherein when a value of the second index information is smaller than a value of the first index information, an index of the second merging candidate is equal to a value of the second index information.

10. 1. A video encoding device, comprising: A memory storing instructions and a processor, the processor executing the instructions to determining a partition type of a coding block when a first prediction and a second prediction corresponding to the coding block exist; deriving first motion information of the first prediction and second motion information of the second prediction of the coding block; obtaining a prediction sample in the coding block based on the first motion information and the second motion information, wherein: obtaining the first motion information of the first prediction based on a first merging candidate among a plurality of merging candidates included in a merging candidate list; obtaining the second motion information of the second prediction based on a second merging candidate among a plurality of merging candidates included in the merging candidate list; encoding first index information for specifying the first merging candidate and second index information for specifying the second merging candidate, A video encoding apparatus, wherein if a value of the second index information is greater than or equal to a value of the first index information, an index of the second merging candidate is equal to a value of the second index information plus 1.

11. The video encoding apparatus of claim 10 , wherein when a value of the second index information is smaller than a value of the first index information, an index of the second merging candidate is equal to a value of the second index information.

Citation Information

Patent Citations

  • Image prediction method and device

    WO2020083403A1

  • Side information signaling for inter prediction with geometric partitioning

    WO2020094052A1

Cited By

  • Encoding / decoding method of video signal, and device for the same

    JP2025011299A