Image signal encoding / decoding method and apparatus therefor

Abstract

A video decoding method according to the present invention comprises the steps of: deriving an L0 motion vector and an L1 motion vector of a current block; deriving an L0 prediction sample point and an L1 prediction sample point of a first position in the current block based on the L0 motion vector and the L1 motion vector; determining whether to apply the bidirectional optical flow to the current block; when it is determined to apply the bi-directional optical flow, deriving a refined motion vector of the sub-block comprising the first position; deriving a refined prediction sample point of the first position based on the refined motion vector; and obtaining a prediction sample point of the first position by using the L0 prediction sample point, the L1 prediction sample point and the refined prediction sample point.

Description

[0001] This application is a divisional application of the invention patent with application date of May 4, 2020, application number 202080006956.6, and invention name “Image signal encoding / decoding method and device thereof”. Technical Field

[0002] The present disclosure relates to a video signal encoding / decoding method and a device thereof. Background Art

[0003] As display panels become larger, higher quality video services are required. The biggest problem with high-definition video services is the greatly increased amount of data. In order to solve the above problems, research on improving video compression rates is being actively conducted. As a representative example, the Joint Collaboration Team on Video Coding (JCT-VC) was established in 2009 by the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union Telecommunication Standardization Sector (ITU-T). JCT-VC proposed High Efficiency Video Coding (HEVC), a video compression standard with a compression performance approximately twice that of H.264 / AVC, and was approved as a standard on January 25, 2013. However, with the rapid development of high-definition video services, the performance of HEVC has gradually shown its limitations. Summary of the invention

[0004] Technical Purpose

[0005] An object of the present disclosure is to provide a method for determining a partition type of a coding block by considering a chroma block size when encoding / decoding a video signal, and an apparatus for performing the method.

[0006] An object of the present disclosure is to provide a prediction method using a block vector when encoding / decoding a video signal, and a device for performing the method.

[0007] An object of the present disclosure is to provide a method of adaptively determining whether to apply bidirectional optical flow when encoding / decoding a video signal, and a device for performing the method.

[0008] Technical objectives obtainable from the present disclosure are not limited to the above-mentioned technical objectives, and other unmentioned technical objectives can be clearly understood from the following description by a person of ordinary skill in the technical field to which the present disclosure belongs.

[0009] Technical Solution

[0010] A video signal decoding / encoding method according to the present disclosure may include: deriving an L0 motion vector and an L1 motion vector of a current block; deriving an L0 prediction sample and an L1 prediction sample of a first position in the current block based on the L0 motion vector and the L1 motion vector; determining whether a bidirectional optical flow (BDOF) is applied to the current block; when it is determined that the bidirectional optical flow is applied, deriving a refined motion vector of a sub-block including the first position; deriving a refined prediction sample of the first position based on the refined motion vector; and obtaining a prediction sample of the first position by using the L0 prediction sample, the L1 prediction sample and the refined prediction sample.

[0011] In the video signal decoding / encoding method according to the present disclosure, whether bidirectional optical flow is applied to the current block is determined based on whether the combined prediction method is applied to the current block, and when the combined prediction method is not applied to the current block, the BDOF encoding method is allowed.

[0012] In the video signal decoding / encoding method according to the present disclosure, the refined prediction sample may be derived based on a first difference between an L1 horizontal gradient and an L0 horizontal gradient and a second difference between an L1 vertical gradient and an L0 vertical gradient at a first position.

[0013] In the video signal decoding / encoding method according to the present disclosure, whether a bidirectional optical flow is applied to the current block may be determined based on at least one of a direction or an output order of an L0 reference picture and an L1 reference picture of the current block.

[0014] In the video signal decoding / encoding method according to the present disclosure, whether the bidirectional optical flow is applied to the current block may be determined based on whether at least one of the width or the height of the current block is equal to or greater than a threshold.

[0015] It should be understood that the foregoing summarized features are exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

[0016] Technical Effects

[0017] According to the present disclosure, encoding / decoding efficiency may be improved by determining a partition type of a coding block based on a chroma block size.

[0018] According to the present disclosure, encoding / decoding efficiency can be improved by obtaining prediction samples using a block vector.

[0019] According to the present disclosure, encoding / decoding efficiency can be improved by adaptively determining whether to apply bidirectional optical flow.

[0020] Effects obtainable from the present disclosure may not be limited to the above-mentioned effects, and other unmentioned effects may be clearly understood from the following description by a person of ordinary skill in the technical field to which the present disclosure pertains. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a diagram illustrating a block diagram of a video encoding device (encoder) according to an embodiment of the present disclosure.

[0022] Figure 2 is a diagram showing a block diagram of a video decoding device (decoder) according to an embodiment of the present disclosure.

[0023] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present disclosure.

[0024] Figure 4 is a diagram illustrating various partition types of a coding block.

[0025] Figure 5 is a diagram illustrating an example of aspects of partitioning a CTU.

[0026] Figure 6 is a diagram showing partitioning aspects of luma component blocks and chroma component blocks.

[0027] Figure 7 is a flowchart of an inter-frame prediction method according to an embodiment of the present disclosure.

[0028] Figure 8 is a diagram illustrating nonlinear motion of an object.

[0029] Fig. 9 is a flowchart of an inter-frame prediction method based on affine motion according to an embodiment of the present disclosure.

[0030] Fig.10 is a diagram showing the affine seed vector of each affine motion model.

[0031] Fig.11 is a diagram showing affine vectors of sub-blocks in a 4-parameter motion model.

[0032] Fig.12 is a flow chart of the process of deriving current block motion information in merge mode.

[0033] Fig.13 is a diagram illustrating candidate blocks for deriving merge candidates.

[0034] Fig.14 is a diagram for explaining the updating aspect of the motion information table.

[0035] Fig.15 is a diagram illustrating an updating aspect of a motion information table.

[0036] Fig.16 is a diagram showing an example in which the index of the pre-saved motion information candidate is updated.

[0037] Fig.17 is a diagram illustrating an example of performing redundancy check on only a part of merge candidates.

[0038] Fig.18 is a diagram illustrating an example in which a redundancy check with a specific merge candidate is omitted.

[0039] Fig.19 is a diagram illustrating an example in which a candidate block included in the same merge processing region as a current block is set to be unusable as a merge candidate.

[0040] Fig. 20 is a diagram illustrating an example of deriving a merge candidate of a current block when the current block is included in a merge processing region.

[0041] Fig.21 is a diagram showing a temporary motion information table.

[0042] Fig. 22 is a diagram showing an example in which a motion information table and a temporary motion information table are unified.

[0043] Fig.23 is a diagram illustrating an example of partitioning a coding block into a plurality of prediction units by using diagonal lines.

[0044] Fig.24 is a diagram illustrating an example in which a coding block is partitioned into two prediction units.

[0045] Fig.25 An example in which a coding block is partitioned into a plurality of prediction units of different sizes is shown.

[0046] Fig.26 is a diagram illustrating neighboring blocks used to derive partition mode merge candidates.

[0047] Fig. 27 is a diagram for explaining an example of determining availability of neighboring blocks per prediction unit.

[0048] Fig.28 and Fig.29 is a diagram illustrating an example of deriving a prediction sample based on a weighted sum operation of a first prediction sample and a second prediction sample.

[0049] Fig.30 is a diagram for explaining an aspect of prediction based on an intra block copy mode.

[0050] Fig.31 is a flowchart illustrating a prediction process of a current block based on an intra block copy mode according to an embodiment of the present disclosure.

[0051] Fig.32is a diagram illustrating the order in which IBC merge candidates are added to an IBC merge candidate list.

[0052] Fig.33 is a diagram illustrating an example of adding an IBC motion information candidate to an IBC merge candidate list.

[0053] Fig.34 is a diagram used to explain the updating aspects of the IBC motion information table.

[0054] Fig.35 is a diagram illustrating an unavailable area according to a location of a current block.

[0055] Fig.36 is a diagram showing an example in which reconstruction information of an encoded / decoded current block is stored in an IBC reference buffer.

[0056] Fig.37 is a diagram for explaining an example of encoding aspects of explaining block vector differences.

[0057] Fig.38 is a diagram illustrating an example of performing determination on each subblock whether to perform transform skip.

[0058] Fig.39 is a diagram showing an example in which sub-blocks use the same transform type.

[0059] Fig.40 is a flow chart illustrating a process for determining block strength.

[0060] Fig.41 Predefined filter candidates are shown. DETAILED DESCRIPTION

[0061] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0062] Image encoding and decoding are performed on a block basis. In an example, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, reconstruction, etc. may be performed on a coding block, a transform block, or a prediction block.

[0063] Hereinafter, the encoding / decoding target block is referred to as a “current block.” In an example, the current block may refer to a coding block, a transform block, or a prediction block according to a current encoding / decoding process.

[0064] In addition, the term "unit" used in this specification means a basic unit for performing a specific encoding / decoding process, and "block" can be understood to mean a sample array having a predetermined size. Unless otherwise specified, "block" and "unit" can be used interchangeably. In the example, the coding block and the coding unit can be understood to have the same meaning as each other in the examples described later.

[0065] Figure 1 is a diagram showing a block diagram of an image encoding device (encoder) according to an embodiment of the present disclosure.

[0066] Reference Figure 1 , the image encoding apparatus 100 may include a picture partition unit 110, prediction units 120 and 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0067] Figure 1 The components described in the figure are shown independently in order to illustrate different characteristic functions in the image encoding device, and the figure does not mean that each component is composed of separate hardware or a software unit. That is, each component is listed only for the convenience of explanation, at least two of the components may constitute one component, or one component may be partitioned into multiple components that can perform their functions. Even embodiments that integrate the various components and embodiments that divide the components are included in the scope of the present disclosure unless they depart from the spirit of the present disclosure.

[0068] In addition, some components are not essential components for performing the essential functions of the present disclosure, but are optional components only for improving performance. The present disclosure may be implemented using essential components for implementing the spirit of the present disclosure other than components only for improving performance, and a structure including only essential components other than optional components only for improving performance is also included in the scope of the present disclosure.

[0069] The picture partition unit 110 may partition the input picture into at least one processing unit. In this regard, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In the picture partition unit 110, a single picture may be partitioned into a plurality of combinations of a plurality of coding units, prediction units, and transform units, and the picture may be encoded by selecting a combination of coding units, prediction units, and transform units according to a predetermined condition (e.g., a cost function).

[0070] For example, a single picture may be partitioned into a plurality of coding units. In order to partition a picture into coding units, a recursive tree structure such as a quadtree structure may be used, and a coding unit derived from a root such as a single image or a maximum coding unit may be partitioned into other coding units and may have as many child nodes as the partitioned coding units. Coding units that are no longer partitioned according to certain restrictions become leaf nodes. That is, when it is assumed that only square partitions are available for a single coding unit, a single coding unit may be partitioned into up to four other coding units.

[0071] Hereinafter, in an embodiment of the present disclosure, a coding unit may be used as a unit for encoding, or may be used as a unit for decoding.

[0072] The prediction units may be obtained by partitioning a single coding unit into at least one square or rectangle having the same size, or may be partitioned into prediction units in such a manner that one prediction unit may be different in shape and / or size from another prediction unit.

[0073] In generating a prediction unit based on a coding block on which intra prediction is being performed, when the coding unit is not a minimum coding unit, intra prediction may be performed without performing partitioning into a plurality of N×N prediction units.

[0074] The prediction units 120 and 125 may include an inter-frame prediction unit 120 that performs inter-frame prediction and an intra-frame prediction unit 125 that performs intra-frame prediction. It may be determined whether inter-frame prediction or intra-frame prediction is performed on the prediction unit, and detailed information (e.g., intra-frame prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this regard, the processing unit on which the prediction is performed may be different from the processing unit on which the prediction method and its details are determined. For example, the prediction method, prediction mode, etc. may be determined based on the prediction unit, but the prediction may be performed based on the transform unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoding unit 165 using the residual value and may be sent to the decoder. When a specific encoding mode is used, the original block is encoded as it is without generating a prediction block through the prediction unit 120 or 125 and is sent to the decoding unit.

[0075] The inter-frame prediction unit 120 may predict the prediction unit based on information about at least one of a previous picture and a subsequent picture of the current picture, or in some cases, may predict the prediction unit based on information about some encoded regions in the current picture. The inter-frame prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0076] The reference picture interpolation unit may receive reference picture information from the memory 155, and may generate pixel information of integer pixels or smaller pixels from the reference picture. In the case of luma pixels, an 8-tap interpolation filter based on DCT with different coefficients may be used to generate pixel information about integer pixels or smaller pixels in units of 1 / 4 pixels. In the case of chroma signals, a 4-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information about integer pixels or smaller pixels in units of 1 / 8 pixels.

[0077] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. Various methods such as a full search based block matching algorithm (FBMA), a three-step search (TSS) algorithm, a new three-step search (NTS) algorithm, etc. may be used as a method for calculating a motion vector. The motion vector may have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixel. The motion prediction unit may predict the current prediction unit by changing the motion prediction method. Various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra-block copy method, etc. may be used as a motion prediction method.

[0078] The motion prediction unit 125 may generate a prediction unit based on information of reference pixels around the current block as pixel information in the current picture. When a neighboring block of the current prediction unit is a block on which inter-frame prediction is performed, and thus the reference pixel is a pixel on which inter-frame prediction is performed, the reference pixel included in the block on which inter-frame prediction is performed may be replaced with information about the reference pixel of the neighboring block on which intra-frame prediction is performed. In other words, when a reference pixel is unavailable, at least one reference pixel among available reference pixels may be used to replace the unavailable reference pixel information.

[0079] The prediction mode under the intra prediction may include a directional prediction mode that uses reference pixel information according to a prediction direction when performing prediction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luma information may be different from the mode for predicting chroma information. In order to predict chroma information, information about the intra prediction mode for predicting luma information or information about the predicted luma signal may be used.

[0080] In the process of performing intra prediction, when the size of the prediction unit is the same as the size of the transformation unit, intra prediction can be performed on the prediction unit based on pixels located to the left, above the left, and above the prediction unit. However, in the process of performing intra prediction, when the size of the prediction unit is different from the size of the transformation unit, intra prediction can be performed by using reference pixels based on the transformation unit. In addition, intra prediction using N×N partitions can be used only for the minimum coding unit.

[0081] In the intra prediction method, a prediction block may be generated after an adaptive intra smoothing (AIS) filter is applied to reference pixels according to a prediction mode. The type of the AIS filter applied to the reference pixels may vary. In order to perform the intra prediction method, an intra prediction mode for a current prediction unit may be predicted from intra prediction modes of prediction units existing around the current prediction unit. In the process of predicting a prediction mode for a current prediction unit by using mode information predicted from a neighboring prediction unit, when the intra prediction mode for the current prediction unit is the same as the intra prediction mode of the neighboring prediction unit, information indicating that the current prediction unit has the same prediction mode as the neighboring prediction unit may be sent by using predetermined flag information. When the prediction mode for the current prediction unit is different from the prediction mode of the neighboring prediction unit, entropy coding may be performed to encode information about the prediction mode for the current block.

[0082] In addition, a residual block including information on a residual value that is a difference value between a prediction unit on which prediction is performed by the prediction unit 120 or 125 and an original block of the prediction unit may be generated. The generated residual block may be input to the transform unit 130.

[0083] The transform unit 130 may perform a transform on the residual block by using a transform method such as discrete cosine transform (DCT) or discrete sine transform (DST), wherein the residual block includes information about a residual value between the original block and the prediction unit generated by the prediction unit 120 or 125. In this regard, the DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST to perform a transform on the residual block may be determined based on information about an intra-frame prediction mode of the prediction unit used to generate the residual block. The transform for the residual block may be skipped. A flag indicating whether to skip the transform for the residual block may be encoded. Transform skipping may be allowed for a residual block whose size is less than or equal to a threshold, a residual block of a luminance component in a 4:4:4 format, or a residual block of a chrominance component.

[0084] The quantization unit 135 may perform quantization on the value transformed into the frequency domain by the transformation unit 130. The quantization coefficient may vary according to the importance or block of the image. The value calculated in the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0085] The rearrangement unit 160 may perform rearrangement on the transform coefficients with respect to the quantized residual values.

[0086] The rearrangement unit 160 may change the coefficients in the two-dimensional block form into coefficients in the one-dimensional vector form by a coefficient scanning method. For example, the rearrangement unit 160 may scan from the DC coefficient to the coefficients in the high frequency domain by using a zigzag scanning method so as to change the coefficients into the one-dimensional vector form. Depending on the size of the transform unit and the intra-frame prediction mode, vertical direction scanning of scanning the coefficients in the two-dimensional block form along the column direction or horizontal direction scanning of scanning the coefficients in the two-dimensional block form along the row direction may be used instead of the zigzag scanning. In other words, which scanning method to use among the zigzag scanning, the vertical direction scanning, and the horizontal direction scanning may be determined according to the size of the transform unit and the intra-frame prediction mode.

[0087] The entropy encoding unit 165 may perform entropy encoding based on the value calculated by the rearrangement unit 160. The entropy encoding may use various encoding methods, for example, exponential Golomb encoding, context-adaptive variable length coding (CAVLC), or context-adaptive binary arithmetic coding (CABAC).

[0088] The entropy encoding unit 165 may encode various types of information obtained from the rearrangement unit 160 and the prediction units 120 and 125 (such as information about residual value coefficients and information about block types of coding units, information about prediction modes, information about partition units, information about prediction units, and information about transform units, information about motion vectors, information about reference frames, information about block interpolation, filtering information, and the like).

[0089] The entropy encoding unit 165 may entropy encode coefficients of the coding unit input from the rearrangement unit 160 .

[0090] The inverse quantization unit 140 may perform inverse quantization on the value quantized in the quantization unit 135, and the inverse transform unit 145 may perform inverse transform on the value transformed in the transform unit 130. The residual value generated by the inverse quantization unit 140 and the inverse transform unit 145 may be added to a prediction unit predicted by a motion estimation unit, a motion compensation unit, or an intra prediction unit included in the prediction units 120 and 125 in order to generate a reconstructed block.

[0091] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive in-loop filter (ALF).

[0092] The deblocking filter can remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform the deblocking operation, it can be determined whether to apply the deblocking filter to the current block based on the pixels included in the rows or columns included in the block. When the deblocking filter is applied to the block, a strong filter or a weak filter is applied according to the required deblocking filter strength. In addition, in the process of applying the deblocking filter, when horizontal filtering and vertical filtering are performed, the horizontal filtering and the vertical filtering can be configured to be processed in parallel.

[0093] The offset correction unit can correct the original image by offsetting in pixels for the image on which the deblocking operation is performed. In order to perform offset correction on a specific picture, a method of applying an offset to an area determined after partitioning the pixels of the image into a predetermined number of areas, or a method of applying an offset according to edge information of each pixel can be used.

[0094] Adaptive in-loop filtering (ALF) may be performed based on a value obtained by comparing a filtered reconstructed image with an original image. Pixels included in an image may be partitioned into predetermined groups, a filter to be applied to each group may be determined, and filtering may be performed separately for each group. Information on whether ALF is applied may be sent for each coding unit (CU) of a luminance signal, and the shape and filter coefficients of the ALF filter to be applied may vary based on each block. Alternatively, an ALF filter having the same shape (fixed shape) may be applied regardless of the characteristics of the block to which the filter is applied.

[0095] In the memory 155, a reconstructed block or a reconstructed picture calculated by the filter unit 150 may be stored. The stored reconstructed block or the reconstructed picture may be provided to the prediction unit 120 or 125 when inter prediction is performed.

[0096] Figure 2 is a diagram showing a block diagram of an image decoding device (decoder) according to an embodiment of the present disclosure.

[0097] Reference Figure 2 , the image decoding apparatus 200 may include an entropy decoding unit 210 , a rearrangement unit 215 , an inverse quantization unit 220 , an inverse transform unit 225 , prediction units 230 and 235 , a filter unit 240 , and a memory 245 .

[0098] When an image bitstream is input from an encoder, the input bitstream may be decoded according to an inverse process of the image encoding apparatus.

[0099] The entropy decoding unit 210 may perform entropy decoding according to an inverse process of entropy encoding performed by an entropy encoding unit of an image encoder. For example, in association with a method performed by an image encoder device, various methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), or context adaptive binary arithmetic coding (CABAC) may be applied.

[0100] The entropy decoding unit 210 may decode information about intra prediction and inter prediction performed by the encoder.

[0101] The rearrangement unit 215 may perform rearrangement on the bit stream entropy-decoded by the entropy decoding unit 210 based on the rearrangement method used in the encoder. The coefficients represented in the form of a one-dimensional vector may be reconstructed, and the coefficients represented in the form of a one-dimensional vector may be rearranged into coefficients in the form of a two-dimensional block. The rearrangement unit 215 may perform rearrangement by receiving information related to coefficient scanning performed in the encoder and performing inverse scanning based on the scanning order performed in the encoder.

[0102] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter received from the encoder and the coefficient value of the rearranged block.

[0103] The inverse transform unit 225 may perform an inverse transform (i.e., inverse DCT or inverse DST) for the transform (i.e., DCT or DST) performed on the quantization result by the transform unit in the image encoder. In this regard, the DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Optionally, when the transform is skipped in the image encoder, the inverse transform is also not performed in the inverse transform unit 225. The inverse transform may be performed based on the transform unit determined by the image encoder. The inverse transform unit 225 of the image decoder may selectively perform a transform method (e.g., DCT or DST) according to a plurality of pieces of information (such as a prediction method, a size of a current block, a prediction direction, etc.).

[0104] The prediction unit 230 or 235 may generate a prediction block based on information related to the prediction block received from the entropy decoding unit 210 and information about a previously decoded block or picture received from the memory 245 .

[0105] As described above, like the operation of the image encoder, in the process of performing intra-frame prediction, when the size of the prediction unit is the same as the size of the transformation unit, intra-frame prediction can be performed on the prediction unit based on the pixels located on the left, upper left, and above the prediction unit. However, in the process of performing intra-frame prediction, when the size of the prediction unit is different from the size of the transformation unit, intra-frame prediction can be performed by using reference pixels based on the transformation unit. In addition, intra-frame prediction using N×N partitions can be used only for the minimum coding unit.

[0106] The prediction units 230 and 235 may include a PU determination module, an inter prediction unit, and an intra prediction unit. The PU determination unit may receive various types of information (such as information about the prediction unit, information about the prediction mode of the intra prediction method, information about the motion prediction of the inter prediction method, etc.) input from the entropy decoding unit 210, may divide the prediction unit in the current coding unit, and may determine whether to perform inter prediction or intra prediction on the prediction unit. By using the information required for inter prediction of the current prediction unit received from the image encoder, the inter prediction unit 230 may perform inter prediction on the current prediction unit based on information about at least one of a previous picture and a subsequent picture of the current picture including the current prediction unit. Optionally, inter prediction may be performed based on information about some pre-reconstructed areas in the current picture including the current prediction unit.

[0107] In order to perform inter prediction, which method of the skip mode, the merge mode, the AMVP mode, or the intra block copy mode is used as a motion prediction method for a prediction unit included in the coding unit may be determined based on the coding unit.

[0108] The intra prediction unit 235 may generate a prediction block based on information about pixels within the current picture. When the prediction unit is a prediction unit on which intra prediction has been performed, intra prediction may be performed based on information about the intra prediction mode of the prediction unit received from the image encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation module, or a DC filter. The AIS filter may perform filtering on the reference pixels of the current block, and may determine whether to apply the filter based on the prediction mode for the current prediction unit. When AIS filtering is performed on the reference pixels of the current block, the prediction mode of the prediction unit received from the image encoder and information about the AIS filter may be used. When the prediction mode for the current block is a mode in which AIS filtering is not applied, the AIS filter may not be applied.

[0109] When the prediction mode of the prediction unit is a prediction mode for performing intra prediction based on a pixel value obtained by interpolating reference pixels, the reference pixel interpolation unit may interpolate the reference pixels so as to generate reference pixels having integer units or smaller units. When the prediction mode for the current prediction unit is a prediction mode for generating a prediction block without interpolating reference pixels, the reference pixels may not be interpolated. When the prediction mode for the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0110] The reconstructed block or the reconstructed picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction module, and an ALF.

[0111] Information about whether a deblocking filter has been applied to a corresponding block or picture and information about whether a strong filter or a weak filter is applied when the deblocking filter is applied may be received from the image encoder. A deblocking filter of an image decoder may receive information about the deblocking filter from the image encoder, and the image decoder may perform deblocking filtering on the corresponding block.

[0112] The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image when encoding is performed, information about the offset value, and the like.

[0113] ALF may be applied to the coding unit based on information on whether ALF is applied, information on an ALF coefficient, etc. received from the encoder. The above ALF information may be provided by being included in a specific parameter set.

[0114] In the memory 245 , the reconstructed picture or the reconstructed block may be stored in order to be used as a reference picture or a reference block, and the reconstructed picture may be provided to the output unit.

[0115] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present disclosure.

[0116] The largest coding block may be defined as a coding tree block. A single picture may be partitioned into a plurality of coding tree units (CTUs). A CTU may be a coding unit of the largest size and may be referred to as a largest coding unit (LCU). Figure 3 is a diagram showing an example in which a single picture is partitioned into a plurality of CTUs.

[0117] The size of the CTU may be defined at a picture level or a sequence level. Similarly, information indicating the size of the CTU may be signaled through a picture parameter set or a sequence parameter set.

[0118] In an example, the size of the CTU for the entire picture within the sequence may be set to 128×128. Alternatively, any one of 128×128 or 256×256 may be determined as the size of the CTU at a picture level. In an example, the CTU may be set to have a size of 128×128 in a first picture, and may be set to have a size of 256×256 in a second picture.

[0119] A coding block may be generated by partitioning the CTU. A coding block represents a basic unit for performing encoding / decoding. In an example, prediction or transformation may be performed for each coding block, or a prediction coding mode may be determined for each coding block. In this regard, the prediction coding mode represents a method for generating a predicted image. In an example, the prediction coding mode may include intra prediction, inter prediction, current picture reference (CPR), intra block copy (IBC), or combined prediction. For a coding block, a prediction block of the coding block may be generated by using a prediction coding mode of at least one of intra prediction, inter prediction, current picture reference, or combined prediction.

[0120] Information indicating the prediction coding mode for the current block may be signaled in the bitstream. In an example, the information may be a 1-bit flag indicating whether the prediction coding mode is an intra mode or an inter mode. When the prediction coding mode for the current block is determined to be an inter mode, current picture reference or combined prediction may be available.

[0121] The current picture reference is to set the current picture as the reference picture and obtain the prediction block of the current block from the area that has been encoded / decoded in the current picture. In this regard, the current picture represents the picture including the current block. Information indicating whether the current picture reference is applied to the current block can be sent by signal in the bitstream. In an example, the information can be a 1-bit flag. When the flag is true, the prediction coding mode for the current block can be determined as the current picture reference, and when the flag is false, the prediction coding mode for the current block can be determined as inter-frame prediction.

[0122] Alternatively, the prediction coding mode for the current block may be determined based on the reference picture index. In an example, when the reference picture index indicates the current picture, the prediction coding mode for the current block may be determined as the current picture reference. When the reference picture index indicates a picture other than the current picture, the prediction coding mode for the current block may be determined as the inter-frame prediction. In other words, the current picture reference is a prediction method using information about an area that has been encoded / decoded within the current picture, and the inter-frame prediction is a prediction method using information about another picture that has been encoded / decoded.

[0123] Combined prediction refers to a combined coding mode that combines at least two of intra prediction, inter prediction, and current picture reference. In the example, when combined prediction is applied, a first prediction block may be generated based on any one of intra prediction, inter prediction, or current picture reference, and a second prediction block may be generated based on another one of intra prediction, inter prediction, or current picture reference. When the first prediction block and the second prediction block are generated, a final prediction block may be generated by calculating an average or weighted sum of the first prediction block and the second prediction block. Information indicating whether combined prediction is applied to the current block may be signaled in the bitstream. The information may be a 1-bit flag.

[0124] Figure 4 is a diagram illustrating various partition types of a coding block.

[0125] The coding block may be partitioned into a plurality of coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. The partitioned coding block may be partitioned into a plurality of coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0126] Quadtree partitioning represents a method of partitioning the current block into four blocks. As a result of quadtree partitioning, the current block can be partitioned into four square partitions (refer to Figure 4 (a) "SPLIT_QT").

[0127] Binary tree partitioning refers to a method of partitioning a current block into two blocks. An operation of partitioning a current block into two blocks along a vertical direction (i.e., using a vertical line passing through the current block) may be referred to as vertical binary tree partitioning, and an operation of partitioning a current block into two blocks along a horizontal direction (i.e., using a horizontal line passing through the current block) may be referred to as horizontal binary tree partitioning. As a result of the binary tree partitioning, the current block may be partitioned into two non-square partitions. Figure 4 "SPLIT_BT_VER" of (b) is a diagram showing the result of binary tree partitioning in the vertical direction. Figure 4 "SPLIT_BT_HOR" of (c) is a diagram showing the result of binary tree partitioning in the horizontal direction.

[0128] Ternary tree partitioning means a method of partitioning a current block into three blocks. The operation of partitioning a current block into three blocks along a vertical direction (i.e., using two vertical lines passing through the current block) may be referred to as vertical ternary tree partitioning, and the operation of partitioning a current block into three blocks along a horizontal direction (i.e., using two horizontal lines passing through the current block) may be referred to as horizontal ternary tree partitioning. As a result of the ternary tree partitioning, the current block may be partitioned into three non-square partitions. In this regard, the width / height of the partition located at the center of the current block may be twice the width / height of the other partitions. Figure 4"SPLIT_TT_VER" of (d) is a diagram showing the result of ternary tree partitioning in the vertical direction, Figure 4 "SPLIT_TT_HOR" of (e) is a diagram showing the result of ternary tree partitioning in the horizontal direction.

[0129] The number of partitions of a CTU may be defined as a partition depth. The maximum partition depth of a CTU may be determined at a sequence level or a picture level. Therefore, the maximum partition depth of a CTU may vary based on a sequence or a picture.

[0130] Optionally, the maximum partition depth may be determined independently for each partitioning method. In an example, the maximum partition depth allowed for quadtree partitioning may be different from the maximum partition depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0131] The encoder may signal information indicating at least one of a partition type and a partition depth of a current block in a bitstream, and the decoder may determine a partition type and a partition depth of a CTU based on the information obtained by parsing the bitstream.

[0132] Figure 5 is a diagram illustrating an example of one aspect of partitioning a CTU.

[0133] An operation of partitioning a coding block by using quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning may be referred to as multitree partitioning.

[0134] A coding block generated by partitioning a coding block by applying multi-tree partitioning may be referred to as a sub coding block. When the partition depth of the coding block is k, the partition depth of the sub coding block is set to k+1.

[0135] In contrast, for a coding block whose partition depth is k+1, the coding block whose partition depth is k may be referred to as a parent coding block.

[0136] The partition type of the current coding block may be determined based on at least one of the partition type of the parent coding block and the partition type of the adjacent coding block. In this regard, the adjacent coding block may be a block adjacent to the current coding block, and may include at least one of an upper adjacent block, a left adjacent block, or an adjacent block adjacent to the upper left corner of the current coding block. In this regard, the partition type may include whether quadtree partitioning is applied, whether binary tree partitioning is applied, the direction of binary tree partitioning, whether ternary tree partitioning is applied, or the direction of ternary tree partitioning.

[0137] In order to determine the partition type of the coding block, information indicating whether the coding block is partitioned may be signaled in the bitstream. This information is a 1-bit flag "split_cu_flag", and when the flag is true, it may indicate that the coding block is partitioned by a multi-tree partitioning method.

[0138] When split_cu_flag is true, information indicating whether the coding block is partitioned by quadtree partitioning may be signaled in the bitstream. This information is a 1-bit flag split_qt_flag, and when this flag is true, the coding block may be partitioned into four blocks.

[0139] In the example, Figure 5 In the example shown in , the CTU is partitioned by quadtree partitioning, and thus four coding blocks with a partition depth of 1 are generated. In addition, it is shown that quadtree partitioning is applied again to the first coding block and the fourth coding block among the four coding blocks generated by quadtree partitioning. As a result, four coding blocks with a partition depth of 2 can be generated.

[0140] In addition, by applying quadtree partitioning again to the coding block with a partition depth of 2, a coding block with a partition depth of 3 may be generated.

[0141] When quadtree partitioning is not applied to a coding block, it may be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block according to at least one of the size of the coding block, whether the coding block is located at a picture boundary, the maximum partition depth, or the partition type of the neighboring block. When it is determined that binary tree partitioning or ternary tree partitioning is performed on the coding block, information indicating the partition direction may be sent by signal in the bitstream. The information may be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it may be determined whether the partition direction is vertical or horizontal. In addition, information indicating which of binary tree partitioning or ternary tree partitioning is applied to the coding block may be sent by signal in the bitstream. The information may be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it may be determined whether binary tree partitioning is applied to the coding block or ternary tree partitioning is applied to the coding block.

[0142] In the example, Figure 5 In the example shown in , vertical binary tree partitioning is applied to a coding block with a partition depth of 1, vertical ternary tree partitioning is applied to a left coding block among the coding blocks generated by the vertical binary tree partitioning, and vertical binary tree partitioning is applied to a right coding block.

[0143] An aspect of applying at least one of binary tree partitioning, ternary tree partitioning, or quadtree partitioning to the current block may be referred to as a partition type of the current block.

[0144] Information indicating whether the partition type of the luminance component block and the partition type of the chrominance component block are independent of each other may be signaled in the bitstream. In an example, the syntax element dual_tree_flag may be signaled in the bitstream. When the value of the flag dual_tree_flag is 1, it indicates that the partition type of the luminance component block and the partition type of the chrominance component block are independent of each other. When the partition type of the luminance component block and the partition type of the chrominance component block are independent, information for determining the partition type of the luminance component block and information for determining the partition type of the chrominance component block may be signaled respectively. When the value of the flag dual_tree_flag is 0, it indicates that the partition type of the chrominance component block is based on the partition type of the luminance component block. In this case, the signaling of the information indicating the partition type of the chrominance component block may be omitted, and only the information for determining the partition type of the luminance component block may be signaled. The partition type of the chrominance component block may be set to be the same as the partition type of the luminance component block.

[0145] When the partition type of the luminance component block and the partition type of the chrominance component block are determined to be independent of each other, it can be called a dual-tree mode or a separate mode. On the other hand, when the partition type of the chrominance component block is determined to be based on the partition type of the luminance component block, it can be called a single-tree mode or a dependent mode.

[0146] A partition type applicable to a coding block may be determined based on at least one of the number of chroma samples included in the chroma component block or a size or a shape of the chroma component block.

[0147] In an example, a partition type in which the number of chroma samples to be included in the chroma component block becomes less than a threshold value may be determined to be unavailable for the coding block. When the minimum number of chroma samples that the chroma component block should include is 16, the partition type in which the number of chroma samples becomes less than 16 may be set to be unavailable for the current block.

[0148] In an example, in single tree mode, when a partition type of a luma component block is applied to a chroma component block, when the number of chroma samples to be included in the chroma component block becomes less than a threshold, the partition type may be set not to be applied to the luma component block.

[0149] Alternatively, in the dual-tree mode, a partition type for which the number of chroma samples included in the chroma component block becomes smaller than a threshold value may be set to be not applied to the chroma component block.

[0150] Whether a specific partition type can be applied to the current block may be determined by considering the prediction mode of the current block. In an example, when the prediction mode of the current block indicates intra-frame prediction, the partition type in which the number of chroma samples included by the chroma component block becomes less than a threshold value may be set to be unavailable for the current block. On the other hand, when the prediction mode of the current block indicates inter-frame prediction, the partition type in which the number of chroma samples included by the chroma component block becomes less than a threshold value may be set to be available for the current block.

[0151] Information for determining the threshold value may be signaled in the bitstream. In an example, information for determining the threshold value may be signaled at a sequence, picture, slice, or block level. The block level represents a coding tree unit, a coding block, or a prediction unit. The information may be a syntax of smallest_chroma_unit_minus4 of a value derived by subtracting an offset from a value of Log2 of the threshold value. In an example, when the threshold value is 32, the value of the syntax of smallest_chroma_unit_minus4 may be set to 1. The threshold value may be derived by a syntax as in Equation 1 below.

[0152] [Equation 1]

[0153] 1<<(smallest_chroma_unit_minus4+4)

[0154] Optionally, the threshold value may be predefined in the encoding device and the decoding device.

[0155] Alternatively, the threshold may be adaptively determined based on at least one of a prediction mode, a color format, or a bit depth.

[0156] A chroma component block including a minimum number of chroma samples may be referred to as a chroma minimum unit block. In an example, when the threshold is 16, a chroma component coding block including 16 samples may be referred to as a chroma minimum unit coding block.

[0157] Optionally, the partition type in which the size of the chroma component block becomes smaller than a threshold value may be determined to be unavailable for coding the block. In this case, the size of the block may represent at least one of the width or height of the block or a value calculated based on the product of the width and height (e.g., taking the value of Log2 of the product of the width and height).

[0158] In an example, when at least one of the width or height of the chroma component block is equal to or less than a threshold value, the binary tree partition may be set to be disallowed. In this case, when the width of the chroma component block is equal to or less than the threshold value, the binary tree partition in the vertical direction may be set to be disallowed, and when the height of the chroma component block is equal to or less than the threshold value, the binary tree partition in the horizontal direction may be set to be disallowed.

[0159] Optionally, when at least one of the width or height of the chroma component block is equal to or less than a threshold value, the ternary tree partition may be set to be disallowed. In this case, when the width of the chroma component block is equal to or less than the threshold value, the ternary tree partition in the vertical direction may be set to be disallowed, and when the height of the chroma component block is equal to or less than the threshold value, the binary tree partition in the horizontal direction may be set to be disallowed.

[0160] The threshold for binary tree partitioning and the threshold for ternary tree partitioning may be the same or different. In an example, the threshold for binary tree partitioning may be 4, and the threshold for ternary tree partitioning may be 8.

[0161] In the above-mentioned example, it is explained that in the single tree mode, a partition type in which the number of chroma samples included in the chroma component block becomes smaller than the threshold value is not applied to the luma component block.

[0162] In another example different from the description, the partition type at which the number of chroma samples to be included in the chroma component block becomes smaller than a threshold value may be set to be valid only for the luma component block and invalid for the chroma component blocks.

[0163] In other words, when a chroma component block corresponding to a luminance component block is a chroma minimum unit coding block, additional partitioning may be set to be allowed for the luminance component block but not allowed for the chroma component blocks.

[0164] Figure 6 is a diagram showing partitioning aspects of luma component blocks and chroma component blocks.

[0165] exist Figure 6 In the example of FIG. 1 , thin lines represent the partitioning aspects of the luminance component blocks, and thick lines represent the partitioning aspects of the chrominance component blocks. In addition, the size of the blocks shown in the figure is calculated based on the chrominance samples. The size of the luminance component blocks corresponding to the size of the blocks shown can be determined according to the color format.

[0166] For ease of description, assume the threshold is 16.

[0167] exist Figure 6 In the example shown in (a), when the ternary tree partition is applied to the luma component block, it can be determined whether the ternary tree partition will also be applied to the chroma component block. Specifically, when the ternary tree partition is to be applied, it can be determined whether the ternary tree partition can be applied to the chroma component block based on whether a chroma component block smaller than the chroma minimum unit coding block is generated.

[0168] If ternary tree partitioning is applied to a chroma component block including 32 chroma samples, the chroma component block will be partitioned into 2 blocks including 8 chroma samples and 1 block including 16 chroma samples. In other words, if ternary tree partitioning is applied to a chroma component block including 32 chroma samples, a chroma component block including chroma samples less than the threshold value (16) will be generated. Therefore, in Figure 6 In the example shown in (a), ternary tree partitioning may be set to be applied only to luma component blocks and may be set not to be applied to chroma component blocks.

[0169] exist Figure 6 In the example shown in (b), when the binary tree partition is applied to the luminance component block, it can be determined whether the binary tree partition will also be applied to the chrominance component block. Specifically, when the binary tree partition is to be applied, it can be determined whether the binary tree partition can be applied to the chrominance component block based on whether a chrominance component block smaller than the chrominance minimum unit coding block is generated.

[0170] If the binary tree partitioning is applied to a chroma component block including 32 chroma samples, the chroma component block will be partitioned into 2 blocks including 16 chroma samples. In other words, if the binary tree partitioning is applied to a chroma component block including 32 chroma samples, a chroma component block including the same number of chroma samples as the threshold value (16) will be generated. Therefore, the binary tree partitioning applied to the luma component block can also be applied to the chroma component block.

[0171] If the binary tree partition is applied again to the chroma component block including 16 chroma samples, the chroma component block will be partitioned into 2 blocks including 8 chroma samples. In other words, if the binary tree partition is applied to the chroma component block including 16 chroma samples, a chroma component block including chroma samples less than the threshold value (16) will be generated. Therefore, the binary tree partition can be set to be applied only to the luma component block, and can be set not to be applied to the chroma component block.

[0172] As a result, Figure 6 In the example shown in (b), the first binary tree partitioning is applied to both the luma component block and the chroma component block, but the additional binary tree partitioning applied again to the block generated by the first binary tree partitioning is applied only to the luma component block and not to the chroma component block.

[0173] As in the example shown, whether the partition type applied to the luma component block is applicable to the chroma component block may be determined based on whether the partition to be generated when the partition type is applied to the chroma component block includes chroma samples less than a threshold value. In other words, when the number of chroma samples included by the chroma component block is greater than twice the threshold value, binary tree partitioning may be applied, and when the number of chroma samples included by the chroma component block is greater than four times the threshold value, ternary tree partitioning may be applied. In the example, when the threshold value is 16, binary tree partitioning may be applied when the number of chroma samples included by the chroma component block is equal to or greater than 32, and ternary tree partitioning may be applied when the number of chroma samples included by the chroma component block is equal to or greater than 64.

[0174] At least one of a sub-partition intra prediction coding method (ISP), a position-based prediction sample modification method (PDPC), or a coding method based on multiple reference sample lines may be set to not be applied to the chroma minimum unit block. When the coding method based on multiple reference sample lines is not applied, it may mean that only reference sample lines adjacent to the current block may be selected, and reference sample lines not adjacent to the current block may not be selected.

[0175] Inter prediction is a prediction coding mode that predicts a current block by using information about a previous picture. In an example, a block located at the same position as the current block in the previous picture (hereinafter referred to as a co-located block) may be set as a prediction block of the current block. Hereinafter, a prediction block generated based on the co-located block of the current block may be referred to as a co-located prediction block.

[0176] In contrast, when an object existing in a previous picture has moved to another position in a current picture, the current block can be effectively predicted by using the motion of the object. For example, when the moving direction and size of the object are determined by comparing the previous picture with the current picture, a prediction block (or a prediction image) of the current block can be generated according to the motion information of the object. Hereinafter, the prediction block generated by using the motion information may be referred to as a motion prediction block.

[0177] The residual block may be generated by subtracting the prediction block from the current block. In this regard, in the case where the object moves, the energy of the residual block may be reduced by using the motion prediction block instead of using the co-located prediction block, and thus the compression performance of the residual block may be improved.

[0178] As above, the operation of generating a prediction block by using motion information may be referred to as motion estimation prediction. In most inter predictions, a prediction block may be generated based on motion compensation prediction.

[0179] The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, and a bidirectional weighting factor index. The motion vector represents the motion direction and magnitude of an object. The reference picture index specifies a reference picture of the current block among the reference pictures included in the reference picture list. The prediction direction indicates any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of L0 direction motion information and L1 direction motion information may be used according to the prediction direction of the current block. The bidirectional weighting factor index specifies a weighting factor applied to the L0 prediction block and a weighting factor applied to the L1 prediction block.

[0180] Figure 7 is a flowchart of an inter-frame prediction method according to an embodiment of the present disclosure.

[0181] Reference Figure 7 The inter-frame prediction method includes: determining an inter-frame prediction mode for a current block (S701), obtaining motion information of the current block according to the determined inter-frame prediction mode (S702), and performing motion compensation prediction on the current block based on the obtained motion information (S703).

[0182] In this regard, the inter prediction mode may represent various methods for determining motion information of the current block, and may include an inter prediction mode using translational motion information, an inter prediction mode using affine motion information. In an example, the inter prediction mode using translational motion information may include a merge mode and a motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. The motion information about the current block may be determined based on neighboring blocks adjacent to the current block or information obtained by parsing a bitstream.

[0183] Hereinafter, an inter prediction method using affine motion information is described in detail.

[0184] Figure 8 is a diagram illustrating nonlinear motion of an object.

[0185] In videos, nonlinear motion of objects may occur. In the example, Figure 8 In the example shown, nonlinear motion of the object may occur (such as camera zooming in, zooming out, rotating, or affine transformation, etc.). For nonlinear motion of the object, the translation motion vector may not be able to effectively represent the motion of the object. Therefore, for areas where nonlinear motion of the object occurs, the encoding efficiency can be improved by using affine motion instead of translation motion.

[0186] Fig. 9 is a flowchart of an inter-frame prediction method based on affine motion according to an embodiment of the present disclosure.

[0187] Whether the inter-frame prediction method based on affine motion is applied to the current block may be determined based on information parsed from the bitstream. Specifically, based on at least one of a flag indicating whether the affine merge mode is applied to the current block or a flag indicating whether the affine motion vector prediction mode is applied to the current block, it is determined whether the inter-frame prediction method based on affine motion is applied to the current block.

[0188] When an inter prediction method based on affine motion is applied to a current block, an affine motion model of the current block may be determined (S901). The affine motion model may be determined as at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model expresses affine motion by using 6 parameters, and the 4-parameter affine motion model expresses affine motion by using 4 parameters.

[0189] Equation 2 represents the affine motion by using 6 parameters. The affine motion represents the translational motion of a predetermined area determined by the affine seed vector.

[0190] [Equation 2]

[0191] v x =ax-by+e

[0192] v y =cx+dy+f

[0193] In the case of expressing affine motion by using 6 parameters, complex motion can be expressed, but since more bits are required to encode each parameter, encoding efficiency is reduced. Therefore, affine motion can be expressed by using 4 parameters. Equation 3 expresses affine motion by using 4 parameters.

[0194] [Equation 3]

[0195] v x =ax-by+e

[0196] v y =bx+ay+f

[0197] Information for determining the affine motion model of the current block may be encoded in the bitstream and signaled. In an example, the information may be a 1-bit flag 'affine_type_flag'. If the value of the flag is 0, it may indicate that a 4-parameter affine motion model is applied, and if the value of the flag is 1, it may indicate that a 6-parameter affine motion model is applied. The flag may be encoded in units of a slice, a tile, or a block (e.g., a coding block or a coding tree unit). When the flag is signaled at a slice level, the affine motion model determined at the slice level may be applied to all blocks belonging to the slice.

[0198] Optionally, based on the affine inter prediction mode of the current block, the affine motion model of the current block may be determined. In an example, when the affine merge mode is applied, the affine motion model of the current block may be determined as a 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information for determining the affine motion model of the current block may be encoded in the bitstream and signaled. In an example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block may be determined based on a 1-bit flag "affine_type_flag".

[0199] Next, an affine seed vector of the current block may be derived (S902). When a 4-parameter affine motion model is selected, a motion vector at two control points of the current block may be derived. On the other hand, when a 6-parameter affine motion model is selected, a motion vector at three control points of the current block may be derived. The motion vector at the control point may be referred to as an affine seed vector. The control point may include at least one of the upper left corner, the upper right corner, or the lower left corner of the current block.

[0200] Fig.10 is a diagram showing the affine seed vector of each affine motion model.

[0201] In a 4-parameter affine motion model, an affine seed vector may be derived for both the upper left corner, the upper right corner, or the lower left corner. Fig.10 In the example shown in (a), when the 4-parameter affine motion model is selected, the affine vector can be derived by using the affine seed vector sv0 of the upper left corner of the current block (e.g., the upper left sample point (x1, y1)) and the affine seed vector sv1 of the upper right corner of the current block (e.g., the upper right sample point (x1, y1)). The affine seed vector of the lower left corner can be used instead of the affine seed vector of the upper left corner, or the affine seed vector of the lower left corner can be used instead of the affine seed vector of the upper right corner.

[0202] In the 6-parameter affine motion model, affine seed vectors for the upper left corner, upper right corner, and lower left corner can be derived. In the example, as in Fig.10 In the example shown in (b), when the 6-parameter affine motion model is selected, the affine vector can be derived by using the affine seed vector sv0 of the upper left corner of the current block (e.g., the upper left sample point (x1, y1)), the affine seed vector sv1 of the upper right corner of the current block (e.g., the upper right sample point (x1, y1)), and the affine seed vector sv2 of the upper left corner of the current block (e.g., the upper left sample point (x2, y2)).

[0203] In the embodiment described later, under the 4-parameter affine motion model, the affine seed vectors at the upper left control point and the upper right control point are respectively referred to as the first affine seed vector and the second affine seed vector. In the embodiment described later using the first affine seed vector and the second affine seed vector, at least one of the first affine seed vector and the second affine seed vector may be replaced by the affine seed vector at the lower left control point (the third affine seed vector) or the affine seed vector at the lower right control point (the fourth affine seed vector).

[0204] In addition, under the 6-parameter affine motion model, the affine seed vectors at the upper left control point, the upper right control point, and the lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiment using the first affine seed vector, the second affine seed vector, and the third affine seed vector described later, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector may be replaced by the affine seed vector at the lower right control point (the fourth affine seed vector).

[0205] The affine vector of each sub-block may be derived by using the affine seed vector (S903). In this regard, the affine vector represents a translation motion vector derived based on the affine seed vector. The affine vector of the sub-block may be referred to as an affine sub-block motion vector or a sub-block motion vector.

[0206] Fig.11 is a diagram showing affine vectors of sub-blocks under a 4-parameter motion model.

[0207] The affine vector of the sub-block may be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. In an example, Equation 4 represents an example of deriving an affine sub-block motion vector.

[0208] [Equation 4]

[0209]

[0210] In Equation 4, (x, y) represents the position of the sub-block. In this regard, the position of the sub-block represents the position of a basic sample point included in the sub-block. The basic sample point may be a sample point located at the upper left corner of the sub-block or a sample point at the center position of at least one of the x-axis or y-axis coordinates. (x0, y0) represents the position of the first control point, and (sv 0x ,sv 0y ) represents the first affine seed vector. In addition, (x1, y1) represents the position of the second control point, and (sv 1x ,sv 1y ) represents the second affine seed vector.

[0211] When the first control point and the second control point correspond to the upper left corner and the upper right corner of the current block, respectively, x1-x0 may be set to the same value as the width of the current block.

[0212] Afterwards, motion compensation prediction of each sub-block may be performed by using the affine vector of each sub-block (S904). As a result of performing the motion compensation prediction, a prediction block of each sub-block may be generated. The prediction block of the sub-block may be set as the prediction block of the current block.

[0213] The affine seed vector of the current block may be derived based on the affine seed vector of the neighboring block adjacent to the current block. When the inter prediction mode of the current block is the affine merge mode, the affine seed vector of the merge candidate included in the merge candidate list may be determined as the affine seed vector of the current block. In addition, when the inter prediction mode of the current block is the affine merge mode, motion information including at least one of a reference picture index, a specific direction prediction flag, or a bidirectional weight of the current block may also be set to be the same as the merge candidate.

[0214] Next, an inter prediction method using translational motion information is described in detail.

[0215] The motion information of the current block may be derived from the motion information of another block. In this regard, the other block may be a block that is encoded / decoded by inter-frame prediction before the current block. Setting the motion information of the current block to be the same as the motion information of the other block may be defined as a merge mode. In addition, setting the motion vector of the other block to the predicted value of the motion vector of the current block may be defined as a motion vector prediction mode.

[0216] Fig.12 is a flow chart of a process for deriving motion information for a current block in merge mode.

[0217] A merge candidate of a current block may be derived (S1201). The merge candidate of the current block may be derived from a block encoded / decoded by inter prediction before the current block.

[0218] Fig.13 is a diagram illustrating candidate blocks for deriving merge candidates.

[0219] The candidate block may include at least one of a neighboring block or a non-neighboring block, wherein the neighboring block includes samples adjacent to the current block and the non-neighboring block includes samples not adjacent to the current block. Hereinafter, samples for determining the candidate block are defined as basic samples. In addition, basic samples adjacent to the current block are referred to as neighboring basic samples, and basic samples not adjacent to the current block are referred to as non-neighboring basic samples.

[0220] The neighboring basic sample points may be included in the neighboring columns of the leftmost column of the current block or the neighboring rows of the uppermost row of the current block. In an example, when the coordinates of the upper left sample point of the current block are (0,0), at least one block among the blocks including the basic sample points at positions (-1,H-1), (W-1,-1), (W,-1), (-1,H) or (-1,1) may be used as a candidate block. Referring to the diagram, neighboring blocks of index 0 to index 4 may be used as candidate blocks.

[0221] The non-neighboring basic sample points represent samples having at least one of an x-axis distance or a y-axis distance with respect to the basic sample points adjacent to the current block having a predefined value. In an example, at least one of a block including a basic sample point having an x-axis distance with respect to the left basic sample point having a predefined value, a block including a non-neighboring sample point having a y-axis distance with respect to the upper basic sample point having a predefined value, or a block including a non-neighboring sample point having an x-axis distance and a y-axis distance with respect to the upper left basic sample point having predefined values ​​may be used as a candidate block. The predefined value may be a natural number such as 4, 8, 12, 16, etc. Referring to the diagram, at least one block of index 5 to index 26 may be used as a candidate block.

[0222] Sample points that are not located on the same vertical line, horizontal line, or diagonal line as adjacent basic sample points may be set as non-adjacent basic sample points.

[0223] The motion information of the merge candidate may be set to be the same as the motion information of the candidate block. In an example, at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index of the candidate block may be set to the motion information of the merge candidate.

[0224] A merge candidate list including merge candidates may be generated ( S1202 ).

[0225] The indexes of the merge candidates in the merge candidate list may be assigned according to a predetermined order. In an example, the indexes may be assigned in the order of: merge candidates derived from the left neighboring block, merge candidates derived from the upper neighboring block, merge candidates derived from the upper right neighboring block, merge candidates derived from the lower left neighboring block, merge candidates derived from the upper left neighboring block, and merge candidates derived from the temporal neighboring block.

[0226] When multiple merge candidates are included in the merge candidate list, at least one of the multiple merge candidates may be selected (S1203). Specifically, information for specifying any one of the multiple merge candidates may be signaled in the bitstream. In the example, information merge_idx indicating an index of any one of the merge candidates included in the merge candidate list may be signaled in the bitstream.

[0227] When the number of merge candidates included in the merge candidate list is less than the threshold value, the motion information candidate included in the motion information table may be added to the merge candidate list as a merge candidate. In this regard, the threshold value may be the maximum number of merge candidates that may be included in the merge candidate list or a value minus an offset from the maximum number of merge candidates. The offset may be a natural number (such as 1 or 2, etc.).

[0228] The motion information table includes motion information candidates derived from a block encoded / decoded based on inter-frame prediction in a current picture. In an example, motion information of the motion information candidate included in the motion information table may be set to be the same as motion information of the block encoded / decoded based on inter-frame prediction. In this regard, the motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index.

[0229] The motion information candidates included in the motion information table may also be referred to as inter region merging candidates or prediction region merging candidates.

[0230] The maximum number of motion information candidates that can be included in the motion information table can be predefined in the encoder and the decoder. In an example, the maximum number of motion information candidates that can be included in the motion information table can be 1, 2, 3, 4, 5, 6, 7, 8 or more (e.g., 16).

[0231] Optionally, information indicating a maximum number of motion information candidates that can be included in the motion information table may be signaled in the bitstream. The information may be signaled at a sequence, picture, or slice level. The information may indicate the maximum number of motion information candidates that can be included in the motion information table. Optionally, the information may indicate a difference between the maximum number of motion information candidates that can be included in the motion information table and the maximum number of merge candidates that can be included in the merge candidate list.

[0232] Alternatively, the maximum number of motion information candidates that can be included in the motion information table may be determined according to a picture size, a slice size, or a coding tree unit size.

[0233] The motion information table may be initialized in units of pictures, slices, tiles, partitions, coding tree units, or coding tree unit lines (rows or columns). In an example, when a slice is initialized, the motion information table is also initialized, so the motion information table may not include any motion information candidates.

[0234] Optionally, information indicating whether the motion information table is to be initialized may be signaled in the bitstream. The information may be signaled at a slice, tile, partition or block level. Before the information indicates initialization of the motion information table, a pre-configured motion information table may be used.

[0235] Alternatively, information about the initial motion information candidate may be signaled in a picture parameter set or a slice header. Although the slice is initialized, the motion information table may include the initial motion information candidate. Therefore, the initial motion information candidate may be used for a block that is the first encoding / decoding target in the slice.

[0236] Optionally, the motion information candidate included in the motion information table of the previous coding tree unit may be set as the initial motion information candidate. In an example, the motion information candidate having the smallest index or the largest index among the motion information candidates included in the motion information table of the previous coding tree unit may be set as the initial motion information candidate.

[0237] The blocks are encoded / decoded in the order of encoding / decoding, and the blocks encoded / decoded based on the inter prediction may be sequentially set as motion information candidates in the order of encoding / decoding.

[0238] Fig.14 is a diagram for explaining the updating aspect of the motion information table.

[0239] For the current block, when inter prediction is performed (S1401), a motion information candidate may be derived based on the current block (S1402). The motion information of the motion information candidate may be set to be the same as the motion information of the current block.

[0240] When the motion information table is empty ( S1403 ), the motion information candidate derived based on the current block may be added to the motion information table ( S1404 ).

[0241] When the motion information table already includes motion information candidates (S1403), a redundancy check of the motion information of the current block (or a motion information candidate derived therefrom) may be performed (S1405). The redundancy check is used to determine whether the motion information of the pre-stored motion information candidate in the motion information table is the same as the motion information of the current block. The redundancy check may be performed on all pre-stored motion information candidates in the motion information table. Optionally, the redundancy check may be performed on motion information candidates having an index exceeding or below a threshold among the pre-stored motion information candidates in the motion information table. Optionally, the redundancy check may be performed for a predefined number of motion information candidates. In an example, two motion information candidates having the smallest index or having the largest index may be determined as targets of the redundancy check.

[0242] When a motion information candidate having the same motion information as the current block is not included, a motion information candidate derived based on the current block may be added to the motion information table (S1408). Whether the motion information candidates are the same may be determined based on whether the motion information (e.g., motion vector / reference picture index, etc.) of the motion information candidates are the same.

[0243] In this regard, when the maximum number of motion information candidates is already stored in the motion information table (S1406), the earliest motion information candidate may be deleted (S1407), and the motion information candidate derived based on the current block may be added to the motion information table (S1408). In this regard, the earliest motion information candidate may be a motion information candidate having a maximum or minimum index.

[0244] The motion information candidates may be identified by their respective indexes. When the motion information candidate derived from the current block is added to the motion information table, the minimum index (e.g., 0) may be assigned to the motion information candidate, and the index of the pre-stored motion information candidate may be increased by 1. In this regard, when the maximum number of motion information candidates is already stored in the motion information table, the motion information candidate with the largest index is removed.

[0245] Optionally, when a motion information candidate derived from the current block is added to the motion information table, the maximum index may be assigned to the motion information candidate. In an example, when the number of pre-stored motion information candidates in the motion information table is less than the maximum value, an index having the same value as the number of pre-stored motion information candidates may be assigned to the motion information candidate. Optionally, when the number of pre-stored motion information candidates in the motion information table is equal to the maximum value, an index obtained by subtracting 1 from the maximum value may be assigned to the motion information candidate. Optionally, the motion information candidate having the smallest index is removed, and the index of the residual pre-stored motion information candidate is reduced by 1.

[0246] Fig.15 is a diagram illustrating an updating aspect of a motion information table.

[0247] It is assumed that when a motion information candidate derived from the current block is added to the motion information table, a maximum index is assigned to the motion information candidate. In addition, it is assumed that the maximum number of motion information candidates has been stored in the motion information table.

[0248] When the motion information candidate HmvpCand[n+1] derived from the current block is added to the motion information table HmvpCandList, the motion information candidate HmvpCand[0] having the smallest index among the pre-stored motion information candidates may be deleted, and the index of the residual motion information candidate may be reduced by 1. In addition, the index of the motion information candidate HmvpCand[n+1] derived from the current block may be set to the maximum value (for Fig.15 The example shown in , n).

[0249] When the same motion information candidate as the motion information candidate derived based on the current block is pre-stored ( S1405 ), the motion information candidate derived based on the current block may not be added to the motion information table ( S1409 ).

[0250] Optionally, when the motion information candidate derived based on the current block is added to the motion information table, the pre-stored motion information candidate identical to the motion information candidate may be removed. In this case, it produces the same effect as when the index of the pre-stored motion information candidate is newly updated.

[0251] Fig.16 is a diagram showing an example in which indexes of pre-stored motion information candidates are updated.

[0252] When the index of the pre-stored motion information candidate that is the same as the motion information candidate mvCand derived from the current block is hIdx, the pre-stored motion information candidate may be removed, and the index of the motion information candidate having an index greater than hIdx may be reduced by 1. In an example, Fig.16 The example shown in shows that HmvpCand[2] identical to mvCand is deleted in the motion information table HvmpCandList, and the indexes from HmvpCand[3] to HmvpCand[n] are decreased by 1.

[0253] And, the motion information candidate mvCand derived based on the current block may be added to the end of the motion information table.

[0254] Alternatively, an index assigned to the same pre-stored motion information candidate as the motion information candidate derived based on the current block may be updated. For example, the index of the pre-stored motion information candidate may be changed to a minimum value or a maximum value.

[0255] The motion information of the blocks included in the predetermined area may be set not to be added to the motion information table. In an example, the motion information candidates derived based on the motion information of the blocks included in the merge processing area may not be added to the motion information table. Since the encoding / decoding order of the blocks included in the merge processing area is not defined, it is inappropriate to use the motion information of any one of them for inter-frame prediction of another one of them. Therefore, the motion information candidates derived based on the blocks included in the merge processing area may not be added to the motion information table.

[0256] Optionally, the motion information of a block smaller than a preset size may be set not to be added to the motion information table. In an example, the motion information candidate derived based on the motion information of a coding block with a width or height less than 4 or 8 or the motion information of a coding block of 4×4 size may not be added to the motion information table.

[0257] Based on the inter prediction mode of the current block, it may be determined whether the current block will be used as a motion information candidate. In an example, a block encoded / decoded based on an affine motion model may be set to be unusable as a motion information candidate. Therefore, although the current block is encoded / decoded by inter prediction, when the inter prediction mode of the current block is an affine prediction mode, the motion information table may not be updated based on the current block.

[0258] The motion information candidate may be set to include additional information in addition to the motion information. In an example, at least one of the size, shape, or partition information of the block may be additionally stored in the motion information candidate. When configuring the merge candidate list of the current block, only the motion information candidates whose size, shape, or partition information is the same as or similar to that of the current block among the motion information candidates may be used, or the motion information candidates whose size, shape, or partition information is the same as or similar to that of the current block may be pre-added to the merge candidate list.

[0259] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, the motion information candidate included in the motion information table may be added to the merge candidate list as a merge candidate. The additional processing is performed in an order reflecting the sorting order of the indexes of the motion information candidates in ascending or descending order. In an example, the motion information candidate with the largest index may be first added to the merge candidate list of the current block.

[0260] When a motion information candidate included in the motion information table is added to the merge candidate list, a redundancy check between the motion information candidate and the pre-stored merge candidates in the merge candidate list may be performed. As a result of the redundancy check, a motion information candidate having the same motion information as the pre-stored merge candidate may not be added to the merge candidate list.

[0261] The redundancy check may be performed only on a portion of the motion information candidates included in the motion information table. In an example, the redundancy check may be performed only on motion information candidates having an index equal to or greater than a threshold or having an index equal to or less than a threshold. Alternatively, the redundancy check may be performed only on N motion information candidates having the largest index or the smallest index.

[0262] Optionally, redundancy check may be performed only on a portion of the pre-stored merge candidates in the merge candidate list. In an example, redundancy check may be performed only on merge candidates whose index is equal to or greater than a threshold or equal to or less than a threshold or merge candidates derived from a block at a specific position. In this regard, the specific position may include at least one of a left neighboring block, an upper neighboring block, an upper right neighboring block, or a lower left neighboring block of the current block.

[0263] Fig.17 is a diagram illustrating an example of performing redundancy check on only a part of merge candidates.

[0264] When the motion information candidate HmvpCand[j] is added to the merge candidate list, a redundancy check with two merge candidates mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1] having the largest index may be performed for the motion information candidate. In this regard, NumMerge may show the number of available spatial merge candidates and temporal merge candidates.

[0265] Unlike the example shown, when the motion information candidate HmvpCand[j] is added to the merge candidate list, a redundancy check with two merge candidates with the smallest index may be performed for the motion information candidate. For example, it may be checked whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j].

[0266] Optionally, a redundancy check may be performed only on merge candidates derived from a specific position. In an example, a redundancy check may be performed on at least one of merge candidates derived from a neighboring block located to the left of the current block or above the current block. When there is no merge candidate derived from a specific position in the merge candidate list, the motion information candidate may be added to the merge candidate list without a redundancy check.

[0267] When the motion information candidate HmvpCand[j] is added to the merge candidate list, a redundancy check with two merge candidates mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1] having the largest index may be performed for the motion information candidate. In this regard, NumMerge may show the number of available spatial merge candidates and temporal merge candidates.

[0268] A redundancy check with merge candidates may be performed only on a portion of the motion information candidates. In an example, a redundancy check may be performed only on N motion information candidates having a large index or a small index among the motion information candidates included in the motion information table. In an example, a redundancy check may be performed only on motion information candidates having an index whose difference between the number of motion information candidates and the index is lower than a threshold among the motion information candidates included in the motion information table. When the threshold is 2, a redundancy check may be performed only on 3 motion information candidates having the largest index value among the motion information candidates included in the motion information table. For motion information candidates other than the above 3 motion information candidates, the redundancy check may be omitted. When the redundancy check is omitted, the motion information candidate may be added to the merge candidate list regardless of whether there is the same motion information as the merge candidate.

[0269] In contrast, the redundancy check is set to be performed only on the motion information candidates having the index whose difference between the number of motion information candidates and the index exceeds the threshold value among the motion information candidates included in the motion information table.

[0270] The number of motion information candidates for performing redundancy check may be redefined in the encoder and the decoder. In an example, the threshold may be an integer (such as 0, 1, or 2).

[0271] Alternatively, the threshold may be determined based on at least one of the number of merge candidates included in the merge candidate list or the number of motion information candidates included in the motion information table.

[0272] When the merge candidate identical to the first motion information candidate is found, redundant checking of the merge candidate identical to the first motion information candidate may be omitted in redundant checking of the second motion information candidate.

[0273] Fig.18 is a diagram illustrating an example in which a redundancy check with a specific merge candidate is omitted.

[0274] When the motion information candidate HmvpCand[i] with index i is added to the merge candidate list, a redundant check is performed between the motion information candidate and the pre-stored merge candidates in the merge candidate list. In this regard, when a merge candidate mergeCandlist[j] identical to the motion information candidate HmvpCand[i] is found, a redundant check can be performed between the motion information candidate HmvpCand[i-1] with index i-1 and the merge candidate without adding the motion information candidate HmvpCand[i] to the merge candidate list. In this regard, the redundant check between the motion information candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0275] In the example, Fig.18 In the example shown, it is determined that HmvpCand[1] and mergeCandList[2] are the same. Therefore, a redundancy check of HmvpCand[i-1] may be performed without HmvpCand[i] being added to the merge candidate list. In this regard, a redundancy check between HmvpCand[i-1] and mergeCandList[2] may be omitted.

[0276] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, at least one of a paired merge candidate or a zero merge candidate may be further included in addition to the motion information candidate. The paired merge candidate indicates a merge candidate having a value obtained by averaging motion vectors of more than 2 merge candidates as a motion vector, and the zero merge candidate indicates a merge candidate having a motion vector of 0.

[0277] For the merge candidate list of the current block, merge candidates may be added in the following order.

[0278] Spatial merging candidate - Temporal merging candidate - Motion information candidate - (affine motion information candidate) - Paired merging candidate - Zero merging candidate

[0279] The spatial merge candidate refers to a merge candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal merge candidate refers to a merge candidate derived from a previous reference picture. The affine motion information candidate refers to a motion information candidate derived from a block encoded / decoded by an affine motion model.

[0280] The motion information table may be used in a motion vector prediction mode. In an example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold value, the motion information candidate included in the motion information table may be set as a motion vector prediction candidate for the current block. Specifically, the motion vector of the motion information candidate may be set as a motion vector prediction candidate.

[0281] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate may be set as a motion vector predictor of the current block. Then, after the motion vector residual value of the current block is decoded, the motion vector of the current block may be obtained by adding the motion vector predictor and the motion vector residual value.

[0282] The motion vector prediction candidate list of the current block may be configured in the following order.

[0283] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Motion information candidate - (affine motion information candidate) - Zero motion vector prediction candidate

[0284] A spatial motion vector prediction candidate represents a motion vector prediction candidate derived from at least one of a neighboring block or a non-neighboring block, and a temporal motion vector prediction candidate represents a motion vector prediction candidate derived from a previous reference picture. An affine motion information candidate represents a motion information candidate derived from a block encoded / decoded by an affine motion model. A zero motion vector prediction candidate represents a candidate whose motion vector value is 0.

[0285] A merge processing area larger than a coding block can be defined. The coding blocks included in the merge processing area can be processed in parallel without being encoded / decoded sequentially. In this regard, not being encoded / decoded sequentially means that the order of encoding / decoding is not defined. Therefore, the encoding / decoding processing of the blocks included in the merge processing area can be processed independently. Optionally, the blocks included in the merge processing area can share merge candidates. In this regard, merge candidates can be derived based on the merge processing area.

[0286] According to the above features, the merged processing region may be referred to as a parallel processing region, a shared merge region (SMR), or a merged estimation region (MER).

[0287] A merge candidate of the current block may be derived based on the coding block. However, when the current block is included in a merge processing region larger than the current block, a candidate block included in the same merge processing region as the current block may be set to be unavailable as a merge candidate.

[0288] Fig.19 is a diagram illustrating an example in which a candidate block included in the same merge processing region as a current block is set to be unusable as a merge candidate.

[0289] exist Fig.19 In the example shown on the left side of , in the encoding / decoding of CU5, a block including basic samples adjacent to CU5 may be set as a candidate block. In this regard, candidate blocks X3 and X4 included in the same merge processing region as CU5 may be set as unusable as merge candidates for CU5. However, candidate blocks X0, X1, and X2 not included in the same merge processing region as CU5 may be set as usable as merge candidates.

[0290] exist Fig.19 In the example shown on the right side of , in the encoding / decoding of CU8, a block including basic samples adjacent to CU8 may be set as a candidate block. In this regard, candidate blocks X6, X7, and X8 included in the same merge processing region as CU8 may be set to be unusable as merge candidates. However, candidate blocks X5 and X9 not included in the same merge processing region as CU8 may be set to be usable as merge candidates.

[0291] Optionally, when the current block is included in the merge processing region, neighboring blocks adjacent to the current block and the merge processing region may be set as candidate blocks.

[0292] Fig. 20 is a diagram illustrating an example of deriving a merge candidate of a current block when the current block is included in a merge processing region.

[0293] As in Fig. 20In the example shown in (a), a neighboring block adjacent to the current block may be set as a candidate block for deriving a merge candidate of the current block. In this regard, a candidate block included in the same merge processing area as the current block may be set to be unusable as a merge candidate. In the example, when deriving a merge candidate for coding block CU3, an upper neighboring block y3 and an upper right neighboring block y4 included in the same merge processing area as coding block CU3 may be set to be unusable as a merge candidate for coding block CU3.

[0294] By scanning neighboring blocks adjacent to the current block in a predefined order, a merge candidate may be derived. In an example, the predefined order may be the order of y1, y3, y4, y0, and y2.

[0295] like Fig. 20 In the example shown in (b), when the number of merge candidates that can be derived from the neighboring blocks adjacent to the current block is less than the value or the maximum number of merge candidates minus the offset, the merge candidate of the current block can be derived by using the neighboring blocks adjacent to the merge processing area. In the example, the neighboring blocks adjacent to the merge processing area including the coding block CU3 can be set as candidate blocks of the coding block CU3. In this regard, the neighboring blocks adjacent to the merge processing area may include at least one of the left neighboring block x1, the upper neighboring block x3, the lower left neighboring block x0, the upper right neighboring block x4, or the upper left neighboring block x2.

[0296] By scanning neighboring blocks adjacent to the merge processing area in a predefined order, a merge candidate may be derived. In an example, the predefined order may be the order of x1, x3, x4, x0, and x2.

[0297] In summary, a merge candidate on the coding block CU3 included in the merge processing area can be derived by scanning the candidate blocks in the following scanning order.

[0298] (y1,y3,y4,y0,y2,x1,x3,x4,x0,x2)

[0299] However, the above scanning order of candidate blocks only shows an example of the present disclosure, and the candidate blocks may be scanned in an order different from the above example. Optionally, the scanning order may be adaptively determined based on at least one of the size or shape of the current block or the merge processing area.

[0300] The merged processing area may be square or non-square. Information for determining the merged processing area may be signaled in the bitstream. The information may include at least one of information indicating the shape of the merged processing area or information indicating the size of the merged processing area. When the merged processing area is non-square, at least one of information indicating the size of the merged processing area, information indicating the width or height of the merged processing area, or information indicating the ratio between the width and height of the merged processing area may be signaled in the bitstream.

[0301] The size of the merge processing area may be determined based on at least one of information signaled in a bitstream, a picture resolution, a size of a slice, or a size of a tile.

[0302] If motion compensation prediction is performed on a block included in the merge processing area, motion information candidates derived based on motion information of the block on which motion compensation prediction is performed may be added to the motion information table.

[0303] However, if the motion information candidate derived from the block included in the merge processing area is added to the motion information table, a situation may occur where the motion information candidate derived from the block is used for encoding / decoding of other blocks in the merge processing area whose encoding / decoding is actually slower than that of the block. In other words, although the dependency between blocks should be excluded in encoding / decoding of the block included in the merge processing area, a situation may occur where motion prediction compensation is performed by using motion information of other blocks included in the merge processing area. In order to solve such a problem, although encoding / decoding of the block included in the merge processing area is completed, the motion information of the encoding / decoding completed block may not be added to the motion information table.

[0304] Alternatively, the motion information table may be updated by using only blocks at predefined positions in the merge processing area. The predefined position may include at least one of a block at an upper left position, a block at an upper right position, a block at a lower left position, a block at a lower right position, a block at a center position, a block adjacent to a right boundary, or a block adjacent to a lower boundary in the merge processing area. In an example, only the motion information of a block adjacent to the lower right corner in the merge processing area may be updated in the motion information table, and the motion information of other blocks may not be updated in the motion information table.

[0305] Alternatively, after all blocks included in the merge processing area are decoded, motion information candidates derived from the blocks may be added to the motion information table. In other words, although the blocks included in the merge processing area are encoded / decoded, the motion information table may not be updated.

[0306] In an example, if motion compensation prediction is performed on a block included in a merge processing area, the motion information candidate derived from the block may be added to the motion information table in a predefined order. In this regard, the predefined order may be determined according to a scanning order of coding blocks in a merge processing area or a coding tree unit. The scanning order may be at least one of a raster scan, a horizontal scan, a vertical scan, or a zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0307] Alternatively, a motion information candidate including unidirectional motion information may be added to the motion information table before a motion information candidate including bidirectional motion information. Conversely, a motion information candidate including bidirectional motion information may be added to the motion information table before a motion information candidate including unidirectional motion information.

[0308] Alternatively, the motion information candidates may be added to the motion information table in the order of high usage frequency or low usage frequency in the merge processing region or the coding tree unit.

[0309] When the current block is included in the merge processing region and the number of merge candidates included in the merge candidate list of the current block is less than the maximum number, the motion information candidate included in the motion information table may be added to the merge candidate list. In this regard, the motion information candidate derived from the block included in the same merge processing region as the current block may be set not to be added to the merge candidate list of the current block.

[0310] Optionally, when the current block is included in the merge processing region, it may be set not to use the motion information candidates included in the motion information table. In other words, although the number of merge candidates included in the merge candidate list of the current block is less than the maximum number, the motion information candidates included in the motion information table may not be added to the merge candidate list.

[0311] In another example, a motion information table for a merge processing region or a coding tree unit may be configured. The motion information table serves to temporarily store motion information of blocks included in the merge processing region. In order to distinguish a general motion information table from a motion information table for a merge processing region or a coding tree unit, the motion information table for the merge processing region or the coding tree unit is referred to as a temporary motion information table. And, the motion information candidates stored in the temporary motion information table are referred to as temporary motion information candidates.

[0312] Fig.21 is a diagram showing a temporary motion information table.

[0313] A temporary motion information table for a coding tree unit or a merge processing area may be configured. When motion compensation prediction is performed on a current block included in the coding tree unit or the merge processing area, the motion information of the block may not be added to the motion information table HmvpCandList. Instead, a temporary motion information candidate derived from the block may be added to the temporary motion information table HmvpMERCandList. In other words, the temporary motion information candidate added to the temporary motion information table may not be added to the motion information table. Therefore, the motion information table may not include a motion information candidate derived based on the motion information of the block included in the coding tree unit or the merge processing area, wherein the coding tree unit or the merge processing area includes the current block.

[0314] Optionally, only motion information of some blocks included in the merge processing area may be added to the temporary motion information table. In an example, only blocks at predefined positions in the merge processing area may be used to update the motion information table. The predefined position may include at least one of a block at an upper left position, a block at an upper right position, a block at a lower left position, a block at a lower right position, a block at a center position, a block adjacent to a right boundary, or a block adjacent to a lower boundary in the merge processing area. In an example, only motion information of a block adjacent to the lower right corner in the merge processing area may be added to the temporary motion information table, and motion information of other blocks may not be added to the temporary motion information table.

[0315] The maximum number of temporary motion information candidates that can be included in the temporary motion information table may be set to be the same as the maximum number of motion information candidates. Alternatively, the maximum number of temporary motion information candidates that can be included in the temporary motion information table may be determined according to the size of the coding tree unit or the merge processing region. Alternatively, the maximum number of temporary motion information candidates that can be included in the temporary motion information table may be set to be smaller than the maximum number of motion information candidates that can be included in the motion information table.

[0316] The current block included in the coding tree unit or the merge processing area may be set not to use the temporary motion information table about the corresponding coding tree unit or the merge processing area. In other words, when the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, the motion information candidates included in the motion information table may be added to the merge candidate list, and the temporary motion information candidates included in the temporary motion information table may not be added to the merge candidate list. Therefore, the motion information of other blocks included in the same coding tree unit or the same merge processing area as the current block may not be used for motion compensation prediction of the current block.

[0317] If encoding / decoding of all blocks included in the coding tree unit or the merge processing area is completed, the motion information table and the temporary motion information table may be unified.

[0318] Fig. 22 is a diagram showing an example in which a motion information table and a temporary motion information table are unified.

[0319] As in Fig. 22 In the example shown in , if encoding / decoding of all blocks included in a coding tree unit or a merge processing region is completed, the temporary motion information candidates included in the temporary motion information table may be updated in the motion information table.

[0320] In this regard, the temporary motion information candidates included in the temporary motion information table may be added to the motion information table in the order of insertion into the temporary motion information table (in other words, in ascending or descending order of index values).

[0321] In another example, the temporary motion information candidates included in the temporary motion information table may be added to the motion information table in a predefined order. In this regard, the predefined order may be determined according to a scanning order of coding blocks in a merge processing area or a coding tree unit. The scanning order may be at least one of a raster scan, a horizontal scan, a vertical scan, or a zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0322] Alternatively, a temporary motion information candidate including unidirectional motion information may be added to the motion information table before a temporary motion information candidate including bidirectional motion information. Conversely, a temporary motion information candidate including bidirectional motion information may be added to the motion information table before a temporary motion information candidate including unidirectional motion information.

[0323] Alternatively, the temporary motion information candidates may be added to the motion information table in the order of high usage frequency or low usage frequency in the merge processing region or the coding tree unit.

[0324] In the case where a temporary motion information candidate included in the temporary motion information table is added to the motion information table, a redundancy check on the temporary motion information candidate may be performed. In an example, when a motion information candidate identical to the temporary motion information candidate included in the temporary motion information table is pre-stored in the motion information table, the temporary motion information candidate may not be added to the motion information table. In this regard, a redundancy check may be performed on a portion of the motion information candidates included in the motion information table. In an example, a redundancy check may be performed on a motion information candidate having an index exceeding a threshold value or below a threshold value. In an example, when the temporary motion information candidate is equal to a motion information candidate having an index exceeding a predefined value, the temporary motion information candidate may not be added to the motion information table.

[0325] The use of motion information candidates derived from blocks included in the same coding tree unit or the same merge processing area as the current block as the merge candidate of the current block may be limited. To this end, address information of the block may be additionally stored for the motion information candidate. The address information of the block may include at least one of the position of the block, the address of the block, the index of the block, the position of the merge processing area including the block, the address of the merge processing area including the block, the index of the merge processing area including the block, the position of the coding tree area including the block, the address of the coding tree area including the block, or the index of the coding tree area including the block.

[0326] The coding block may be partitioned into a plurality of prediction units, and prediction may be performed for each of the partitioned prediction units. In this case, the prediction unit refers to a basic unit for performing prediction.

[0327] The coding block may be partitioned by using at least one of a vertical line, a horizontal line, a slanted line, or a diagonal line. The prediction unit partitioned by the partition line may have a shape such as a triangle, a quadrilateral, a trapezoid, or a pentagon. In an example, the coding block may be partitioned into two triangular prediction units, two trapezoidal prediction units, two quadrilateral prediction units, or one triangular prediction unit and one pentagonal prediction unit.

[0328] Information for determining at least one of the number, angle, or position of lines for partitioning the coding block may be signaled in the bitstream. In an example, information indicating one of the partition type candidates of the coding block may be signaled in the bitstream, or information specifying one of a plurality of line candidates for partitioning the coding block may be signaled in the bitstream. In an example, index information indicating one of the plurality of line candidates may be signaled in the bitstream.

[0329] For each of the plurality of line candidates, at least one of the angle or position may be different. The number of line candidates available for the current block may be determined based on the size or shape of the current block, the number of available merge candidates, or whether a neighboring block at a specific position is available as a merge candidate.

[0330] Optionally, information for determining the number or type of line candidates may be signaled in the bitstream. In an example, whether a slanted line with an angle greater than a diagonal line and / or a slanted line with an angle less than a diagonal line may be used as a line candidate may be determined by using a 1-bit flag. The information may be signaled at a sequence, picture, or slice level.

[0331] Optionally, at least one of the number, angle or position of lines for partitioning the coding block may be adaptively determined based on at least one of the intra prediction mode or inter prediction mode of the coding block, the position of available merge candidates or the partition type of neighboring blocks.

[0332] When a coding block is partitioned into a plurality of prediction units, intra prediction or inter prediction may be performed on the prediction unit of each partition.

[0333] Fig.23 is a diagram illustrating an example of partitioning a coding block into a plurality of prediction units by using diagonal lines.

[0334] As in Fig.23 (a) and Fig.23 In the example shown in (b), the coding block can be partitioned into two triangular prediction units by using diagonal lines.

[0335] Fig.23 (a) and Fig.23 (b) shows that the coding block is partitioned into two prediction units by using a diagonal line connecting two vertices of the coding block. However, the coding block may be partitioned into two prediction units by using an oblique line in which at least one end of the line does not pass through a vertex of the coding block.

[0336] Fig.24 is a diagram illustrating an example in which a coding block is partitioned into two prediction units.

[0337] As in Fig.24 (a) and Fig.24 In the example shown in (b) of FIG. 5 , the coding block may be partitioned into two prediction units by using a slanted line whose ends are adjacent to the upper boundary and the lower boundary of the coding block respectively.

[0338] Optionally, as in Fig.24 (c) and Fig.24 In the example shown in (d) of FIG. 1 , the coding block may be partitioned into two prediction units by using a slanted line whose ends are adjacent to the left boundary and the right boundary of the coding block, respectively.

[0339] Optionally, the coding block may be partitioned into two prediction units with different sizes. In an example, the coding block may be partitioned into two prediction units with different sizes by setting a slash for partitioning the coding block to satisfy two boundaries forming a vertex.

[0340] Fig.25 An example of partitioning a coding block into a plurality of prediction blocks of different sizes is shown.

[0341] As in Fig.25 (a) and Fig.25 In the example shown in (b), the coding block can be partitioned into two prediction units with different sizes by setting the diagonal line connecting the upper left and lower right of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block instead of the upper left corner or lower right corner.

[0342] Optionally, as in Fig.25(c) and Fig.25 In the example shown in (d), the coding block can be partitioned into two prediction units with different sizes by setting the diagonal line connecting the upper right and lower left of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block instead of the upper left corner or lower right corner.

[0343] Each prediction unit generated by partitioning the coding block is referred to as an "Nth prediction unit". Figure 23 to Figure 25 In the example shown, PU1 may be defined as a first prediction unit, and PU2 may be defined as a second prediction unit. The first prediction unit may represent a prediction unit including a sample at a lower left position or a sample at an upper left position in the coding block, and the second prediction unit may represent a prediction unit including a sample at an upper right position or a sample at a lower right position in the coding block.

[0344] Conversely, a prediction unit including a sample at an upper right position or a sample at a lower right position in the coding block may be defined as a first prediction unit, and a prediction unit including a sample at a lower left position or a sample at an upper left position in the coding block may be defined as a second prediction unit.

[0345] When a coding block is partitioned by using horizontal, vertical, diagonal or oblique lines, it may be referred to as a prediction unit partition. A prediction unit generated by applying the prediction unit partition may be referred to as a triangular prediction unit, a quadrilateral prediction unit or a pentagonal prediction unit according to its shape.

[0346] In the following embodiments, it will be assumed that the coding block is partitioned by using a diagonal line. Specifically, when the coding block is partitioned into two prediction units by using a diagonal line, it is called diagonal partitioning or triangular partitioning. However, even when the coding block is partitioned by using an oblique line having an angle different from a vertical line, a horizontal line, or a diagonal line, the prediction unit can be encoded / decoded according to the following embodiments. In other words, matters related to the encoding / decoding of the triangular prediction unit described below can also be applied to the encoding / decoding of the quadrilateral prediction unit or the pentagonal prediction unit.

[0347] Whether prediction unit partitioning is applied to a coding block may be determined based on at least one of a slice type, a maximum number of merge candidates that may be included in a merge candidate list, a size of a coding block, a shape of a coding block, a prediction coding mode of the coding block, or a partitioning aspect of a parent node.

[0348] In an example, whether to apply prediction unit partitioning to a coding block may be determined based on whether the current slice is of type B. Prediction unit partitioning may be allowed only when the current slice is of type B.

[0349] Alternatively, whether to apply prediction unit partitioning to the coding block may be determined based on whether the maximum number of merge candidates included in the merge candidate list is equal to or greater than 2. Prediction unit partitioning may be allowed only when the maximum number of merge candidates included in the merge candidate list is equal to or greater than 2.

[0350] Optionally, when at least one of the width or height is greater than 64, disadvantages may occur during the implementation of hardware that redundantly accesses a data processing unit of 64×64 size. Therefore, when at least one of the width or height of the coding block is greater than a threshold, partitioning the coding block into multiple prediction units may not be allowed. In an example, when at least one of the width or height of the coding block is greater than 64 (for example, when at least one of the width or height is 128), prediction unit partitioning may not be used.

[0351] Optionally, by considering the maximum number of samples that can be processed simultaneously by the implemented hardware, prediction unit partitioning may not be allowed for coding blocks with a number of samples greater than a threshold. In an example, prediction unit partitioning may not be allowed for coding tree blocks with a number of samples greater than 4096.

[0352] Optionally, for a coding block in which the number of samples included in the coding block is less than a threshold, prediction unit partitioning may not be allowed. In an example, when the number of samples included in the coding block is less than 64, prediction unit partitioning may be set not to be applied to the coding block.

[0353] Alternatively, whether to apply the prediction unit partition to the coding block may be determined based on at least one of whether the aspect ratio of the coding block is less than a first threshold or whether the aspect ratio of the coding block is greater than a second threshold. In this case, as shown in Equation 5 below, the aspect ratio whRatio of the coding block may be determined as a ratio of a width CbW to a height CbH of the coding block.

[0354] [Equation 5]

[0355] whRatio=abs(Log2(CbW / CbH))

[0356] Optionally, when the aspect ratio of the coding block is less than a first threshold or greater than a second threshold, prediction unit partitioning may be applied to the coding block. In an example, when the first threshold is 4, prediction unit partitioning may not be allowed for coding blocks of 64×4 or 4×64 size.

[0357] Optionally, based on the partition type of the parent node, it can be determined whether prediction unit partitioning is allowed. In the example, when the coding block (parent node) is partitioned based on quadtree partitioning, prediction unit partitioning can be applied to the coding block (leaf node). On the other hand, when the coding block (parent node) is partitioned based on binary tree partitioning or ternary tree partitioning, prediction unit partitioning can be set to be not allowed for the coding block (leaf node).

[0358] Optionally, based on the prediction coding mode of the coding block, it can be determined whether prediction unit partitioning is allowed. In an example, prediction unit partitioning can be allowed only when the coding block is encoded by intra-frame prediction, when the coding block is encoded by inter-frame prediction, or when the coding block is encoded by a predefined inter-frame prediction mode. In this case, the predefined inter-frame prediction mode may include at least one of a merge mode, a motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0359] Optionally, based on the size of the parallel processing region, it may be determined whether prediction unit partitioning is allowed. In an example, when the size of the coding block is larger than the size of the parallel processing region, prediction unit partitioning may not be used.

[0360] By considering two or more of the conditions listed above, it may be determined whether to apply prediction unit partitioning to a coding block.

[0361] In another example, information indicating whether prediction unit partition is applied to a coding block may be signaled in the bitstream. The information may be signaled at a sequence, picture, slice, or block level. For example, a flag triangle_partition_flag indicating whether prediction unit partition is applied to a coding block may be signaled at a coding block level.

[0362] When it is determined to apply prediction unit partitioning to a coding block, information indicating the number of lines partitioning the coding block or positions of the lines may be signaled in a bitstream.

[0363] In the example, when the coding block is partitioned by a diagonal line, information indicating the direction of the diagonal line that partitions the coding block may be sent in the bitstream with a signal. In the example, a flag triangle_partition_type_flag indicating the direction of the diagonal line may be sent in the bitstream with a signal. The flag indicates whether the coding block is partitioned by a diagonal line connecting the upper left and lower right, or whether the coding block is partitioned by a diagonal line connecting the upper right and lower left. When the coding block is partitioned by a diagonal line connecting the upper left and lower right, it may be referred to as a left triangle partition type, and when the coding block is partitioned by a diagonal line connecting the upper right and lower left, it may be referred to as a right triangle partition type. In the example, when the value of the flag is 0, it may indicate that the partition type of the coding block is a left triangle partition type, and when the value of the flag is 1, it may indicate that the partition type of the coding block is a right triangle partition type.

[0364] In addition, information indicating whether the size of the prediction unit is the same or information indicating the position of the diagonal line for partitioning the coding block may be sent by signaling in the bitstream. In the example, when the information indicating the size of the prediction unit indicates that the size of the prediction unit is the same, the encoding of the information indicating the position of the diagonal line may be omitted, and the coding block may be partitioned into two prediction units by using a diagonal line passing through two vertices of the coding block. On the other hand, when the information indicating the size of the prediction unit indicates that the size of the prediction unit is not the same, the position of the diagonal line for partitioning the coding block may be determined based on the information indicating the position of the diagonal line. In the example, when the left triangle partition type is applied to the coding block, the position information may indicate whether the diagonal line meets the left and lower boundaries of the coding block, or whether the diagonal line meets the upper and right boundaries. Optionally, when the right triangle partition type is applied to the coding block, the position information may indicate whether the diagonal line meets the right and lower boundaries of the coding block, or whether the diagonal line meets the upper and left boundaries.

[0365] Information indicating the partition type of a coding block may be signaled at a coding block level. Thus, a partition type may be determined for each coding block to which prediction unit partitioning is applied.

[0366] In another example, information indicating the partition type of a sequence, picture, slice, tile, or coding tree unit may be sent by a signal. In this case, the partition types of the coding blocks, pictures, slices, tiles, or coding tree units to which diagonal partitioning is applied in sequence may be set to be the same.

[0367] Optionally, information for determining a partition type of a first coding unit to which a prediction unit partition is applied in a coding tree unit may be encoded and signaled, and a second coding unit to which a prediction unit partition is applied or a subsequent coding unit to which a prediction unit partition is applied may be set to use the same partition type as the first coding unit.

[0368] In another example, the partition type of the coding block may be determined based on the partition type of the neighboring block. In this case, the neighboring block may include a neighboring block adjacent to the upper left corner of the coding block, a neighboring block adjacent to the upper right corner, a neighboring block adjacent to the lower left corner, a neighboring block located above, or at least one of the neighboring blocks located on the left. In the example, the partition type of the current block may be set to be the same as the partition type of the neighboring block. Optionally, the partition type of the current block may be determined based on whether the left triangle partition type is applied to the upper left neighboring block or the right triangle partition type is applied to the upper right neighboring block or the lower left neighboring block.

[0369] In order to perform motion prediction compensation on the first prediction unit and the second prediction unit, motion information of each of the first prediction unit and the second prediction unit may be derived. In this case, the motion information of the first prediction unit and the second prediction unit may be derived from the merge candidates included in the merge candidate list. In order to distinguish a general merge candidate list from a merge candidate list for deriving motion information of a prediction unit, the merge candidate list for deriving motion information of a prediction unit is referred to as a partition mode merge candidate list or a triangle merge candidate list. In addition, the merge candidates included in the partition mode merge candidate list are referred to as partition mode merge candidates or triangle merge candidates. However, applying the above-mentioned method of deriving merge candidates and the above-mentioned method of constituting a merge candidate list to derive partition mode merge candidates and constitute a partition mode merge candidate list is also included in the scope of the present disclosure.

[0370] Information for determining a maximum number of partition mode merge candidates that can be included in the partition mode merge candidate list may be signaled in a bitstream. The information may represent a difference between the maximum number of merge candidates that can be included in the merge candidate list and the maximum number of partition mode merge candidates that can be included in the partition mode merge candidate list.

[0371] Partition mode merge candidates may be derived from spatial and temporal neighboring blocks of a coding block.

[0372] Fig.26 is a diagram illustrating neighboring blocks used to derive partition mode merge candidates.

[0373] The partition mode merge candidate may be derived by using at least one of a neighboring block located above the coding block, a neighboring block located on the left side of the coding block, or a co-located block included in a different picture from the coding block. The upper neighboring block may include: a block including a sample (xCb+CbW-1, yCb-1) located above the coding block, a block including a sample (xCb+CbW, yCb-1) located above the coding block, or a block including a sample (xCb-1, yCb-1) located above the coding block. The left neighboring block may include: a block including a sample (xCb-1, yCb+CbH-1) located on the left side of the coding block or at least one of a block including a sample (xCb-1, yCb+CbH) located on the left side of the coding block. The co-located block may be determined as one of a block including samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coding block or a block including samples (xCb / 2, yCb / 2) located at the center of the coding block in the co-located picture.

[0374] The neighboring blocks may be searched in a predefined order, and the partition mode merge candidate list may be configured with the partition mode merge candidates according to the predefined order. In an example, the partition mode merge candidates may be searched in the order of B1, A1, B0, A0, C0, B2, and C1 to configure the partition mode merge candidate list.

[0375] The motion information of the prediction unit may be derived based on the partition mode merge candidate list. In other words, the prediction units may share a single partition mode merge candidate list.

[0376] In order to derive motion information of a prediction unit, information for specifying at least one of the partition mode merge candidates included in the partition mode merge candidate list may be signaled in a bitstream. In an example, index information merge_triangle_idx for specifying at least one of the partition mode merge candidates may be signaled in a bitstream.

[0377] The index information may specify a combination of a merge candidate of the first prediction unit and a merge candidate of the second prediction unit. In an example, the following Table 1 is an example representing a combination of merge candidates according to the index information merge_triangle_idx.

[0378] [Table 1]

[0379]

[0380]

[0381] When the value of the index information merge_triangle_idx is 1, it indicates that the motion information of the first prediction unit is derived from the merge candidate with an index of 1, and the motion information of the second prediction unit is derived from the merge candidate with an index of 0. The partition mode merge candidate for deriving the motion information of the first prediction unit and the partition mode merge candidate for deriving the motion information of the second prediction unit can be determined by the index information merge_triangle_idx. The partition type of the coding block to which the diagonal partition is applied can also be determined based on the index information. In other words, the index information may specify a combination of the merge candidate of the first prediction unit, the merge candidate of the second prediction unit, and the partition direction of the coding block. When the partition type of the coding block is determined by the index information, the information triangle_partition_type_flag indicating the direction of the diagonal for partitioning the coding block may not be encoded. Table 2 shows the partition type of the coding block of the index information merge_triangle_idx.

[0382] [Table 2]

[0383] merge_triangle_idx 0 1 2 3 4 5 6 7 8 TriangleDir 0 1 1 0 0 1 1 1 0 merge_triangle_idx 9 10 11 12 13 14 15 16 17 TriangleDir 0 0 0 1 0 0 0 0 1 merge_triangle_idx 18 19 20 21 22 23 24 25 26 TriangleDir 1 1 1 0 0 1 1 1 1 merge_triangle_idx 27 28 29 30 31 32 33 34 35 TriangleDir 1 1 1 0 0 1 0 1 0 merge_triangle_idx 36 37 38 39 TriangleDir 0 1 0 0

[0384] When the variable TriangleDir is 0, it indicates that the left triangle partition type is applied to the coding block, and when the variable TriangleDir is 1, it indicates that the right triangle partition type is applied to the coding block. By combining Table 1 and Table 2, the index information merge_triangle_idx can be set to specify a combination of a merge candidate of the first prediction unit, a merge candidate of the second prediction unit, and a partition direction of the coding block. In another example, the available signal can send index information for only one of the first prediction unit and the second prediction unit, and the index of the merge candidate for the other of the first prediction unit and the second prediction unit can be determined based on the index information. In the example, the merge candidate of the first prediction unit can be determined based on the index information merge_triangle_idx representing the index of one of the partition mode merge candidates. And, the merge candidate of the second prediction unit can be specified based on merge_triangle_idx. In the example, the merge candidate of the second prediction unit can be derived by adding an offset to the index information merge_triangle_idx or subtracting an offset from the index information merge_triangle_idx. The offset can be an integer such as 1 or 2. In an example, the merge candidate of the second prediction unit may be determined as a partition mode merge candidate having a value obtained by adding 1 to merge_triangle_idx as an index. When merge_triangle_idx indicates a partition mode merge candidate having a maximum index value among the partition mode merge candidates, the motion information of the second prediction unit may be derived from a partition mode merge candidate having an index of 0 or a partition mode merge candidate having a value subtracted from merge_triangle_idx as an index. Optionally, the motion information of the second prediction unit may be derived from a partition mode merge candidate having the same reference picture as the partition mode merge candidate of the first prediction unit specified by the index information. In this case, the partition mode merge candidate having the same reference picture as the partition mode merge candidate of the first prediction unit may represent a partition mode merge candidate having the same reference picture as the partition mode merge candidate of the first prediction unit, at least one of the L0 reference picture or the L1 reference picture, which is the same as the partition mode merge candidate of the first prediction unit. When there are multiple partition mode merge candidates having the same reference picture as the partition mode merge candidate of the first prediction unit, any one of the partition mode merge candidates may be selected based on whether the merge candidate includes bidirectional motion information or at least one of the difference between the index of the merge candidate and the index information.

[0385] In another example, index information may be signaled for each of the first prediction unit and the second prediction unit. In an example, first index information 1st_merge_idx for determining a partition mode merge candidate of the first prediction unit and second index information 2nd_merge_idx for determining a partition mode merge candidate of the second prediction unit may be signaled in a bitstream. Motion information of the first prediction unit may be derived from the partition mode merge candidate determined based on the first index information 1st_merge_idx, and motion information of the second prediction unit may be derived from the partition mode merge candidate determined based on the second index information 2nd_merge_idx.

[0386] The first index information 1st_merge_idx may represent an index of one of the partition mode merge candidates included in the partition mode merge candidate list. The partition mode merge candidate of the first prediction unit may be determined as the partition mode merge candidate indicated by the first index information 1st_merge_idx.

[0387] The partition mode merge candidate indicated by the first index information 1st_merge_idx may be set to be unusable as the partition mode merge candidate of the second prediction unit. Therefore, the second index information 2nd_merge_idx of the second prediction unit may represent the index of any one of the remaining partition mode merge candidates other than the partition mode merge candidate indicated by the first index information. When the value of the second index information 2nd_merge_idx is less than the value of the first index information 1st_merge_idx, the partition mode merge candidate of the second prediction unit may be determined as the partition mode merge candidate having the index information represented by the second index information 2nd_merge_idx. On the other hand, when the value of the second index information 2nd_merge_idx is equal to or greater than the value of the first index information 1st_merge_idx, the partition mode merge candidate of the second prediction unit may be determined as the partition mode merge candidate having a value obtained by adding 1 to the value of the second index information 2nd_merge_idx as an index.

[0388] Optionally, depending on the number of partition mode merge candidates included in the partition mode merge candidate list, it may be determined whether to signal the second index information. In the example, when the maximum number of partition mode merge candidates that may be included in the partition mode merge candidate list does not exceed 2, signaling the second index information may be omitted. When signaling the second index information is omitted, the second partition mode merge candidate may be derived by adding an offset to the first index information or subtracting the offset from the first index information. In the example, when the maximum number of partition mode merge candidates that may be included in the partition mode merge candidate list is 2 and the first index information indicates index 0, the second partition mode merge candidate may be derived by adding 1 to the first index information. Optionally, when the maximum number of partition mode merge candidates that may be included in the partition mode merge candidate list is 2 and the first index information indicates 1, the second partition mode merge candidate may be derived by subtracting 1 from the first index information.

[0389] Optionally, when the second index information is omitted from being signaled, the second index information may be inferred as a default value. In this case, the default value may be 0. The second partition mode merge candidate may be derived by comparing the first index information with the second index information. In the example, when the second index information is less than the first index information, the merge candidate with an index of 0 may be set as the second partition mode merge candidate, and when the second index information is the same as or greater than the first index information, the merge candidate with an index of 1 may be set as the second partition mode merge candidate.

[0390] When the partition mode merge candidate has unidirectional motion information, the unidirectional motion information of the partition mode merge candidate may be set as the motion information of the prediction unit. On the other hand, when the partition mode merge candidate has bidirectional motion information, only one of the L0 motion information or the L1 motion information may be set as the motion information of the prediction unit. Which of the L0 motion information or the L1 motion information is to be adopted may be determined based on the index of the partition mode merge candidate or the motion information of another prediction unit.

[0391] In an example, when the index of the partition mode merge candidate is an even number, the L0 motion information of the prediction unit may be set to 0, and the L1 motion information of the partition mode merge candidate may be set to the L1 motion information of the prediction unit. On the other hand, when the index of the partition mode merge candidate is an odd number, the L1 motion information of the prediction unit may be set to 0, and the L0 motion information of the partition mode merge candidate may be set to 0. Conversely, when the index of the partition mode merge candidate is an even number, the L0 motion information of the partition mode merge candidate may be set to the L0 motion information of the prediction unit, and when the index of the partition mode merge candidate is an odd number, the L1 motion information of the partition mode merge candidate may be set to the L1 motion information of the prediction unit. Optionally, for the first prediction unit, when the partition mode merge candidate for the first prediction unit is an even number, the L0 motion information of the partition mode merge candidate may be set to the L0 motion information of the first prediction unit, but for the second prediction unit, when the partition mode merge candidate for the second prediction unit is an odd number, the L1 motion information of the partition mode merge candidate may be set to the L1 motion information of the second prediction unit.

[0392] Alternatively, when the first prediction unit has L0 motion information, the L0 motion information of the second prediction unit may be set to 0, and the L1 motion information of the partition mode merge candidate may be set to the L1 information of the second prediction unit. On the other hand, when the first prediction unit has L1 motion information, the L1 motion information of the second prediction unit may be set to 0, and the L0 motion information of the partition mode merge candidate may be set to the L0 motion information of the second prediction unit.

[0393] The partition mode merge candidate list used to derive motion information of the first prediction unit may be set to be different from the partition mode merge candidate list used to derive motion information of the second prediction unit.

[0394] In an example, when a partition mode merge candidate for deriving motion information of a first prediction unit in a partition mode merge candidate list is specified based on index information of the first prediction unit, motion information of a second prediction unit may be derived by using a partition mode merge list including remaining partition mode merge candidates other than the partition mode merge candidate indicated by the index information. Specifically, the motion information of the second prediction unit may be derived from one of the remaining partition mode merge candidates.

[0395] Therefore, the maximum number of partition mode merge candidates included in the partition mode merge candidate list of the first prediction unit may be different from the maximum number of partition mode merge candidates included in the partition mode merge candidate list of the second prediction unit. In an example, when the partition mode merge candidate list of the first prediction unit includes M merge candidates, the partition mode merge candidate list of the second prediction unit may include M-1 merge candidates in addition to the partition mode merge candidate indicated by the index information of the first prediction unit.

[0396] In another example, the availability of the neighboring blocks may be determined by deriving a merge candidate for each prediction unit based on neighboring blocks adjacent to the coding block but by considering the shape or position of the prediction unit.

[0397] Fig. 27 is a diagram for explaining an example of determining availability of neighboring blocks per prediction unit.

[0398] Neighboring blocks that are not adjacent to the first prediction unit may be set to be unavailable for the first prediction unit, and neighboring blocks that are not adjacent to the second prediction unit may be set to be unavailable for the second prediction unit.

[0399] In the example, as in Fig. 27 In the example shown in (a), when the left triangle partition type is applied to the coding block, blocks A1, A0, and A2 adjacent to the first prediction unit in the neighboring blocks adjacent to the coding block can be determined to be available for the first prediction unit, but blocks B0 and B1 can be determined to be unavailable for the first prediction unit. Therefore, the partition mode merge candidate list of the first prediction unit may include partition mode merge candidates derived from blocks A1, A0, and A2, but it may not include partition mode merge candidates derived from blocks B0 and B1.

[0400] As in Fig. 27 In the example shown in (b), when the left triangle partition type is applied to the coding block, blocks B0 and B1 adjacent to the second prediction unit may be determined to be available for the second prediction unit, but blocks A1, A0, and A2 may be determined to be unavailable for the second prediction unit. Therefore, the partition mode merge candidate list of the second prediction unit may include the partition mode merge candidate derived from blocks B0 and B1, but it may not include the partition mode merge candidate derived from blocks A1, A0, and A2.

[0401] Therefore, the number of partition mode merging candidates or the range of partition mode merging candidates that a prediction unit can use may be determined based on at least one of the position of the prediction unit or the partition type of the coding block.

[0402] Motion compensated prediction of the coding block may be performed based on each of the motion information of the first prediction unit and the motion information of the second prediction unit. In this case, quality degradation may be generated at the boundary of the first prediction unit and the second prediction unit. In an example, quality continuity may be degraded around an edge at the boundary of the first prediction unit and the second prediction unit. In order to reduce quality degradation at the boundary, prediction samples may be derived through a smoothing filter or weighted prediction.

[0403] The prediction samples in the coding block to which the diagonal partitioning is applied may be derived according to a weighted sum operation of a first prediction sample obtained based on motion information of the first prediction unit and a second prediction sample obtained based on motion information of the second prediction unit. Alternatively, the prediction samples of the first prediction unit may be derived from a first prediction block determined based on motion information of the first prediction unit, and the prediction samples of the second prediction unit may be derived from a second prediction block determined based on motion information of the second prediction unit, but the prediction samples on the boundary area of ​​the first prediction unit and the second prediction unit may be derived based on a weighted sum operation of the first prediction sample included in the first prediction block and the second prediction sample included in the second prediction block. In the example, the following Equation 6 represents an example of deriving the prediction samples of the first prediction unit and the second prediction unit.

[0404] [Equation 6]

[0405] P(x, y)=w1*P1(x, y)+(1-w1)*P2(x, y)

[0406] In Equation 6, P1 represents a first prediction sample, and P2 represents a second prediction sample. w1 represents a weight applied to the first prediction sample, and (1-w1) represents a weight applied to the second prediction sample. As in the example shown in Equation 6, the weight applied to the second prediction sample can be derived by subtracting the weight applied to the first prediction sample from a constant value.

[0407] When the left triangle partition type is applied to the coding block, the boundary area may include prediction samples having the same x-axis coordinate and y-axis coordinate. On the other hand, when the right triangle partition type is applied to the coding block, the boundary area may include prediction samples having a sum of the x-axis coordinate and the y-axis coordinate equal to or greater than the first threshold and equal to or less than the second threshold.

[0408] The size of the boundary area may be determined based on at least one of the size of the coding block, the shape of the coding block, the motion information of the prediction unit, the motion vector difference of the prediction unit, the picture order count of the reference picture, or the difference between the first prediction sample and the second prediction sample on the diagonal boundary.

[0409] Fig.28 and Fig.29is a diagram illustrating an example of deriving a prediction sample based on a weighted sum operation of a first prediction sample and a second prediction sample. Fig.28 shows the case where the left triangle partition type is applied to the coding block, Fig.29 The case where the right triangle partition type is applied to the coding block is shown. Fig.28 (a) and FIG. 29 (a) are diagrams showing the prediction aspect of the luminance component, and Fig.28 (b) and Fig.29 (b) is a diagram showing the prediction aspect of the chrominance component.

[0410] In the illustrated diagram, the numbers marked on the prediction samples around the boundary of the first prediction unit and the second prediction unit represent the weights applied to the first prediction samples. In an example, when the number marked on the prediction sample is N, the prediction sample can be derived by applying a weight of N / 8 to the first prediction sample and a weight of (1-(N / 8)) to the second prediction sample.

[0411] In the non-boundary area, the first prediction sample point or the second prediction sample point can be determined as the prediction sample point. Fig.28 In the example of , the first prediction sample derived based on the motion information of the first prediction unit may be determined as the prediction sample in the region belonging to the first prediction unit. On the other hand, the second prediction sample derived based on the motion information of the second prediction unit may be determined as the prediction sample in the region belonging to the second prediction unit.

[0412] Check Fig.29 In the example of , the first prediction sample derived based on the motion information of the first prediction unit may be determined as a prediction sample in an area where the sum of the x-axis coordinate and the y-axis coordinate is less than a first threshold. On the other hand, the second prediction sample derived based on the motion information of the second prediction unit may be determined as a prediction sample in an area where the sum of the x-axis coordinate and the y-axis coordinate is greater than a second threshold.

[0413] The threshold for determining the non-boundary area may be determined based on at least one of the size of the coding block, the shape of the coding block, or the color component. In an example, when the threshold of the luma component is set to N, the threshold of the chroma component may be set to N / 2.

[0414] The prediction samples included in the boundary area may be derived based on a weighted sum operation of the first prediction samples and the second prediction samples. In this case, the weights applied to the first prediction samples and the second prediction samples may be determined based on at least one of a position of the prediction samples, a size of the coding block, a shape of the coding block, or a color component.

[0415] In the example, as in Fig.28In the example shown in (a) of FIG. 1 , a prediction sample having the same x-axis coordinate and y-axis coordinate can be derived by applying the same weight to the first prediction sample and the second prediction sample. A prediction sample having an absolute value of a difference between the x-axis coordinate and the y-axis coordinate of 1 can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0416] Optionally, as in Fig.28 In the example shown in (b) of , a prediction sample having the same x-axis coordinate and y-axis coordinate can be derived by applying the same weight to the first prediction sample and the second prediction sample, and a prediction sample having an absolute value of a difference between the x-axis coordinate and the y-axis coordinate of 1 can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0417] In the example, as in Fig.29 In the example shown in (a), a prediction sample whose sum of x-axis coordinate and y-axis coordinate is 1 less than the width or height of the coding block can be derived by applying the sample weight to the first prediction sample and the second prediction sample. A prediction sample whose sum of x-axis coordinate and y-axis coordinate is the same as the width or height of the coding block or is 2 less than the width or height of the coding block can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (3:1) or (1:3). A prediction sample whose sum of x-axis coordinate and y-axis coordinate is 1 greater than the width or height of the coding block or 3 less than the width or height of the coding block can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0418] Optionally, as in Fig.29 In the example shown in (b), a prediction sample whose sum of x-axis coordinate and y-axis coordinate is 1 less than the width or height of the coding block can be derived by applying the sample weight to the first prediction sample and the second prediction sample. A prediction sample whose sum of x-axis coordinate and y-axis coordinate is the same as the width or height of the coding block or is 2 less than the width or height of the coding block can be derived by setting the weight ratio applied to the first prediction sample and the second prediction sample to (7:1) or (1:7).

[0419] In another example, the weight may be determined by considering the position of the prediction sample or the shape of the coding block. Equations 7 to 9 represent examples of deriving weights when the left triangle partition type is applied to the coding block. Equation 7 represents an example of deriving a weight applied to the first prediction sample when the coding block is a square.

[0420] [Equation 7]

[0421] w1=(x-y+4) / 8

[0422] In Equation 7, x and y represent the positions of the prediction samples. When the coding block is non-square, the weight applied to the first prediction sample may be derived as in the following Equation 8 or Equation 9. Equation 8 indicates that the width of the coding block is greater than the height, and Equation 9 indicates that the width of the coding block is less than the height.

[0423] [Equation 8]

[0424] w1=((x / whRatio)-y+4) / 8

[0425] [Equation 9]

[0426] w1=(x-(y*whRatio)+4) / 8

[0427] When the right triangle partition type is applied to the coding block, the weight applied to the first prediction sample may be determined as in Equations 10 to 12. Equation 10 represents an example of deriving the weight applied to the first prediction sample when the coding block is a square.

[0428] [Equation 10]

[0429] w1=(CbW-1-xy)+4) / 8

[0430] In Equation 10, CbW represents the width of the coding block. When the coding block is non-square, the weight applied to the first prediction sample may be derived as in the following Equation 11 or Equation 12. Equation 11 indicates that the width of the coding block is greater than the height, and Equation 12 indicates that the width of the coding block is less than the height.

[0431] [Equation 11]

[0432] w1=(CbH-1-(x*whRatio)-y)+4) / 8

[0433] [Equation 12]

[0434] w1=(CbW-1-x-(y*whRatio)+4) / 8

[0435] In Equation 11, CbH represents the height of a coding block.

[0436] When diagonal partitioning is applied to a coding block, a combined prediction mode in which an intra prediction mode and a merge mode are combined may be set to not be applied to the coding block.

[0437] When the encoding / decoding of the coding block is completed, the motion information of the coding block where the encoding / decoding is completed may be stored for subsequent encoding / decoding of the coding block. The motion information may be stored in units of sub-blocks of a preset size. In the example, the sub-block of the preset size may have a 4×4 size. Optionally, the size or shape of the sub-block may be determined differently depending on the size or shape of the coding block.

[0438] When the subblock belongs to the first prediction unit, the motion information of the first prediction unit may be stored as the motion information of the subblock. On the other hand, when the subblock belongs to the second prediction unit, the motion information of the second prediction unit may be stored as the motion information of the subblock.

[0439] When the sub-block is on the boundary of the first prediction unit and the second prediction unit, any one of the motion information of the first prediction unit and the motion information of the second prediction unit may be set as the motion information of the sub-block. In an example, the motion information of the first prediction unit may be set as the motion information of the sub-block, or the motion information of the second prediction unit may be set as the motion information of the sub-block.

[0440] In another example, when the sub-block is on the boundary of the first prediction unit and the second prediction unit, any one of the L0 motion information and the L1 motion information of the sub-block may be derived from the first prediction unit, and the other of the L0 motion information and the L1 motion information of the sub-block may be derived from the second prediction unit. In the example, the L0 motion information of the first prediction unit may be set as the L0 motion information of the sub-block, and the L1 motion information of the second prediction unit may be set as the L1 motion information of the sub-block. However, when the first prediction unit and the second prediction unit have only the L0 motion information or only the L1 motion information, the motion information of the sub-block may be determined by selecting any one of the first prediction unit or the second prediction unit. Optionally, the average of the motion vectors of the first prediction unit and the second prediction unit may be set as the motion vector of the sub-block.

[0441] The motion information of the encoding / decoding completed coding block may be updated in the motion information table. In this case, the motion information of the coding block to which the prediction unit partition is applied may be set not to be added to the motion information table.

[0442] Alternatively, only the motion information of any one of the plurality of prediction units generated by partitioning the coding block may be added to the motion information table. In the example, although the motion information of the first prediction unit may be added to the motion information table, the motion information of the second prediction unit may not be added to the motion information table. In this case, the prediction unit to be added to the motion information table may be selected based on at least one of the size of the coding block, the shape of the coding block, the size of the prediction unit, the shape of the prediction unit, or whether bidirectional prediction is performed on the prediction unit.

[0443] Optionally, motion information of each of a plurality of prediction units generated by partitioning a coding block may be added to a motion information table. In this case, the order in which the motion information table is added may be predefined in the encoding device and the decoding device. In an example, the motion information of a prediction unit including an upper left corner sample or a lower left corner sample may be added to the motion information table before the motion information of another prediction unit. Optionally, the order in which the motion information table is added may be determined based on at least one of a merge index or a reference picture index of each prediction unit or a size of a motion vector.

[0444] Alternatively, motion information combining motion information of the first prediction unit with motion information of the second prediction unit may be added to the motion information table. Any one of the L0 motion information and the L1 motion information of the combined motion information may be derived from the first prediction unit, and the other of the L0 motion information and the L1 motion information may be derived from the second prediction unit.

[0445] Optionally, based on whether the reference picture of the first prediction unit is the same as the reference picture of the second prediction unit, the motion information to be added to the motion information table may be determined. In the example, when the reference picture of the first prediction unit is different from the reference picture of the second prediction unit, the motion information of any one of the first prediction unit and the second prediction unit or the motion information of the first prediction unit and the second prediction unit combined may be added to the motion information table. On the other hand, when the reference picture of the first prediction unit is the same as the reference picture of the second prediction unit, the average of the motion vector of the first prediction unit and the motion vector of the second prediction unit may be added to the motion information table.

[0446] Optionally, based on the size of the coding block, the shape of the coding block, or the partition shape of the coding block, the motion vector to be added to the motion information table may be determined. In the example, when the right triangular partition is applied to the coding block, the motion information of the first prediction unit may be added to the motion information table. On the other hand, when the left triangular partition is applied to the coding block, the motion information of the second prediction unit may be added to the motion information table, or the motion information combining the motion information of the first prediction unit and the motion information of the second prediction unit may be added to the motion information table.

[0447] A motion information table for storing motion information of coding blocks to which prediction unit partitioning is applied may be defined separately. In the example, the motion information of coding blocks to which prediction unit partitioning is applied may be stored in a partition mode motion information table. The partition mode motion information table may be referred to as a triangle motion information table. In other words, the motion information of coding blocks to which prediction unit partitioning is not applied may be stored in a general motion information table, and the motion information of coding blocks to which prediction unit partitioning is applied may be stored in a partition mode motion information table. An embodiment of adding motion information of coding blocks to which the above-mentioned prediction unit partitioning is applied to a motion information table may be applied to update the partition mode motion information table. In the example, motion information of a first prediction unit, motion information of a second prediction unit, motion information combining the motion information of the first prediction unit and the motion information of the second prediction unit, and motion information averaging the motion vector of the first prediction unit and the motion vector of the second prediction unit may be added to the partition mode motion information table.

[0448] When prediction mode partition is not applied to the coding block, the merge candidate can be derived by using the general motion information table. On the other hand, when prediction mode partition is applied to the coding block, the merge candidate can be derived by using the partition mode motion information table.

[0449] When a bidirectional motion vector is applied to a current block, a motion vector of the current block may be derived, and a refined motion vector may be derived per subblock based on the derived motion vector. When the refined motion vector is derived, a refined prediction sample may be obtained by using the refined motion vector.

[0450] The prediction samples in the current block may be derived based on the L0 prediction samples, the L1 prediction samples, and the refined prediction samples. Equation 13 represents an example of deriving the prediction samples in the current block.

[0451] [Equation 13]

[0452] pred BDOF =(I (o) (x,y)+I (1) (x, y)+b(x, y)+Offset)>>shift

[0453] In Equation 13, pred BDOF Represents the predicted sample at position (x, y) in the current block. (0) (x, y) represents the L0 prediction sample at position (x, y), and I (1)(x, y) represents the L1 prediction sample at the position (x, y). b(x, y) represents the refined prediction sample at the position (x, y). The refined prediction sample can be derived based on the refined motion vector. The offset can be set to a fixed value or can be derived based on the bit depth of the current picture or the size / shape of the block. The shift parameter shift can be set to a fixed value or can be derived based on the bit depth of the current picture or the size / shape of the block.

[0454] The refined prediction sample may be derived by applying the bidirectional optical flow to the current block. In the example, Equation 14 represents an example of deriving a refined prediction sample of a position of (x, y) in the current block.

[0455] [Equation 14]

[0456]

[0457] In equation 14, (v x ,v y ) represents the refined motion vector.

[0458] represents the L1 gradient in the horizontal direction, and Represents the L0 gradient in the horizontal direction. represents the L1 gradient in the vertical direction, and Represents the L0 gradient in the vertical direction.

[0459] The refined motion vector may be derived based on the horizontal gradient and the vertical gradient. Specifically, the refined motion vector may be derived based on an autocorrelation value, wherein the autocorrelation value is derived based on the horizontal gradient and the vertical gradient. In an example, Equation 15 represents an example of deriving the refined motion vector.

[0460] [Equation 15]

[0461]

[0462] In Equation 15, S1 to S6 represent autocorrelation values.

[0463] The refined motion vector may be determined in units of sub-blocks. In other words, the samples in a sub-block may share one refined motion vector. Therefore, a refined prediction sample for each sample in a sub-block may be derived based on the gradient of each sample and the refined motion vector determined at the sub-block level.

[0464] The size and / or shape of the sub-block may be predefined in the encoding device and the decoding device. Alternatively, the size and / or shape of the sub-block may be adaptively determined according to the size and / or shape of the current block.

[0465] Equation 16 and Equation 17 represent examples of deriving a horizontal direction gradient and a vertical direction gradient, respectively.

[0466] [Equation 16]

[0467]

[0468] [Equation 17]

[0469]

[0470] In Equation 16 and Equation 17, k indicates 0 or 1. When k is 0, it indicates the L0 direction, and when k is 1, it indicates the L1 direction.

[0471] The horizontal gradient can be derived by the difference between the neighboring sample points adjacent to the predicted sample point at position (i, j). Specifically, the horizontal gradient can be derived based on the predicted sample point I adjacent to the right side of the position (i, j). (k) (i+1,j) and the predicted sample I adjacent to the left of position (i,j) (k) The horizontal gradient is derived by taking the difference between (i-1, j).

[0472] The vertical gradient can be derived by the difference between the neighboring sample points adjacent to the predicted sample point at position (i, j). Specifically, the vertical gradient can be derived based on the predicted sample point I adjacent to the top of position (i, j). (k) (i, j+1) and the predicted sample point I adjacent to the bottom of position (i, j) (k) The vertical gradient is derived by taking the difference between (i, j-1).

[0473] In Equation 16 and Equation 17, it is shown that the gradient is derived by the difference between two samples, but the gradient can be derived by using more samples. In an example, the horizontal direction gradient can be derived based on N samples at the left position and N samples at the right position of the prediction sample, and the vertical direction gradient can be derived based on N samples at the upper position and N samples at the lower position of the prediction sample. In this case, N can be a real number such as 1, 2, 3 or 4. The number of samples used to derive the gradient can be determined differently according to the size / shape of the current block.

[0474] The shift parameter shift1 may be set to a fixed value, or may be derived based on the bit depth of the current picture or the size / shape of the block.

[0475] The autocorrelation value shown in Equation 15 can be derived as in Equation 18 below.

[0476] [Equation 18]

[0477]

[0478] The parameters shown in Equation 18 can be derived as in Equation 19 below.

[0479] [Equation 19]

[0480]

[0481] As in the above example, deriving prediction samples based on refined prediction samples derived by refining motion vectors may be referred to as a bidirectional optical flow (BDOF) encoding method.

[0482] Whether to use the BDOF encoding method can be determined based on at least one of the following: the size / shape of the current block, the output order / type of the L0 reference picture, the output order / type of the L1 reference picture, whether motion compensation in sub-block units (e.g., ATMVP or STMVP) is performed, whether an affine motion model is applied, whether symmetric MVD is applied, whether bidirectional weights (e.g., L0 weights and L1 weights) are the same, or whether a prediction mode with partitioning is applied.

[0483] In an example, whether to use the BDOF encoding method can be determined by considering whether the difference between the current picture and the L0 reference picture is the same as the difference between the current picture and the L1 reference picture or the time direction of the L0 reference picture and the L1 reference picture. In the example, the BDOF encoding method can be allowed only when the difference between the current picture and the L0 reference picture is the same as the difference between the current picture and the L1 reference picture, or when the time direction of the L0 reference picture is different from the time direction of the L1 reference picture. In this case, the time direction indicates whether the reference picture is in the forward direction or in the backward direction. The forward reference picture indicates a reference picture whose POC is greater than the current picture, and the backward reference picture indicates a reference picture whose POC is less than the current picture.

[0484] Optionally, when an affine motion model is applied to the current block, the BDOF encoding method may be set to be disallowed.

[0485] Optionally, the BDOF encoding method can be applied only when the value of the flag regular_merge_flag indicating whether the general merge mode is applied to the current block is 1. When the value of the flag regular_merge_flag is 1, it indicates that an encoding method (such as motion compensation in units of sub-blocks, prediction mode partitioning, or CIIP, etc.) is not applied to the current block.

[0486] Alternatively, the BDOF encoding method may be applied only when motion compensation (eg, ATMVP or STMVP) in units of sub-blocks is not applied to the current block.

[0487] Optionally, the BDOF encoding method may be applied only when the symmetric MVD encoding method is not applied to the current block.

[0488] Optionally, the BDOF encoding method can be applied only when the L0 weight is the same as the L1 weight.

[0489] Optionally, the BDOF encoding method may be applied only when both the L0 reference picture and the L1 reference picture are short-term reference pictures.

[0490] Optionally, the BDOF encoding method may be allowed only when at least one of the width, height or size of the current block is equal to or greater than a threshold value. The size of the current block represents a value derived by multiplying the width by the height. In an example, a value obtained by multiplying the width by the height by Log2 may be set as the size value of the current block. Optionally, the number of samples included in the current block may be set as the size value of the current block. The threshold value may be an integer such as 4, 8, 16 or 32.

[0491] Alternatively, the BDOF encoding method may be set to be disallowed for a block to which a prediction mode with partitioning is applied. In an example, when the current block is partitioned into two triangle prediction units, it may be set to "no" to apply the BDOF encoding method.

[0492] Optionally, when a prediction method combining intra prediction and inter prediction (combined intra-frame inter prediction, CIIP) is applied, the BDOF encoding method may be set to be disallowed. In other words, when a prediction method combining intra prediction and inter prediction (combined intra-frame inter prediction, CIIP) is not applied, the BDOF encoding method may be set to be allowable.

[0493] Alternatively, whether the BDOF encoding method is allowed may be determined by combining multiple of the conditions listed above.

[0494] Intra Block Copy (IBC) is used to predict a current block from a region reconstructed before the current block in the current picture. Specifically, the prediction samples of the current block can be obtained by using a reference block in a pre-reconstructed region in the current picture.

[0495] Information indicating whether the intra block copy mode is used may be signaled in the bitstream. In an example, a flag pred_mode_ibc_flag indicating whether the intra block copy mode is applied to the current block may be signaled in the bitstream.

[0496] When the information indicating the encoding mode of the current block indicates inter-frame prediction, a flag may be signaled. In the example, when the value of pred_mode_ibc_flag is 0, it indicates that the prediction of the current block is performed by using a pre-decoded picture different from the current picture as a reference picture. On the other hand, when the value of pred_mode_ibc_flag is 1, it indicates that the prediction of the current block is performed by using the current picture as a reference picture.

[0497] Optionally, when the signaling flag pred_mode_ibc_flag is omitted, whether to use the intra-block copy mode may be determined based on at least one of the size or shape of the current block, the coding mode, or the slice type. In the example, when the size of the current block is 4×4 or when the coding mode of the current block is intra-frame mode, whether to apply the IBC mode may be determined based on whether the skip mode is applied to the current block. Specifically, when the size of the current block is 4×4 or when the coding mode of the current block is intra-frame mode, the current block may be predicted by replacing the skip mode with the IBC mode. In other words, when the size of the current block is 4×4 or when the coding mode of the current block is intra-frame mode, the reconstructed block of the current block may be derived by using the current picture as a reference picture. The flag cu_skip_flag indicating whether the skip mode is applied may be signaled in the bitstream. When the skip mode is applied, the predicted sample of the current block may be set to the reconstructed sample.

[0498] When the size of the current block is larger than a threshold, the IBC mode may be set to be unavailable. The threshold may be 64×64 or 128×128. Alternatively, whether to use the intra block copy mode may be determined based on the size, time ID or color component of the current picture.

[0499] Fig.30 is a diagram for explaining an aspect of prediction based on an intra block copy mode.

[0500] The coordinate difference between the current block and the reference block may be defined as a motion vector. In particular, the motion vector in the intra block mode may be referred to as a block vector (BV). For example, dX (the difference between the x-coordinate of the upper left sample of the current block and the x-coordinate of the upper left sample of the reference block) may be defined as a block vector in the horizontal direction (or an x-axis block vector), and the difference between the y-coordinate of the upper left sample of the current block and the y-coordinate of the upper left sample of the reference block may be defined as a block vector in the vertical direction (or a y-axis block vector).

[0501] Hereinafter, the intra block copy mode will be described in detail with reference to the accompanying drawings.

[0502] Fig.31 Detailed description is a flowchart illustrating a prediction process of a current block based on an intra block copy mode according to an embodiment of the present disclosure.

[0503] First, a block vector of the current block may be derived (S3101). The block vector of the current block may be derived based on a neighboring block adjacent to the current block. Specifically, the block vector of the current block may be derived by setting the block vector of the neighboring block to be the same as the block vector of the current block or by adding a difference vector to the block vector of the neighboring block.

[0504] The derivation method of the motion vector defined in the inter prediction mode can also be applied to derive the block vector in the intra block copy mode. In an example, the block vector of the neighboring block can be set as the block vector of the current block by applying the merge mode to the current block. Alternatively, the block vector of the current block can be derived by applying the motion vector prediction mode to the current block and adding the difference vector to the block vector of the neighboring block.

[0505] For ease of description, the prefix "IBC" is added before the element used to derive the block vector in the intra block copy mode. In the example, the merge mode and the motion vector prediction mode in the intra block copy mode are respectively referred to as the IBC merge mode and the IBC motion vector prediction mode. Unless otherwise described, the embodiments of the inter prediction mode can also be applied to the intra block copy mode. In the example, the method for deriving a merge candidate and the method for updating the motion information table in the inter prediction mode can be applied to deriving an IBC merge candidate and updating the IBC motion information table.

[0506] In addition, when it is necessary to classify the inter prediction mode and the intra block copy mode, the prefix "general" is added before the element used to derive the motion vector in the inter prediction mode.

[0507] Information indicating whether the IBC merge mode is applied to the current block may be signaled in the bitstream. In an example, a flag IBC_merge_flag may be signaled in the bitstream. When the value of IBC_merge_flag is 1, it indicates that the IBC merge mode is applied to the current block, and when the value of IBC_merge_flag is 0, it indicates that the IBC merge mode is not applied to the current block. When the number of available IBC merge candidates is at least one or more, IBC_merge_flag may be signaled.

[0508] When the value of IBC_merge_flag is 0, the IBC motion vector prediction mode may be applied to the current block.

[0509] In the merge mode, the block vector of the current block may be derived from at least one of the neighboring blocks adjacent to the current block. In this case, the neighboring block may include at least one of an upper neighboring block adjacent to the upper side of the current block, a left neighboring block adjacent to the left side of the current block, a lower left neighboring block adjacent to the lower left corner of the current block, an upper right neighboring block adjacent to the upper right corner of the current block, or an upper left neighboring block adjacent to the upper left corner of the current block.

[0510] Specifically, the block vector of the current block can be derived from a neighboring block encoded by an intra block copy mode among neighboring blocks adjacent to the current block. The block vector of the available neighboring block first searched among the neighboring blocks adjacent to the current block can be derived as the block vector of the current block.

[0511] Optionally, an IBC merge candidate may be derived from neighboring blocks encoded by an intra-block copy mode around the current block, and a block vector of the current block may be derived from one of the IBC merge candidates. The IBC merge candidate may also be derived by using a co-located block of the current block. The IBC merge candidate may also be derived by using a block that is not adjacent to the current block.

[0512] The IBC merge candidates may be added to the IBC merge candidate list according to a predefined order.

[0513] Fig.32 is a diagram illustrating the order in which IBC merge candidates are added to an IBC merge candidate list.

[0514] When the coordinates of the upper left sample point of the current block are (0,0), block A1 may represent a block including a basic sample point at a position (-1,H-1), block B1 may represent a block including a basic sample point at a position (W-1,-1), block B0 may represent a block including a basic sample point at a position (W,-1), block A0 may represent a block including a basic sample point at a position (-1,H), and block B2 may represent a block including a basic sample point at a position (-1,-1). According to the order of A1, B1, B0, A0, and B2, an IBC merge candidate may be added to the IBC merge candidate list.

[0515] Optionally, the IBC merge candidate list may be configured in an order different from that shown. In an example, the IBC merge candidate list may be configured in the order of B1, B0, A1, A0, and B2 or in the order of B1, A1, B0, A1, and B2.

[0516] Alternatively, an IBC merge candidate may be derived by using only one of the candidate blocks at the upper position of the current block and one of the candidate blocks at the left position of the current block. In an example, an IBC merge candidate may be derived by using only the A1 block and the B1 block.

[0517] Optionally, the number or position of neighboring blocks used to derive IBC merge candidates may be set differently depending on the size or shape of the current block. In an example, the number of available neighboring blocks may be 0, 1, 2, 3, 4, or 5 depending on the size of the current block.

[0518] When the IBC merge candidate list includes a plurality of IBC merge candidates, index information for specifying one of the plurality of IBC merge candidates may be signaled in the bitstream.In an example, a block vector of a current block may be derived from an IBC merge candidate specified by a syntax element IBC_merge_idx.

[0519] A block encoded by inter-frame prediction may be set to be unavailable as an IBC merge candidate. Alternatively, an IBC merge candidate may be derived from a block encoded by inter-frame prediction. In this case, a block vector of an IBC merge candidate may be derived by scaling the motion vector of a block encoded by inter-frame prediction. Scaling may be performed based on the difference between the output order of the current picture (e.g., POC) and the output order of the reference picture of the block encoded by inter-frame prediction.

[0520] The block vector of the chroma (color) component can be derived by scaling the block vector of the luma (luminance) component. The scaling can be performed by bit-shifting the block vector of the luma component to the right by a shift variable. The shift variable can be determined according to the chroma format.

[0521] The maximum number of IBC merge candidates that can be included by the IBC merge candidate list may be predefined in the encoding device and the decoding device. In an example, the maximum number of merge candidates that can be included by the IBC merge candidate list may be set to 4, 5, or 6.

[0522] Optionally, the maximum number of IBC merge candidates that can be included by the IBC merge candidate list can be set to be the same as the maximum number of merge candidates that can be included by the general merge candidate list. In an example, a syntax element six_minus_max_num_merge_cand indicating the maximum number of merge candidates that can be included by the general merge candidate list can be signaled in the bitstream. The maximum number of merge candidates that can be included by the general merge candidate list and the maximum number of IBC merge candidates that can be included by the IBC merge candidate list can be derived by subtracting the value indicated by six_minus_max_num_merge_cand from 6.

[0523] Alternatively, a syntax element indicating the maximum number of IBC merge candidates that may be included in the IBC merge candidate list and a syntax element indicating the maximum number of merge candidates that may be included in the general merge candidate list may be defined separately. In an example, the maximum number of IBC merge candidates that may be included in the IBC merge candidate list may be determined by the syntax element six_minus_max_IBC_num_merge_cand, and the maximum number of merge candidates that may be included in the general merge candidate list may be determined by the syntax element six_minus_max_num_merge_cand.

[0524] Optionally, when information indicating the maximum number of IBC merge candidates is not signaled, the maximum number of IBC merge candidates may be set to a predefined value.

[0525] When the number of IBC merge candidates derived from at least one neighboring block is less than a threshold, the IBC motion information candidate included in the IBC motion information table may be added to the IBC merge candidate list as an IBC merge candidate. In this case, the threshold may be a value minus an offset from the maximum number of IBC merge candidates that may be included in the IBC merge candidate list or the maximum number of IBC merge candidates. The offset may be a natural number such as 1 or 2, etc.

[0526] Optionally, information indicating whether it is allowed to configure the IBC merge candidate list by using the IBC motion information table can be signaled in the bitstream. In the example, the syntax isUsedHibcCandList_flag indicates whether the IBC motion information table can be referenced when generating the IBC merge candidate list. The flag can be signaled at the sequence, picture, slice or block level.

[0527] Fig.33 is a diagram illustrating an example of adding an IBC motion information candidate to an IBC merge candidate list.

[0528] In the example shown, IBCMeergeCandList[i] represents the IBC merge candidate with index i, and HIbcCandList[j] represents the IBC motion information candidate with index j.

[0529] When the number of IBC merge candidates included in the IBC merge candidate list is less than the threshold, the IBC motion information candidates included in the IBC motion information table may be added to the IBC merge candidate list. In the example, when the threshold is assumed to be 5, Fig.33 The number of IBC merge candidates included in the IBC merge candidate list in (a) is 2, so the number of IBC merge candidates is less than the threshold.

[0530] Therefore, the IBC motion information candidates included in the IBC motion information table may be added as IBC merge candidates to the IBC merge candidate list. Fig.33 (b) shows that three IBC motion information candidates are added to the IBC merge candidate list.

[0531] When there is an IBC merge candidate having the same block vector as an IBC motion information candidate, the corresponding IBC motion information candidate may not be added to the IBC merge candidate list.The goal of redundancy check is the same as the example described by the redundancy check method between general motion information candidates and general merge candidates.

[0532] When the number of IBC merge candidates included in the IBC merge candidate list is less than a threshold value, although the IBC motion information candidate is added to the IBC merge candidate list, the IBC merge candidate may be derived from the general merge candidate or the general motion information candidate. Specifically, a block vector may be derived based on the motion vector of the general merge candidate or the motion vector of the general motion information candidate, and the IBC merge candidate having the derived block vector as motion information may be added to the IBC merge candidate list. The block vector may be derived by scaling the motion vector. Scaling may be performed based on the output step difference between the reference picture of the general merge candidate or the general motion information candidate and the current picture. When the general motion information has a bidirectional motion vector, one of the L0 motion information or the L1 motion information may be randomly selected, or one of the L0 motion information and the L1 motion information having a smaller output step difference with the current picture may be selected.

[0533] Optionally, when the number of IBC merge candidates included by the IBC merge candidate list is less than a threshold, although the IBC motion information candidate is added to the IBC merge candidate list, a zero block vector may be added to the IBC merge candidate list. A zero block vector indicates a block vector whose horizontal and vertical components are zero.

[0534] The IBC motion information table includes IBC motion information candidates derived from blocks encoded / decoded by the intra block copy mode. In an example, the motion information of the IBC motion information candidates included in the IBC motion information table may be set to be the same as the motion information of the blocks encoded / decoded based on the intra block copy mode. In this case, the motion information may include at least one of a block vector, a motion vector resolution, or whether an IBC merge mode (e.g., a value of IBC_merge_flag) is applied.

[0535] Fig.34 is a diagram used to explain the updating aspects of the IBC motion information table.

[0536] An IBC motion information candidate may be derived from a block encoded / decoded by an intra block copy mode or an IBC merge mode, and the derived IBC motion information candidate may be added to an IBC motion information table HIbcCandList. In an example, a block vector may be added to the IBC motion information table according to an encoding / decoding order of the block.

[0537] When the number of IBC motion information candidates included by the IBC motion information table is less than the maximum number, the block vector of the block being encoded / decoded may be added to the IBC motion information table as a new IBC motion information candidate. On the other hand, when the number of IBC motion information candidates included by the IBC motion information table is the maximum number, one of the IBC motion information candidates included by the IBC motion information table may be deleted, and the block vector of the block being encoded / decoded may be added to the IBC motion information table. In an example, as Fig.34 As shown in , when the block vector of block B4 is added to the IBC motion information table, the IBC motion information candidate with the lowest index may be removed from the IBC motion information table.

[0538] The maximum number of IBC motion information candidates that can be included by the IBC motion information table may be predefined in the encoding device and the decoding device. In an example, the maximum number of IBC motion information candidates that can be included by the IBC motion information table may be set to 1, 2, 3, 4, 5 or 6, etc.

[0539] Optionally, information indicating the maximum number of IBC motion information candidates that can be included by the IBC motion information table can be signaled in the bitstream. The information can indicate the maximum number of IBC motion information candidates that can be included by the IBC motion information table or the difference between the maximum number of IBC motion information candidates that can be included by the IBC motion information table and the maximum number of general motion information candidates that can be included by the general motion information table. The information can be signaled at a picture, slice, or sequence level.

[0540] When encoding / decoding the current block, the block vector of the current block may be added to the IBC motion information table. In this case, when the same IBC motion information candidate as the block vector of the current block preexists, the block vector of the current block may not be added to the IBC motion information table.

[0541] Alternatively, when an IBC motion information candidate identical to the block vector of the current block exists, the IBC motion information candidate identical to the block vector of the current block may be omitted, and the block vector of the current block may be added to the IBC motion information table, which causes the same effect as updating the index of the IBC motion information candidate identical to the block vector of the current block.

[0542] When the size of the current block is less than a threshold value, the block vector of the current block may not be added to the IBC motion information table. The threshold value may represent at least one of a width, a height, or a number of samples. In an example, when the number of samples included by the current block is equal to or less than 16, the IBC motion information candidate derived from the current block may not be added to the IBC motion information table.

[0543] Although the intra block copy mode is applied to the current block, the embodiments related to the merge processing area can be applied. In other words, when a neighboring block adjacent to the current block is included in the same merge processing area as the current block, the block vector of the neighboring block can be set to be unavailable as an IBC merge candidate for the current block.

[0544] The size and shape of the merge processing region for the intra block copy mode may be set to be the same as those of the merge processing region for the inter prediction mode.

[0545] Optionally, information for determining the size and / or shape of the merge processing area for the intra block copy mode may be signaled in the bitstream. In an example, information indicating the difference between the size of the merge processing area for the inter prediction mode and the size of the merge processing area for the intra block copy mode may be signaled in the bitstream.

[0546] When the current block is included in the merge processing region, although the block included in the merge processing region is encoded / decoded, a block vector of the encoded / decoded block may not be added to the IBC motion information table.

[0547] Optionally, the IBC motion information table may be updated by using only blocks at predefined positions in the merge processing area. The predefined position may include at least one of a block at an upper left position, a block at an upper right position, a block at a lower left position, a block at a lower right position, a block at a center position, a block adjacent to a right boundary, or a block adjacent to a lower boundary in the merge processing area.

[0548] The IBC motion information table may be initialized per preset unit. The preset unit may be a coding tree unit, a plurality of coding tree units, a parallel block, or a slice. In an example, the IBC motion information table may be initialized per N coding tree units or a coding tree unit row. In this case, N may be a natural number including 1.

[0549] When the IBC motion information table is initialized per coding tree unit row, it means that the IBC motion information table is initialized every time encoding / decoding of a coding tree unit adjacent to the left boundary of a picture starts.

[0550] When the IBC motion information table is empty or when the IBC motion information table is initialized, an initial IBC motion information candidate may be added to the IBC motion information table. The initial IBC motion information candidate may have a predefined block vector in the encoding device and the decoding device.

[0551] Alternatively, the initial IBC motion information candidate may be derived from a block encoded / decoded by an intra block copy mode in a coding tree unit adjacent to the current coding tree unit. In an example, the initial IBC motion information candidate may be derived from a block included in a coding tree unit adjacent to the left or above the current coding tree unit.

[0552] The block vector of the current block may be derived by combining a block vector prediction value and a block vector difference value in the IBC motion vector prediction mode. The block vector prediction value may be derived from a neighboring block adjacent to the current block. In an example, a first block vector prediction candidate may be derived from a block at an upper position of the current block, and a second block vector prediction candidate may be derived from a block at a left position of the current block.

[0553] When multiple block vector prediction candidates are available, information specifying at least one of the multiple block vector prediction candidates may be signaled in the bitstream. In an example, a flag IBC_mvp_flag specifying one of two block vector prediction candidates may be signaled in the bitstream.

[0554] The block vector difference may be determined based on information signaled in the bitstream. The information may include information for determining the size of the block vector and information for determining the direction of the block vector.

[0555] In an example, the block vector of the current block may be derived as in Equation 20 below.

[0556] [Equation 20]

[0557] BV[0]=BV p [0]+BVd[0]

[0558] BV[1]=BVp[1]+BVd[1]

[0559] BV represents a block vector of the current block, BVp represents a block vector prediction value, and BVd represents a block vector difference value. [0] represents a vector component in the horizontal direction (ie, an x-axis vector component), and [1] represents a vector component in the vertical direction (ie, a y-axis vector component).

[0560] A reference block may be specified by a block vector ( S2802 ), and samples in the reference block specified by the block vector may be set as prediction samples of the current block ( S2803 ).

[0561] The search for the reference block may be performed in a predetermined area. Therefore, the maximum value of the block vector of the current block may be determined as the difference between the boundary of the current block and the boundary of the predetermined area. In an example, the maximum value of the block vector in the horizontal direction may be set to the difference between the left boundary of the current block and the left boundary of the predetermined area, and the maximum value of the block vector in the vertical direction may be set to the difference between the upper boundary of the current block and the upper boundary of the predetermined area.

[0562] In IBC merge mode, when the maximum value of the block vector of the IBC merge candidate is greater than the maximum value of the block vector of the current block, the IBC merge candidate may be set to be unavailable. Optionally, when the maximum value of the block vector of the IBC merge candidate is greater than the block vector of the current block, the block vector of the IBC merge candidate may be changed to the maximum value of the block vector of the current block.

[0563] The predetermined area may include at least one of the following: a coding tree unit including the current block, a neighboring coding tree unit adjacent to the current coding tree unit, a current slice including the current block, or a current tile including the current block.

[0564] In the example, the reference block can be searched from the current coding tree unit and the neighboring coding tree unit. In other words, a block not included in the current coding tree unit or the neighboring coding tree unit cannot be set as a reference block. In this case, the neighboring coding tree unit may include at least one of a left coding tree unit adjacent to the left side of the current coding tree unit, an upper coding tree unit adjacent to the top of the current coding tree unit, an upper left coding tree unit adjacent to the upper left corner of the current coding tree unit, an upper right coding tree unit adjacent to the upper right corner of the current coding tree unit, or a lower left coding tree unit adjacent to the lower left corner of the current coding tree unit.

[0565] Some areas of the adjacent coding tree unit may be set as unavailable areas. It may be set that the blocks included in the unavailable areas may not be set as reference blocks of the current block. The unavailable area may be determined based on the position of the current block in the current coding tree unit. Specifically, at least one of the size or the number of the unavailable area may be determined based on the position of the current block.

[0566] Fig.35 is a diagram illustrating an unavailable area according to a location of a current block.

[0567] After partitioning the current coding tree unit and the neighboring coding tree units into a plurality of regions having the same size and shape, the unavailable region may be determined differently according to the region to which the current block in the current coding tree unit belongs. Fig.35 In FIG. 1 , it is shown that the current coding tree unit and the neighboring coding tree units are partitioned into 4 regions, but the coding tree unit may be partitioned into more or less than 4 regions.

[0568] The determination of the region to which the current block belongs may be performed based on the position of a predetermined sample point in the current block. The predetermined sample point may include at least one of an upper left sample point, an upper right sample point, a lower left sample point, a lower right sample point, or a center sample point.

[0569] In an example, when the current block belongs to an upper left region in the current coding tree unit, an upper left region of a left coding tree unit may be set as an unavailable region.

[0570] When the current block belongs to the upper right region in the current coding tree unit, the upper left region and the upper right region of the left coding tree unit may be set as unavailable regions.

[0571] When the current block belongs to the lower left region in the current coding tree unit, the upper left region, the upper right region, and the lower left region of the left coding tree unit may be set as unavailable regions.

[0572] When the current block belongs to the lower right area in the current coding tree unit, the entire area of ​​the left coding tree unit may be set as an unavailable area.

[0573] As in Fig.35 In the example shown, prediction of the current block can be performed by referring to at least one of the three coding tree units that are encoded / decoded before the current coding tree unit. To this end, a memory for storing information about the three coding tree units should be reserved. In the example, when the size of the coding tree unit is 64×64, a memory of 192×192 size is necessary.

[0574] However, as in Fig.35 In the illustrated example, when the position of the referenceable coding tree unit varies according to the position of the current coding tree unit, a problem arises in that a process of determining the position of the referenceable coding tree unit per coding unit should be additionally performed.

[0575] In order to solve such a problem, after separately setting a reference buffer of a predetermined size in the intra block copy mode, the blocks encoded / decoded in the current picture can be stored in the buffer of the predetermined size. The buffer can be called an IBC reference buffer.

[0576] The size of the IBC reference buffer may be predefined in the encoding device and the decoding device. The IBC reference buffer may have a square shape with the same width and height. In an example, the size of the IBC reference buffer may be defined as 128×128 or 256×256 size. Optionally, the IBC reference buffer may have a non-square shape with different width and height. In an example, the size of the IBC reference buffer may be defined as 96×128 or 192×256 size.

[0577] In another example, the size of the IBC reference buffer may be determined according to the size of the coding tree unit. In an example, the IBC reference buffer may be set to 256*N, and N may be derived as 128 or 64 divided by the size of the coding tree unit.

[0578] In another example, information indicating the size of the IBC reference buffer may be signaled in the bitstream. The information may be signaled at the sequence, picture or slice level.

[0579] The IBC reference buffer includes reconstruction information of the block after encoding / decoding. Specifically, when encoding / decoding the current block, the reconstructed samples of the current block can be added to the buffer. In this case, the reconstructed samples can be samples before or after applying the in-loop filter.

[0580] The reconstructed samples may be stored in the IBC reference buffer according to a predetermined bit depth. In this case, the bit depth of the IBC reference buffer may be the same as or different from the bit depth of the current picture.

[0581] Information for determining the bit depth of the IBC reference buffer may be signaled in the bitstream. In an example, at least one of information indicating whether the bit depth of the IBC reference buffer is the same as the bit depth of the current picture or information indicating the bit depth of the IBC reference buffer may be signaled in the bitstream. The information indicating the bit depth of the IBC reference buffer may also indicate a difference between the bit depth of the IBC reference buffer and the bit depth of the current picture.

[0582] In another example, the bit depth of the IBC reference buffer may be predefined in the encoding device and the decoding device. In an example, the bit depth of the IBC reference buffer may be set to 7, 8, 9 or 10.

[0583] Alternatively, the bit depth of the IBC reference buffer may be determined based on at least one of a color component, a color format, a picture type, or whether it is an HDR picture.

[0584] Fig.36 is a diagram showing an example in which reconstruction information of a current block being encoded / decoded is stored in an IBC reference buffer.

[0585] The storage position of the current block in the IBC reference buffer may be determined based on the position of the current block in the current picture. In this case, the position of the current block represents the position of a predefined sample point in the current block. The predefined sample point may include at least one of an upper left sample point, an upper right sample point, a lower left sample point, a lower right sample point, or a center sample point.

[0586] In an example, the x-axis storage position of the current block may be determined based on the x-axis position of the upper left sample of the current block in the current picture and the width of the IBC reference buffer, and the y-axis storage position of the current block may be determined based on the y-axis position of the upper left sample of the current block in the current picture and the height of the IBC reference buffer.

[0587] Equation 21 represents an example of determining the storage location of the reconstructed sample points in the current block.

[0588] [Equation 21]

[0589] IBC_Biff[x%M][y%N]=RecCu[x][y]

[0590] In Equation 21, RecCU[x][y] represents the position of the (x, y) sample in the reconstructed current block, IBC_Buff[x][y] represents the position of the (x, y) sample in the IBC reference buffer, and M and N represent the width and height of the IBC reference buffer, respectively.

[0591] As a result, the storage location of the reconstructed current block may be determined as coordinates having a value derived through a modulo operation with M at the x-axis location and a value derived through a modulo operation with N at the y-axis location.

[0592] When a pre-stored reconstruction sample point exists at the storage location of the current reconstruction sample point in the IBC reference buffer, the pre-stored reconstruction sample point may be replaced by the current reconstruction sample point.

[0593] When an IBC reference buffer is used, the block vector may represent the difference between the storage position of the current block in the IBC reference buffer and the position of the reference block. The maximum size of the block vector may be determined based on the storage position of the current block in the buffer. In an example, the horizontal direction component of the block vector may be determined as the maximum value of the difference between the storage position of the current block in the buffer and the left side boundary of the buffer or the difference between the storage position of the current block in the buffer and the right side boundary of the buffer. The vertical direction component of the block vector may be determined as the difference between the storage position of the current block in the buffer and the upper boundary of the buffer.

[0594] As in the above examples, the block vector of the current block may be derived from the IBC merge candidate, or may be derived by combining a block vector prediction value and a block vector difference value.

[0595] In this case, the modified block vector difference values ​​may be encoded / decoded based on the size of the IBC reference buffer to efficiently encode / decode the block vector difference values.

[0596] Fig.37 is a diagram for explaining an example of encoding aspects of explaining block vector differences.

[0597] The block vector prediction value may be subtracted from the block vector of the current block to derive a block vector difference value BVd[0] in the horizontal direction and a block vector difference value BVd[1] in the vertical direction.

[0598] Based on the size of the ISP reference buffer, the modified block vector difference value BVd'[0] in the horizontal direction and the modified block vector difference value BVd'[1] in the vertical direction can be derived, and the target to be encoded can be selected by comparing the block vector difference value before modification with the block vector difference value after modification.

[0599] In an example, one of the block vector difference value BVd[0] in the horizontal direction and the modified block vector difference value BVd'[0] in the horizontal direction derived by adding the width M of the IBC reference buffer to the block vector difference value BVd[0] in the horizontal direction having a smaller absolute value may be encoded as the block vector difference value in the horizontal direction. In addition, one of the block vector difference value BVd[1] in the vertical direction and the modified block vector difference value BVd'[1] in the vertical direction derived by adding the height N of the IBC reference buffer to the block vector difference value BVd[1] in the vertical direction having a smaller absolute value may be encoded as the block vector difference value in the vertical direction.

[0600] In the example, Fig.37 In the example shown in , it is assumed that the storage position of the current block in the IBC reference buffer is (96, 94), the position of the reference block is (16, 64), and the block vector prediction value is (-10, 0). The block vector representing the difference between the storage positions of the current block and the reference block can be derived as (-80, 0), and the block vector difference representing the difference between the block vector and the block vector prediction value can be derived as (-70, 0).

[0601] The modified block vector difference value in the horizontal direction may be derived as 58 by adding the width 128 of the IBC buffer to the block vector difference value -70 in the horizontal direction, and the modified block vector difference value in the vertical direction may be derived as 128 by adding the height 128 of the IBC buffer to the block vector difference value 0 in the vertical direction. Since the absolute value of the modified block vector difference value in the horizontal direction is smaller than the block vector difference value in the horizontal direction, the modified block vector difference value in the horizontal direction may be encoded as the block vector difference value. In addition, the absolute value of the modified block vector difference value in the vertical direction is larger than the block vector difference value in the vertical direction, and the modified block vector in the vertical direction may be encoded as the modified block vector in the vertical direction as it is.

[0602] In other words, a block vector difference value set to (58,0) may be encoded.

[0603] When encoding a block vector difference value having a smaller absolute value among the block vector difference value and the modified block vector difference value, the sizes of the block vector difference value in the horizontal direction and the block vector difference value in the vertical direction may not exceed M / 2 and N / 2, respectively. Therefore, the number of bits required to encode the block vector difference value may be reduced.

[0604] In the decoding device, the reference block of the current block can be specified by using the size of the IBC reference buffer and the block vector. In this case, the block vector can be derived by combining the block vector prediction value and the block vector difference value.

[0605] Specifically, the x-axis position of the reference block may be specified by performing a modulo operation using a value derived by adding the x-axis position of the current block to the block vector in the horizontal direction and the width M of the IBC reference buffer. Additionally, the y-axis position of the reference block may be specified by performing a modulo operation using a value derived by adding the y-axis position of the current block to the block vector in the vertical direction and the height N of the IBC reference buffer. Equation 22 represents an example of determining the position of the reference block based on the block vector.

[0606] [Equation 22]

[0607] RefX=x+((BVd[0]+BVp[0])%M

[0608] RefY=y+(BVd[1]+BVp[1])%N

[0609] In Equation 22, (x, y) represents the position of the current block, and (RefX, RefY) represents the position of the reference block. Fig.37 In the example shown in , the block vector of the current block can be derived as (48, 0) by adding the block vector prediction value (-10, 0) and the block vector difference value (58, 0).

[0610] When the coordinates of the upper left sample point of the current block are (96, 46), the value of adding the block vector to the coordinates of the upper left sample point of the current block is derived as (144, 46). By using the modulo operation of the width 128 and the height 128 of the IBC reference buffer, the x-axis position of the reference block can be derived as 16 (144% 128), and the y-axis position of the reference block can be derived as 46 (46% 128).

[0611] As a result, when the value of the block vector added to the position of the upper left sample of the current block is outside the boundary of the IBC reference buffer, the reference block can be derived by applying the vector outside the boundary of the IBC reference buffer to the relative boundary.

[0612] In the above example, it is described that the horizontal direction component of the block vector difference value is derived based on the width M of the IBC reference buffer, and the vertical direction component of the block vector difference value is derived based on the height N of the IBC reference buffer.

[0613] In contrast to the described example, the horizontal component of the block vector difference is derived based on the width M of the IBC reference buffer, but the vertical component of the block vector difference may be derived based on the height of the coding tree unit or half the height of the IBC reference buffer.

[0614] In an example, a value having a smaller absolute value of a block vector difference value in the vertical direction and a value obtained by adding the height of the coding tree unit to the block vector difference value in the vertical direction may be encoded as the block vector difference value in the vertical direction. In the decoding device, the Y-axis position of the reference block may be determined by performing a modulo operation of a value derived by adding the block vector in the vertical direction to the Y-axis position of the current block and the height of the coding tree unit.

[0615] The IBC motion information table may be initialized per preset unit. The preset unit may be a coding tree unit, a plurality of coding tree units, a parallel block, or a slice. In an example, the buffer may be initialized per N coding tree units or one or more coding tree unit rows. In this case, N may be a natural number including 1.

[0616] Intra prediction is a method for performing prediction on a current block by using reconstructed samples that have been encoded / decoded and are around the current block. In this regard, the reconstructed samples before applying the in-loop filter can be used for intra prediction of the current block.

[0617] The intra prediction method includes matrix-based intra prediction and intra prediction according to a direction with adjacent reconstruction samples. Information indicating the intra prediction method of the current block can be sent by signal in the bit stream. The information can be a 1-bit flag. Optionally, the intra prediction of the current block can be determined based on at least one of the position of the current block, the size of the current block, the shape of the current block, or the intra prediction method of the neighboring block. In the example, when there is a current block across the picture boundary, it can be set so that the matrix-based intra prediction method is not applied to the current block.

[0618] The matrix-based intra prediction method is a method for obtaining a prediction block of the current block based on the matrix stored in the encoder and decoder and the matrix product of the reconstructed samples around the current block. Information for specifying any one of a plurality of pre-stored matrices can be signaled in the bitstream. The decoder can determine the matrix for performing intra prediction on the current block based on the above information and the size of the current block.

[0619] Generally, intra prediction is a method of obtaining a prediction block of a current block based on a non-directional intra prediction mode or a directional intra prediction mode.

[0620] The residual image can be derived by subtracting the predicted image from the original image. In this regard, when the residual image is converted to the frequency domain, the subjective image quality of the image will not be significantly reduced even if the high-frequency components are removed from the frequency components. Therefore, when the value of the high-frequency component is transformed to a small value, or when the value of the high-frequency component is set to 0, the compression efficiency can be improved without causing large visual distortion. Reflecting the above characteristics, a transformation can be performed on the current block so as to decompose the residual image into two-dimensional frequency components. The transformation can be performed by using a transformation method such as DCT (discrete cosine transform), DST (discrete sine transform), etc.

[0621] DCT is to decompose (or transform) the residual image into two-dimensional frequency components by using cosine transform, while DST is to synthesize (or transform) the residual image into two-dimensional frequency components by using sine transform. As a result of transforming the residual image, the frequency components can be represented as a basic image. In the example, when DCT transform is performed on a block of size N×N, N 2 The size of each basic pattern component included in the N×N size block can be obtained by transformation. According to the transformation method used, the size of the basic pattern component can be called a DCT coefficient or a DST coefficient.

[0622] The transform method DCT is mainly used to transform images with many non-zero low-frequency components. The transform method DST is mainly used to transform images with many high-frequency components.

[0623] The residual image may also be transformed by using a transform method other than DCT or DST.

[0624] Hereinafter, transforming the residual image into two-dimensional frequency components is referred to as two-dimensional image transform. In addition, the size of the basic pattern component obtained by the transform is referred to as a transform coefficient. In the example, the transform coefficient may represent a DCT coefficient or a DST coefficient. When both the first transform and the second transform described below are applied, the transform coefficient may represent the basic pattern component generated by the result of the second transform. In addition, the residual sample points skipped by applying the transform are also referred to as transform coefficients.

[0625] The transformation method may be determined in units of blocks. The transformation method may be determined based on at least one of a prediction coding mode of the current block, a size of the current block, or a shape of the current block. In an example, when the current block is encoded by an intra prediction mode and the size of the current block is less than N×N, the transformation may be performed by using a DST transformation method. On the other hand, when the condition is not met, the transformation may be performed by using a DCT transformation method.

[0626] The two-dimensional image transform may not be performed on some blocks of the residual image. Not performing the two-dimensional image transform may be referred to as transform skipping. Transform skipping may mean that the first transform and the second transform are not performed on the current block. When transform skipping is applied, quantization may be applied to the residual values ​​for which the transform is not performed.

[0627] Whether transform skipping is allowed for the current block may be determined based on at least one of the size or shape of the current block. In an example, transform skipping may be applied only when the size of the current block is less than a threshold. The threshold is related to at least one of the width, height or number of samples of the current block and may be defined as 32×32, etc. Optionally, transform skipping may be allowed only for square blocks. In an example, transform skipping may be allowed for square blocks of 32×32, 16×16, 8×8 or 4×4 sizes. Optionally, transform skipping may be allowed only when a sub-partition intra coding method is not used.

[0628] Optionally, when the sub-partition intra coding method is applied to the current block, whether to apply transform skip may be determined for each sub-partition.

[0629] Fig.38 is a diagram illustrating an example of performing determination on each subblock whether to perform transform skip.

[0630] Transform skipping may be applied to only a portion of the plurality of sub-blocks. Fig.38 In the example shown, it may be set to skip applying the transform to the sub-block at the upper position of the current block, and it may be set to not skip applying the transform to the sub-block at the lower position.

[0631] The transform type of a subblock for which transform skipping is not allowed may be determined based on information signaled in a bitstream. In an example, the transform type may be determined based on tu_mts_idx, which will be described below.

[0632] Optionally, the transform type of the sub-block may be determined based on the size of the sub-block. In an example, the horizontal transform type may be determined based on whether the width of the sub-block is equal to or greater than and / or equal to or less than a threshold, and the vertical transform type may be determined based on whether the height of the sub-block is equal to or greater than and / or equal to or less than a threshold.

[0633] After the current block is transformed by using DCT or DST, the transformed current block can be transformed again. In this regard, the transformation based on DCT or DST can be defined as a first transformation, and the block to which the first transformation is applied can be transformed again as a second transformation.

[0634] The first transform may be performed by using any one of a plurality of transform kernel candidates. In an example, the first transform may be performed by using any one of DCT2, DCT8, or DST7.

[0635] Different transform kernels may be used for the horizontal direction and the vertical direction. Information indicating the combination of the transform kernel for the horizontal direction and the transform kernel for the vertical direction may be signaled in the bitstream.

[0636] The processing unit of the first transform may be different from that of the second transform. In an example, the first transform may be performed on an 8×8 block, and the second transform may be performed on a 4×4 sized sub-block within the transformed 8×8 block. Optionally, the second transform may be performed on transform coefficients belonging to three 4×4 sized sub-blocks. The three sub-blocks may include a sub-block located at the upper left of the current block, a sub-block adjacent to the right of the sub-block, and a sub-block adjacent to the bottom of the sub-block. Optionally, the second transform may be performed on an 8×8 sized block.

[0637] The transform coefficients in the remaining region where the second transform is not performed may also be set to zero.

[0638] Alternatively, the first transform may be performed on the 4×4 block, and the second transform may be performed on a region having a size of 8×8 including the transformed 4×4 block.

[0639] Information indicating whether the second transformation is performed may be sent by a signal in the bitstream. In an example, a flag indicating whether the second transformation is performed, or index information specifying whether the second transformation is performed and a transformation core for the second transformation may be sent by a signal. In an example, when the index information is 0, it indicates that the second transformation is not performed on the current block. On the other hand, when the index information is greater than 0, the transformation core for the second transformation may be determined by the index information.

[0640] Alternatively, whether to perform the second transform may be determined based on whether the horizontal transform kernel and the vertical transform kernel are identical to each other. In one example, the second transform may be performed only when the horizontal transform kernel and the vertical transform kernel are identical to each other. Alternatively, the second transform may be performed only when the horizontal transform kernel and the vertical transform kernel are different from each other.

[0641] Optionally, the second transform may be allowed only when a predefined transform kernel is used for horizontal transform and vertical transform. In one example, the second transform may be allowed when a DCT2 transform kernel is used for horizontal transform and vertical transform. Optionally, when a sub-partition intra coding method is applied to the current block, the second transform may be allowed only when a DCT2 transform kernel is used for horizontal transform and vertical transform.

[0642] Alternatively, whether to perform the second transform may be determined based on the number of non-zero transform coefficients of the current block. In one example, when the number of non-zero transform coefficients of the current block is less than or equal to a threshold, the prediction method may be configured not to use the second transform. When the number of non-zero transform coefficients of the current block is greater than a threshold, the prediction method may be configured to use the second transform. As long as the current block is encoded using intra prediction, the prediction method may be configured to use the second transform.

[0643] Alternatively, it may be determined whether to perform the second transform based on the position of the last non-zero transform coefficient of the current block. In an example, when at least one of the x-axis coordinate or the y-axis coordinate of the last non-zero transform coefficient of the current block is greater than a threshold, or when at least one of the x-axis coordinate or the y-axis coordinate of the sub-block to which the last non-zero transform coefficient of the current block belongs is greater than a threshold, the second transform may not be performed. In this case, the threshold may be predefined in the encoding device and the decoding device. Alternatively, the threshold may be determined based on the size or shape of the current block.

[0644] Optionally, when there is only a transform coefficient of a DC component in the current block, it may be set not to perform the second transform. In this case, the DC component represents a transform coefficient at the upper left position in the current block.

[0645] Optionally, when matrix-based intra prediction is applied to the current block, it may be set not to perform the second transform.

[0646] Information indicating a transform type of a current block may be signaled in a bitstream. The information may be index information tu_mts_idx indicating one of a combination of a transform type for a horizontal direction and a transform type for a vertical direction.

[0647] Based on the transform type candidates specified by the index information tu_mts_idx, a transform kernel for a vertical direction and a transform kernel for a horizontal direction may be determined. Table 3 represents transform type combinations according to tu_mts_idx.

[0648] [Table 3]

[0649]

[0650] The transform type may be determined as one of DCT2, DST7, or DCT 8. Alternatively, transform skip may be inserted as a transform type candidate.

[0651] When Table 3 is used, when tu_mts_idx is 0, DCT2 may be applied in the horizontal direction and the vertical direction. When tu_mts_idx is 2, DCT8 may be applied in the horizontal direction, and DCT7 may be applied in the vertical direction.

[0652] When the sub-partition intra coding method is applied, the transform kernel of the sub-block can be determined independently. In an example, information for specifying the transform type combination candidate can be encoded and signaled per sub-block. Therefore, the transform kernel between sub-blocks can be different.

[0653] Optionally, the subblocks may use the same transform type. In this case, tu_mts_idx specifying the transform type combination candidate may be signaled only for the first subblock. Optionally, tu_mts_idx may be signaled at the level of the coding block, and the transform type of the subblock may be determined by referring to the tu_mts_idx signaled at the level of the coding block. Optionally, the transform type may be determined based on at least one of the size, shape, or intra prediction mode of one of the subblocks, and the determined transform type may be set for all subblocks.

[0654] Fig.39 is a diagram showing an example in which sub-blocks use the same transform type.

[0655] When the coding block is partitioned in the horizontal direction, the transform type of the sub-block (Sub-CU0) at the upper position of the coding block can be set to be the same as the transform type of the sub-block (Sub-CU1) at the lower position. Fig.39 In the example shown in (a) of FIG. 2 , when the horizontal transform type and the vertical transform type are determined based on tu_mts_idx signaled for an upper subblock, the determined transform type may also be applied to a lower subblock.

[0656] When the coding block is partitioned in the vertical direction, the transform type of the sub-block (Sub-CU0) at the left position of the coding block may be set to be the same as the transform type of the sub-block (Sub-CU1) at the right position. Fig.39 In the example shown in (b) of FIG. 2 , when the horizontal transform type and the vertical transform type are determined based on tu_mts_idx signaled for the left subblock, the determined transform type may also be applied to the right subblock.

[0657] Whether to encode the index information may be determined based on at least one of a size or shape of the current block, the number of non-zero coefficients, whether to perform a second transform, or whether to apply a sub-partition intra encoding method. In an example, when the sub-partition intra encoding method is applied to the current block, or when the number of non-zero coefficients is equal to or less than a threshold, signaling the index information may be omitted. When signaling the index information is omitted, a default transform type may be applied to the current block.

[0658] The default transform type may include at least one of DCT2 or DST7. When there are multiple default transform types, one of the multiple default transform types may be selected by considering at least one of the size, shape, or intra-frame prediction mode of the current block, whether the second transform is performed, or whether the sub-partition intra-frame coding method is applied. In an example, one of the multiple transform types may be determined as a horizontal transform type based on whether the width of the current block is within a preset range, and one of the multiple transform types may be determined as a vertical transform type based on whether the height of the current block is within a preset range. Optionally, the default mode may be determined differently according to the size, shape, or intra-frame prediction mode of the current block or whether the second transform is performed.

[0659] Optionally, when there is only a transform coefficient of a DC component in the current block, the horizontal transform type and the vertical transform type may be set to a default transform type. In an example, when there is only a transform coefficient of a DC component in the current block, the horizontal transform type and the vertical transform type may be set to DCT2.

[0660] The threshold may be determined based on the size or shape of the current block. In an example, when the size of the current block is equal to or less than 32×32, the threshold may be set to 2, and when the current block is larger than 32×32 (e.g., when the current block is a 32×64 or 64×32 size coding block), the threshold may be set to 4.

[0661] A plurality of lookup tables may be pre-stored in the encoding device / decoding device. At least one of an index value assigned to a transform type combination candidate, a type of the transform type combination candidate, or the number of transform type combination candidates may be different for each of the plurality of lookup tables.

[0662] A lookup table for the current block may be selected based on at least one of a size, a shape, or an intra prediction mode of the current block, whether a second transform is applied, or whether a transform is skipped from being applied to a neighboring block.

[0663] In an example, when the size of the current block is equal to or less than 4×4, or when the current block is encoded by inter-frame prediction, a first lookup table can be used, and when the size of the current block is greater than 4×4, or when the current block is encoded by intra-frame prediction, a second lookup table can be used.

[0664] Alternatively, information indicating one of a plurality of lookup tables may be signaled in a bitstream, and the decoding device may select a lookup table for the current block based on the information.

[0665] In another example, the index assigned to the transform type combination candidate may be adaptively determined based on at least one of the size, shape, prediction coding mode or intra prediction mode of the current block, whether the second transform is applied, or whether transform skipping is applied to a neighboring block. In the example, the index assigned to the transform skip when the size of the current block is 4×4 may be smaller than the index assigned to the transform skip when the size of the current block is greater than 4×4. Specifically, when the size of the current block is 4×4, index 0 may be assigned to the transform skip, and when the current block is greater than 4×4 and equal to or less than 16×16, an index greater than 0 (e.g., index 1) may be assigned to the transform skip. When the current block is greater than 16×16, a maximum value (e.g., 5) may be assigned to the index of the transform skip.

[0666] Alternatively, when the current block is encoded by inter prediction, an index of 0 may be assigned to the transform skip. When the current block is encoded by intra prediction, an index greater than 0 (eg, index 1) may be assigned to the transform skip.

[0667] Alternatively, when the current block is a 4×4 sized block encoded by inter-frame prediction, an index of 0 may be assigned to transform skip. On the other hand, when the current block is not encoded by inter-frame prediction, or when the current block is larger than 4×4, an index greater than 0 (e.g., index 1) may be assigned to transform skip.

[0668] Transform type combination candidates different from the transform type combination candidates enumerated in Table 3 may also be used. In an example, a transform type combination candidate consisting of a transform skip applied to one of a horizontal direction transform or a vertical direction transform and a transform kernel (such as DCT2, DCT8, or DST7, etc.) applied to the other of the horizontal direction transform or the vertical direction transform may be used. In this case, whether the transform skip is to be used as a transform type candidate in the horizontal direction or a transform type candidate in the vertical direction may be determined based on at least one of a size (e.g., width and / or height), a shape, a prediction coding mode, or an intra prediction mode of a current block.

[0669] Information indicating whether index information for determining a transform type of a current block is explicitly signaled may be signaled in a bitstream. In an example, information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for a block encoded by intra prediction and / or information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for a block encoded by inter prediction may be signaled at a sequence level.

[0670] When explicit transform type determination is allowed, the transform type of the current block may be determined based on index information tu_mts_idx signaled in the bitstream. On the other hand, when explicit transform type determination is not allowed, the transform type may be determined based on at least one of the size or shape of the current block, whether transform is allowed to be performed in units of subblocks, the position of the subblock including non-zero transform coefficients, whether the second transform is performed, or whether the sub-partition intra coding method is applied. In an example, the horizontal transform type of the current block may be determined based on the width of the current block, and the vertical transform type of the current block may be determined based on the height of the current block. For example, when the width of the current block is less than 4 or greater than 16, the horizontal transform type may be determined as DCT2. Otherwise, the horizontal transform type may be determined as DST7. When the height of the current block is less than 4 or greater than 16, the vertical transform type may be determined as DCT2. Otherwise, the vertical transform type may be determined as DST7. In this case, a threshold to be compared with the width and height may be determined based on at least one of the size, shape, or intra prediction mode of the current block to determine the horizontal transform type and the vertical transform type.

[0671] Alternatively, when the current block has a square shape with the same height and width, the horizontal direction transform type and the vertical direction transform type may be set to be the same, but when the current block has a non-square shape with a height and a width different from each other, the horizontal direction transform type and the vertical direction transform type may be set differently. In an example, when the width of the current block is greater than the height, the horizontal direction transform type may be determined as DST7, and the vertical direction transform type may be determined as DCT2. When the height of the current block is greater than the width, the vertical direction transform type may be determined as DST7, and the horizontal direction transform type may be determined as DCT2.

[0672] The number and / or type of transform type candidates or the number and / or type of transform type combination candidates may differ depending on whether explicit transform type determination is allowed. In an example, when explicit transform type determination is allowed, DCT2, DST7, and DCT8 may be used as transform type candidates. Therefore, each of the horizontal direction transform type and the vertical direction transform type may be set to DCT2, DST8, or DCT8. When explicit transform type determination is not allowed, only DCT2 and DST7 may be used as transform type candidates. Therefore, each of the horizontal direction transform type and the vertical direction transform type may be determined as DCT2 or DST7.

[0673] In the decoding device, an inverse transform of the second transform (second inverse transform) may be performed, and an inverse transform of the first transform (first inverse transform) may be performed on the result thereof. As a result of performing the second inverse transform and the first inverse transform, a residual signal of the current block may be obtained.

[0674] When transform and quantization are performed in the encoding device, the decoding device can obtain a residual block through inverse quantization and inverse transform. The reconstructed block of the current block can be obtained by adding the predicted block and the residual block in the decoding device.

[0675] When obtaining a reconstructed block of the current block, information loss occurring in the quantization and encoding process can be reduced via in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive in-loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter is referred to as a first reconstructed block, and the reconstructed block after applying the in-loop filter is referred to as a second reconstructed block.

[0676] The second reconstructed block may be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block. In this case, the SAO or ALF may be applied after the deblocking filter is applied.

[0677] The deblocking filter is used to mitigate block artifacts on block boundaries generated when quantization is performed in units of blocks. In order to apply the deblocking filter, a block strength between a first reconstructed block and adjacent reconstructed blocks may be determined.

[0678] Fig.40 is a flow chart illustrating a process for determining block strength.

[0679] exist Fig.40 In the example shown in , P represents the first reconstructed block and Q represents the adjacent reconstructed block. In this case, the adjacent reconstructed block may be adjacent to the left side or above the current block.

[0680] exist Fig.40 In the illustrated example, it is shown that block strength is determined by considering the prediction encoding mode of P and Q, whether non-zero transform coefficients are included, whether inter-frame prediction is performed by using the same reference picture, or whether the difference of the motion vector is equal to or greater than a threshold.

[0681] Whether to apply a deblocking filter may be determined based on block strength. In an example, when the block strength is 0, no filtering may be performed.

[0682] SAO is to mitigate ring artifacts generated when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the style of the first reconstructed image. Methods for determining the offset include edge offset (EO) or band offset. EO represents a method of determining the offset of the current sample according to the style of the surrounding pixels. B0 represents a method of applying a common offset to a group of pixels with similar brightness values ​​in an area. Specifically, the pixel brightness can be divided into 32 uniform parts, and pixels with similar brightness values ​​can be set as a set. In the example, 4 adjacent bands of 32 bands can be set as a group, and the same offset value can be applied to samples belonging to the 4 bands.

[0683] ALF is a method of generating a second reconstructed image by applying a filter having a predefined size / shape to a first reconstructed image or a reconstructed image to which a deblocking filter is applied. Equation 23 below represents an example of applying ALF.

[0684] [Equation 23]

[0685]

[0686] Any one of the predefined filter candidates may be selected in units of a picture, a coding tree unit, a coding block, a prediction block, or a transform block. For each of the filter candidates, any one of a size or a shape may be different.

[0687] Fig.41 Predefined filter candidates are shown.

[0688] As in Fig.41 In the example shown, at least one of the diamond shapes of 5×5, 7×7, or 9×9 sizes may be selected.

[0689] Only diamond shapes of 5×5 size are available for chroma components.

[0690] Applying the embodiments described with respect to the decoding process or the encoding process to the encoding process or the decoding process, respectively, may be included within the scope of the present disclosure. Within the scope of the present disclosure, embodiments in which operations occur in a predetermined order may be modified to embodiments in which operations occur in an order different from the predetermined order.

[0691] Although the above-mentioned embodiment is described based on a series of operations or flow charts, the embodiment does not limit the time series order of the operation of the method to this. In another example, the operation can be performed simultaneously or in a different order as needed. In addition, in the above-mentioned embodiment, each component (e.g., unit, module, etc.) constituting the block diagram can be implemented in the form of hardware devices or software. Multiple components can be combined with each other into a single component, wherein a single component can be implemented using a single hardware device or software. The above-mentioned embodiment can be implemented using program instructions that can be executed via various computer components. Instructions can be recorded in a computer-readable storage medium. The computer-readable storage medium can include program instructions, data files, data structures, etc., individually or in combination with each other. Examples of computer-readable storage media include magnetic media (such as hard disks, floppy disks, and tapes) that are specially configured to store and execute program instructions therein, optical storage media (such as CD-ROMs, DVDs), magneto-optical media (such as optical disks), and hardware devices (such as ROMs, RAMs, flash memories, etc.). Hardware devices can be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

[0692] Industrial Applicability

[0693] The present disclosure may be applied to an electronic device that encodes / decodes a video.

Claims

1. A method for decoding a video, the method comprising: Derivation of the L0 motion vector and the L1 motion vector of the current block; deriving an L0 prediction sample for a first position in the current block based on the L0 motion vector; deriving an L1 prediction sample for the first position in the current block based on the L1 motion vector; Determining whether to apply bidirectional optical flow to the current block; When it is determined to apply the bidirectional optical flow, deriving a refined motion vector for the sub-block including the first position; deriving refined prediction data for the first position based on the refined motion vector; Obtaining a prediction sample point for the first position by using the L0 prediction sample point, the L1 prediction sample point and the refined prediction data; as well as reconstructing the current block based on the prediction samples, wherein the determination of whether to apply the bidirectional optical flow to the current block is based on whether a combined prediction method is applied to the current block, the combined prediction method indicating a prediction method in which intra prediction and inter prediction are combined, and When the combined prediction method is not applied to the current block, the bidirectional optical flow is applied to the current block.

2. The method according to claim 1, wherein: Even if the inter prediction according to the combined prediction method is performed bidirectionally, the bidirectional optical flow is not applied to the current block.

3. The method according to claim 1, wherein: The refined prediction data is derived based on a first difference between an L1 horizontal direction gradient and an L0 horizontal direction gradient and a second difference between an L1 vertical direction gradient and an L0 vertical direction gradient at a first position.

4. The method according to claim 1, wherein: The determination of whether to apply the bidirectional optical flow to the current block is further based on whether a first difference in picture order counts between the current picture and the L0 reference picture is the same as a second difference in picture order counts between the L1 reference picture and the current picture, and When the first difference is the same as the second difference, the bidirectional optical flow is applied to the current block.

5. The method according to claim 1, wherein: The determination of whether to apply the bidirectional optical flow to the current block is also based on whether at least one of a width or a height of the current block is equal to or greater than a threshold.

6. A method for encoding a video, the method comprising: Derivation of the L0 motion vector and the L1 motion vector of the current block; deriving an L0 prediction sample for a first position in the current block based on the L0 motion vector; deriving an L1 prediction sample for the first position in the current block based on the L1 motion vector; Determining whether to apply bidirectional optical flow to the current block; When it is determined to apply the bidirectional optical flow, deriving a refined motion vector for the sub-block including the first position; deriving refined prediction data for the first position based on the refined motion vector; Obtaining a prediction sample point for the first position by using the L0 prediction sample point, the L1 prediction sample point and the refined prediction data; as well as The residual sample is obtained by subtracting the predicted sample from the original sample, wherein the determination of whether to apply the bidirectional optical flow to the current block is based on whether a combined prediction method is applied to the current block, the combined prediction method indicating a prediction method in which intra prediction and inter prediction are combined, and When the combined prediction method is not applied to the current block, the bidirectional optical flow is applied to the current block.

7. The method according to claim 6, wherein: Even if the inter prediction according to the combined prediction method is performed bidirectionally, the bidirectional optical flow is not applied to the current block.

8. The method according to claim 6, wherein: The refined prediction data is derived based on a first difference between an L1 horizontal direction gradient and an L0 horizontal direction gradient and a second difference between an L1 vertical direction gradient and an L0 vertical direction gradient at a first position.

9. The method according to claim 6, wherein: The determination of whether to apply the bidirectional optical flow to the current block is further based on whether a first difference in picture order counts between the current picture and the L0 reference picture is the same as a second difference in picture order counts between the L1 reference picture and the current picture, and When the first difference is the same as the second difference, the bidirectional optical flow is applied to the current block.

10. The method according to claim 6, wherein: The determination of whether to apply the bidirectional optical flow to the current block is also based on whether at least one of a width or a height of the current block is equal to or greater than a threshold.

11. A device for transmitting compressed video data, the device comprising: a processor configured to obtain the compressed video data; as well as a sending unit, configured to send the compressed video data, Wherein, obtaining the compressed video data comprises: Derivation of the L0 motion vector and the L1 motion vector of the current block; deriving an L0 prediction sample for a first position in the current block based on the L0 motion vector; deriving an L1 prediction sample for the first position in the current block based on the L1 motion vector; Determining whether to apply bidirectional optical flow to the current block; When it is determined to apply the bidirectional optical flow, deriving a refined motion vector for the sub-block including the first position; deriving refined prediction data for the first position based on the refined motion vector; Obtaining prediction samples for the first position by using the L0 prediction samples, the L1 prediction samples and the refined prediction data; and The residual sample is obtained by subtracting the predicted sample from the original sample, wherein the determination of whether to apply the bidirectional optical flow to the current block is based on whether a combined prediction method is applied to the current block, the combined prediction method indicating a prediction method in which intra prediction and inter prediction are combined, and When the combined prediction method is not applied to the current block, the bidirectional optical flow is applied to the current block.

Images