Method for encoding / decoding image signal and apparatus therefor

Abstract

Provided are a method for encoding / decoding an image signal and a device therefor. The method for decoding an image according to the present invention comprises the steps of: partitioning an encoding block into a plurality of sub-blocks in a horizontal direction or a vertical direction; determining whether to apply a second inverse transform to the encoded block; and applying a second inverse transform to at least one of the plurality of sub-blocks when it is determined that the second inverse transform is applied to the encoded block.

Description

[0001] This application is a divisional application of a patent application with an application date of June 10, 2020, application number 202080006953.2, and invention name “Method and device for encoding / decoding image signals”. Technical Field

[0002] The present disclosure relates to a video signal encoding / decoding method and a device thereof. Background Art

[0003] As display panels become larger, higher quality video services are required. The biggest problem with high-definition video services is that the amount of data has greatly increased. In order to solve the above problems, research on improving video compression rates is being actively conducted. As a representative example, the Joint Collaboration Team on Video Coding (JCT-VC) was established in 2009 by the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T). JCT-VC proposed High Efficiency Video Coding (HEVC), a video compression standard with a compression performance that is about twice that of H.264 / AVC and was approved as a standard on January 25, 2013. However, with the rapid development of high-definition video services, the performance of HEVC has gradually shown its limitations. Summary of the invention

[0004] Technical Purpose

[0005] An object of the present disclosure is to provide a method for performing a second transform in encoding / decoding a video signal and an apparatus thereof.

[0006] An object of the present disclosure is to provide a method for determining whether to apply a second transform to a coding block to which a subpartition intra coding method is applied in encoding / decoding a video signal, and an apparatus thereof.

[0007] An object of the present disclosure is to provide a method for applying a second transform to a sub-block included in a coding block in encoding / decoding a video signal, and an apparatus thereof.

[0008] Technical objectives obtainable from the present disclosure are not limited to the above-mentioned technical objectives, and other unmentioned technical objectives may be clearly understood by a person of ordinary skill in the technical field to which the present disclosure pertains from the following description.

[0009] Technical Solution

[0010] The video signal decoding method according to the present disclosure includes: partitioning a coding block into a plurality of sub-blocks in a horizontal direction or a vertical direction, determining whether to apply a second inverse transform to the coding block, and when determining to apply the second inverse transform to the coding block, applying the second inverse transform to at least one of the plurality of sub-blocks. In this case, the determination of whether to apply the second inverse transform may be based on index information signaled via a bitstream, and the index information may be signaled only when a size of at least one of the plurality of sub-blocks is equal to or greater than a threshold value.

[0011] The video signal encoding method according to the present disclosure includes: partitioning a coding block into a plurality of sub-blocks in a horizontal direction or a vertical direction, determining whether to apply a second transform to the coding block, and when it is determined that the second transform is applied to the coding block, applying the second transform to at least one of the plurality of sub-blocks. In this case, whether to encode index information specifying whether to perform the second transform is determined by comparing the size of at least one of the plurality of sub-blocks with a threshold.

[0012] In the video signal decoding method according to the present disclosure, when the size of at least one of the multiple sub-blocks is smaller than a threshold, decoding of the index information may be omitted, and when decoding of the information is omitted, it may be inferred that the value of the index information indicates that the second inverse transform is not applied.

[0013] In the video signal decoding method according to the present disclosure, the second inverse transform may be performed based on an inverse transform matrix, and the inverse transform matrix may be determined based on the intra prediction mode of the coding block and the information.

[0014] In the video signal decoding method according to the present disclosure, based on the intra prediction mode, an inverse transform matrix set may be determined, and one of a plurality of inverse transform matrix candidates included in the determined inverse transform matrix set may be selected based on the index information.

[0015] In the video signal decoding method according to the present disclosure, the inverse transform matrix set selected when the intra prediction mode is a non-directional mode may be different from the inverse transform matrix set selected when the intra prediction mode is a directional mode.

[0016] In the video signal decoding method according to the present disclosure, when the BDPCM scheme is applied to the coding block, decoding of index information may be omitted.

[0017] The features briefly summarized for the present disclosure are merely exemplary aspects of the detailed description of the present disclosure described below and do not limit the scope of the present disclosure.

[0018] Technical Effects

[0019] According to the present disclosure, encoding / decoding efficiency can be improved by performing multiple transforms on a residual signal.

[0020] According to the present disclosure, encoding / decoding efficiency may be improved by allowing a second transform to be performed on a coding block to which a sub-partition intra coding method is applied.

[0021] Effects obtainable from the present disclosure may not be limited to the above-mentioned effects, and other unmentioned effects may be clearly understood from the following description by a person of ordinary skill in the technical field to which the present disclosure belongs. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a diagram illustrating a block diagram of a video encoding device (encoder) according to an embodiment of the present disclosure.

[0023] Figure 2 is a diagram showing a block diagram of a video decoding device (decoder) according to an embodiment of the present disclosure.

[0024] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present disclosure.

[0025] Figure 4 is a diagram illustrating various partition types of a coding block.

[0026] Figure 5 is a diagram illustrating an example of aspects of partitioning a CTU.

[0027] Figure 6 is a flowchart of an inter-frame prediction method according to an embodiment of the present disclosure.

[0028] Figure 7 is a flow chart of a process for deriving motion information for a current block in merge mode.

[0029] Figure 8 is a diagram illustrating candidate blocks for deriving merge candidates.

[0030] Fig. 9 is a diagram illustrating candidate blocks for deriving merge candidates.

[0031] Fig.10 is a diagram for explaining the updating aspect of the motion information table.

[0032] Fig.11 is a diagram illustrating an updating aspect of a motion information table.

[0033] Fig.12 is a diagram showing an example in which the index of the stored motion information candidate is updated.

[0034] Fig.13 is a diagram showing the positions of representative sub-blocks.

[0035] Fig.14 is a diagram illustrating an example of performing redundancy check on only a part of merge candidates.

[0036] Fig.15 is a diagram showing an example in which a redundancy check on a specific merge candidate is omitted.

[0037] Fig.16 is a diagram illustrating an example in which a candidate block included in the same merge processing region as a current block is set to be unusable as a merge candidate.

[0038] Fig.17 is a diagram illustrating an example of deriving a merge candidate for a current block when the current block is included in a merge processing region.

[0039] Fig.18 is a diagram showing a temporary motion information table.

[0040] Fig.19 is a diagram showing an example of making the motion information table and the temporary motion information table consistent.

[0041] Fig. 20 is a flowchart of an intra-frame prediction method according to an embodiment of the present disclosure.

[0042] Fig.21 is a diagram showing an intra prediction mode.

[0043] Fig. 22 and Fig.23 is a diagram showing an example of a one-dimensional array in which reference spots are arranged in rows.

[0044] Fig.24 : is a diagram showing an angle formed by directional intra prediction modes having a straight line parallel to the x-axis.

[0045] Fig.25 is a diagram illustrating an aspect of obtaining prediction samples when a current block has a non-square shape.

[0046] Fig.26 is a diagram illustrating a wide-angle intra prediction mode.

[0047] Fig. 27 : is a diagram showing an example of partitioning in a vertical direction and partitioning in a horizontal direction.

[0048] Fig.28 is a diagram illustrating an example of determining a partition shape of a coding block.

[0049] Fig.29 is a diagram illustrating an example of determining a partition shape of a coding block.

[0050] Fig.30 is a diagram illustrating an example of determining whether to perform transform skip on a per sub-block basis.

[0051] Fig.31 is a diagram showing an example in which sub-blocks use the same transform type.

[0052] Fig.32 and 33 is a diagram illustrating application aspects of the sub-transform block coding method.

[0053] Fig.34 and 35 The horizontal direction transform type and the vertical direction transform type according to the position of the subblock as the transform target are shown.

[0054] Fig.36 is a diagram showing aspects of encoding of transform coefficients when the simplification factor is 16.

[0055] Fig.37 and Fig.38 is a diagram showing a region that is a target of the second transformation.

[0056] Fig.39 is an example showing various inseparable transformation matrix candidates. DETAILED DESCRIPTION

[0057] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0058] Image encoding and decoding are performed on a block basis. In an example, for a coding block, a transform block, or a prediction block, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, reconstruction, etc. may be performed.

[0059] Hereinafter, the encoding / decoding target block is referred to as a “current block.” In an example, the current block may refer to a coding block, a transform block, or a prediction block according to a current process of encoding / decoding.

[0060] In addition, the term "unit" used in this specification means a basic unit for performing a specific encoding / decoding process, and "block" can be understood to mean a sample array having a predetermined size. Unless otherwise specified, "block" and "unit" can be used interchangeably. In examples, in the examples described later, the coding block and the coding unit can be understood to have the same meaning as each other.

[0061] Figure 1 is a diagram showing a block diagram of an image encoding device (encoder) according to an embodiment of the present disclosure.

[0062] Reference Figure 1, the image encoding apparatus 100 may include a picture partition unit 110, prediction units 120 and 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0063] Figure 1 The components described in the figure are shown independently in order to illustrate different characteristic functions in the image encoding device, and the figure does not indicate that each component is composed of a separate hardware or a software unit. That is, each component is listed only for the convenience of explanation, and at least two of the components may constitute one component, or one component may be partitioned into multiple components that can perform its functions. Even embodiments in which the components are integrated and embodiments in which the components are partitioned are included in the scope of the present disclosure unless they depart from the spirit of the present disclosure.

[0064] In addition, some components are not necessary parts for performing the necessary functions of the present disclosure, but are optional components only for improving performance. The present disclosure can be implemented using necessary components for realizing the spirit of the present disclosure other than components only for improving performance, and a structure including only necessary components other than optional components only for improving performance is also included in the scope of the present disclosure.

[0065] The picture partition unit 110 may partition an input picture into at least one processing unit. In this regard, a processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In the picture partition unit 110, a single picture may be partitioned into a combination of a plurality of coding units, prediction units, and transform units, and the picture may be encoded by selecting a combination of coding units, prediction units, and transform units according to a predetermined condition (e.g., a cost function).

[0066] For example, a single picture may be partitioned into a plurality of coding units. In order to partition a picture into coding units, a recursive tree structure such as a quadtree structure may be used, and a coding unit derived from a root such as a single image or a maximum coding unit may be partitioned into other coding units and may have as many child nodes as the partitioned coding units. Coding units that are no longer partitioned according to certain restrictions become leaf nodes. That is, when it is assumed that only square partitions are available for a single coding unit, a single coding unit may be partitioned into up to four other coding units.

[0067] Hereinafter, in an embodiment of the present disclosure, a coding unit may be used as a unit for encoding or may be used as a unit for decoding.

[0068] The prediction units may be obtained by partitioning a single coding unit into at least one square or rectangle having the same size, or may be partitioned into prediction units in such a manner that one prediction unit may be different in shape and / or size from another prediction unit.

[0069] When a prediction unit is generated based on a coding block on which intra prediction is being performed, when the coding unit is not a minimum coding unit, intra prediction may be performed without performing partitioning into a plurality of N×N prediction units.

[0070] The prediction units 120 and 125 may include an inter-frame prediction unit 120 that performs inter-frame prediction and an intra-frame prediction unit 125 that performs intra-frame prediction. It may be determined whether inter-frame prediction or intra-frame prediction is performed on the prediction unit, and detailed information (e.g., intra-frame prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this regard, the processing unit on which the prediction is performed may be different from the processing unit in which the prediction method and its details are determined. For example, the prediction method, prediction mode, etc. may be determined based on the prediction unit, and the prediction may be performed based on the transform unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoding unit 165 using the residual value and may be sent to the decoder. When a specific encoding mode is used, the original block is encoded as it is without generating a prediction block through the prediction unit 120 or 125 and is sent to the decoding unit.

[0071] The inter-frame prediction unit 120 may predict the prediction unit based on information about at least one of a previous picture and a subsequent picture of the current picture, or in some cases, may predict the prediction unit based on information about some coding regions in the current picture. The inter-frame prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0072] The reference picture interpolation unit may receive reference picture information from the memory 155 and generate pixel information of integer pixels or smaller pixels from the reference picture. In the case of luma pixels, an 8-tap DCT-based interpolation filter with different coefficients may be used to generate pixel information about integer pixels or smaller pixels for 1 / 4 pixel units. In the case of chroma signals, a 4-tap DCT-based interpolation filter with different filter coefficients may be used to generate pixel information about integer pixels or smaller pixels for 1 / 8 pixel units.

[0073] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. As a method for calculating a motion vector, various methods may be used, such as a full search based block matching algorithm (FBMA), a three-step search (TSS) algorithm, a new three-step search (NTS) algorithm, etc. Based on the interpolated pixels, the motion vector may have a motion vector size in units of 1 / 2 pixels or 1 / 4 pixels. The motion prediction unit may predict the current prediction unit by changing the motion prediction method. As a motion prediction method, various methods may be used, such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra-block copy method, etc.

[0074] The intra prediction unit 125 may generate a prediction unit based on information about reference pixels around the current block (which is pixel information in the current picture). When the neighboring block of the current prediction unit is a block on which inter prediction is performed, and therefore the reference pixel is a pixel on which inter prediction is performed, the reference pixel included in the block on which inter prediction is performed may be replaced with information about reference pixels of the neighboring block on which intra prediction is performed. In other words, when the reference pixel is unavailable, at least one reference pixel among the available reference pixels may be used to replace the unavailable reference pixel information.

[0075] The prediction mode in the intra prediction may include a directional prediction mode using reference pixel information according to a prediction direction and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luma information may be different from the mode for predicting chroma information. In order to predict chroma information, information about the intra prediction mode for predicting luma information or information about the predicted luma signal may be used.

[0076] When performing intra prediction, when the size of the prediction unit is the same as the transform unit, intra prediction can be performed on the prediction unit based on pixels located to the left, above the left, and above the prediction unit. However, when performing intra prediction, when the size of the prediction unit is different from the transform unit, intra prediction can be performed based on the transform unit by using reference pixels. In addition, intra prediction using N×N partitions can be used only for the minimum coding unit.

[0077] In the intra prediction method, a prediction block may be generated after an adaptive intra smoothing (AIS) filter is applied to reference pixels according to a prediction mode. The type of the AIS filter applied to the reference pixels may vary. In order to perform the intra prediction method, an intra prediction mode for a current prediction unit may be predicted from intra prediction modes of prediction units existing around the current prediction unit. When the prediction mode for the current prediction unit is predicted by using mode information predicted from neighboring prediction units, when the intra prediction mode for the current prediction unit is the same as the intra prediction mode of the neighboring prediction unit, information indicating that the current prediction unit and the neighboring prediction unit have the same prediction mode may be sent by using predetermined flag information. When the prediction mode for the current prediction unit is different from the prediction mode of the neighboring prediction unit, entropy coding may be performed to encode information about the prediction mode for the current block.

[0078] Also, a residual block including information on a residual value which is a difference value between a prediction unit predicted by the prediction unit 120 or 125 and an original block of the prediction unit may be generated. The generated residual block may be input to the transform unit 130.

[0079] The transform unit 130 may perform a transform on a residual block by using a transform method such as a discrete cosine transform (DCT) or a discrete sine transform (DST), wherein the residual block includes information about a residual value between an original block and a prediction unit generated by the prediction unit 120 or 125. In this regard, the DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST in order to perform a transform on the residual block may be determined based on information about an intra prediction mode of a prediction unit used to generate the residual block. The transform for the residual block may be skipped. A flag indicating whether to skip the transform for the residual block may be encoded. Transform skipping may be allowed for a residual block whose size is less than or equal to a threshold, a residual block of a luminance component, or a residual block of a chrominance component in a 4:4:4 format.

[0080] The quantization unit 135 may perform quantization on the value transformed into the frequency domain by the transformation unit 130. The quantization coefficient may vary according to the importance of the block or image. The value calculated in the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0081] The rearrangement unit 160 may perform rearrangement on coefficient values ​​with respect to the quantized residual values.

[0082] The rearrangement unit 160 may change the coefficients in the two-dimensional block form into the coefficients in the one-dimensional vector form by a coefficient scanning method. For example, the rearrangement unit 160 may scan from the DC coefficient to the coefficients in the high frequency domain by using a zigzag scanning method so as to change the coefficients into the form of a one-dimensional vector. Depending on the size of the transform unit and the intra-frame prediction mode, vertical direction scanning or horizontal direction scanning may be used instead of zigzag scanning, wherein the coefficients in the two-dimensional block form are scanned in the column direction in the vertical direction scanning and the coefficients in the two-dimensional block form are scanned in the row direction in the horizontal direction scanning. In other words, which scanning method to use among zigzag scanning, vertical direction scanning, and horizontal direction scanning may be determined according to the size of the transform unit and the intra-frame prediction mode.

[0083] The entropy encoding unit 165 may perform entropy encoding based on the value calculated by the rearrangement unit 160. The entropy encoding may use various encoding methods, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), or context adaptive binary arithmetic coding (CABAC).

[0084] The entropy coding unit 165 may encode various types of information obtained from the rearrangement unit 160 and the prediction units 120 and 125 (such as information about residual value coefficients and information about block types of coding units, information about prediction modes, information about partition units, information about prediction units, information about partition units, information about prediction units and information about sending units, information about motion vectors, information about reference frames, information about block interpolation, filtering information, and the like).

[0085] The entropy encoding unit 165 may entropy encode coefficients of the coding unit input from the rearrangement unit 160 .

[0086] The inverse quantization unit 140 may perform inverse quantization on the value quantized in the quantization unit 135, and the inverse transform unit 145 may perform inverse transform on the value transformed in the transform unit 130. The residual value generated by the inverse quantization unit 140 and the inverse transform unit 145 may be added to the prediction unit predicted by the motion estimation unit, the motion compensation unit, or the intra prediction unit included in the prediction units 120 and 125 in order to generate a reconstructed block.

[0087] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0088] The deblocking filter can remove block distortion that occurs due to the boundaries between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it can be determined whether to apply the deblocking filter to the current block based on the pixels included in the number of rows and columns included in the block. When the deblocking filter is applied to the block, a strong filter or a weak filter is applied according to the required deblocking filter strength. In addition, when applying the deblocking filter, when horizontal direction filtering and vertical direction filtering are performed, the horizontal direction filtering and the vertical direction filtering can be configured to be processed in parallel.

[0089] The offset correction unit can correct the original image by offsetting in pixels relative to the image on which deblocking is performed. In order to perform offset correction on a specific picture, a method of applying an offset to an area determined after partitioning pixels of the image into a predetermined number of areas or a method of applying an offset according to edge information of each pixel can be used.

[0090] Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing a filtered reconstructed image with an original image. Pixels included in an image may be partitioned into predetermined groups, a filter to be applied to each of the groups may be determined, and filtering may be performed separately for each group. Information on whether ALF is applied may be sent for each coding unit (CU) of a luminance signal, and the shape and filter coefficient of the ALF filter to be applied may vary based on each block. Alternatively, an ALF filter having the same shape (fixed shape) may be applied regardless of the characteristics of the block to which the filter is applied.

[0091] In the memory 155, a reconstructed block or picture calculated by the filter unit 150 may be stored. When inter prediction is performed, the stored reconstructed block or picture may be provided to the prediction unit 120 or 125.

[0092] Figure 2 is a diagram showing a block diagram of an image decoding device (decoder) according to an embodiment of the present disclosure.

[0093] Reference Figure 2 , the image decoding apparatus 200 may include: an entropy decoding unit 210 , a rearrangement unit 215 , an inverse quantization unit 220 , an inverse transform unit 225 , prediction units 230 and 235 , a filter unit 240 , and a memory 245 .

[0094] When an image bitstream is input from an encoder, the input bitstream may be decoded according to an inverse process of the image encoding apparatus.

[0095] The entropy decoding unit 210 may perform entropy decoding according to an inverse process of entropy encoding performed by an entropy encoding unit of an image encoder. For example, in association with a method performed by an image encoder device, various methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), or context adaptive binary arithmetic coding (CABAC) may be applied.

[0096] The entropy decoding unit 210 may decode information about intra prediction and inter prediction performed by the encoder.

[0097] The rearrangement unit 215 may perform rearrangement on the bitstream entropy-decoded by the entropy decoding unit 210 based on the rearrangement method used in the encoder. Coefficients represented in the form of a one-dimensional vector may be reconstructed and rearranged into coefficients in the form of a two-dimensional block. The rearrangement unit 215 may perform rearrangement by receiving information related to coefficient scanning performed in the encoder and performing inverse scanning based on the scanning order performed in the encoder.

[0098] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter received from the encoder and the coefficient value of the rearranged block.

[0099] The inverse transform unit 225 may perform an inverse transform (i.e., inverse DCT or inverse DST) opposite to the transform (i.e., DCT or DST) performed on the quantization result by the transform unit in the image encoder. In this regard, the DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Optionally, when the transform is skipped in the image encoder, the inverse transform is not performed in the inverse transform unit 225. The inverse transform may be performed based on the transmission unit determined by the image encoder. The inverse transform unit 225 of the image decoder may selectively perform a transform method (e.g., DCT or DST) according to a plurality of pieces of information such as a prediction method, a size of a current block, a prediction direction, etc.

[0100] The prediction unit 230 or 235 may generate a prediction block based on information related to the prediction block received from the entropy decoding unit 210 and information about a previously decoded block or picture received from the memory 245 .

[0101] As described above, as an operation of the image encoder, when intra prediction is performed, when the size of the prediction unit is the same as the transform unit, intra prediction may be performed on the prediction unit based on pixels located to the left, above the left, and above the prediction unit. However, when intra prediction is performed, when the size of the prediction unit is different from the transform unit, intra prediction may be performed based on the transform unit by using reference pixels. In addition, intra prediction using N×N partitions may be used only for the minimum coding unit.

[0102] The prediction units 230 and 235 may include a PU determination module, an inter prediction unit, and an intra prediction unit. The PU determination unit may receive various types of information (such as information about the prediction unit, information about the prediction mode of the intra prediction method, information about the motion prediction of the inter prediction method, etc.) input from the entropy decoding unit 210, partition the prediction unit in the current coding unit, and determine whether to perform inter prediction or intra prediction on the prediction unit. By using the information required in the inter prediction of the current prediction unit received from the image encoder, the inter prediction unit 230 may perform inter prediction on the current prediction unit based on information about at least one of a previous picture and a subsequent picture of the current picture including the current prediction unit. Optionally, inter prediction may be performed based on information about some pre-reconstructed areas in the current picture including the current prediction unit.

[0103] In order to perform inter prediction, which method of the skip mode, the merge mode, the AMVP mode, or the intra block copy mode to use as a motion prediction method for a prediction unit included in the coding unit may be determined based on the coding unit.

[0104] The intra prediction unit 235 may generate a prediction block based on information about pixels within the current picture. When the prediction unit is a prediction unit for which intra prediction has been performed, intra prediction may be performed based on information about the intra prediction mode of the prediction unit received from the image encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation module, or a DC filter. The AIS filter may perform filtering on the reference pixels of the current block, and may determine whether to apply the filter based on the prediction mode for the current prediction unit. When AIS filtering is performed on the reference pixels of the current block, information about the AIS filter received from the image encoder and the prediction mode of the prediction unit may be used. When the prediction mode for the current block is a mode in which AIS filtering is not applied, the AIS filter may not be applied.

[0105] When the prediction mode of the prediction unit is a prediction mode in which intra prediction is performed based on a pixel value obtained by interpolating reference pixels, the reference pixel interpolation unit may interpolate the reference pixels so as to generate reference pixels having an integer or smaller unit. When the prediction mode for the current prediction unit is a prediction mode in which a prediction block is generated without interpolating reference pixels, the reference pixels may not be interpolated. When the prediction mode for the current block is a DC mode, a DC filter may generate a prediction block by filtering.

[0106] The reconstructed block or the reconstructed picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction module, and an ALF.

[0107] Information about whether a deblocking filter has been applied to a corresponding block or a corresponding picture and information about whether a strong filter or a weak filter is applied when the deblocking filter is applied may be received from the image encoder. A deblocking filter of an image decoder may receive information about the deblocking filter from the image encoder, and the image decoder may perform deblocking filtering on the corresponding block.

[0108] The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image when encoding is performed, information about the offset value, and the like.

[0109] ALF may be applied to the coding unit based on information on whether ALF is applied, information on an ALF coefficient, etc. received from the encoder. The above ALF information may be provided by being included in a specific parameter set.

[0110] In the memory 245 , the reconstructed picture or the reconstructed block may be stored so as to be used as a reference picture or a reference block, and the reconstructed picture may be provided to the output unit.

[0111] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present disclosure.

[0112] The largest coding block may be defined as a coding tree block. A single picture may be partitioned into a plurality of coding tree units (CTUs). A CTU may be a coding unit of the largest size and may be referred to as a largest coding unit (LCU). Figure 3 is a diagram illustrating an example of partitioning a single picture into a plurality of CTUs.

[0113] The size of a CTU may be defined at a picture level or a sequence level. Similarly, information indicating the size of a CTU may be signaled via a picture parameter set or a sequence parameter set.

[0114] In an example, the size of the CTU for the entire picture within the sequence may be set to 128×128. Alternatively, any one of 128×128 or 256×256 may be determined as the size of the CTU at the picture level. In an example, the CTU may be set to have a size of 128×128 in the first picture and a size of 256×256 in the second picture.

[0115] The coding block may be generated by partitioning the CTU. The coding block represents a basic unit for performing encoding / decoding. In an example, prediction or transformation may be performed for each coding block, or a prediction coding mode may be determined for each coding block. In this regard, the prediction coding mode represents a method for generating a predicted image. In an example, the prediction coding mode may include intra prediction, inter prediction, current picture reference (CPR), intra block copy (IBC), or combined prediction. For a coding block, a prediction block of the coding block may be generated by using a prediction coding mode of at least one of intra prediction, inter prediction, current picture reference, or combined prediction.

[0116] Information indicating the prediction coding mode for the current block may be signaled in a bitstream. In an example, the information may be a 1-bit flag indicating whether the prediction coding mode is an intra mode or an inter mode. When the prediction coding mode for the current block is determined to be an inter mode, current picture reference or combined prediction may be available.

[0117] The current picture reference sets the current picture as the reference picture, and obtains the prediction block of the current block from the area within the current picture that has been encoded / decoded. In this regard, the current picture refers to the picture that includes the current block. Information indicating whether the current picture reference is applied to the current block can be signaled in a bitstream. In an example, the information can be a 1-bit flag. When the flag is true, the prediction coding mode for the current block can be determined as the current picture reference, and when the flag is false, the prediction coding mode for the current block can be determined as inter-frame prediction.

[0118] Alternatively, the prediction coding mode of the current block may be determined based on the reference picture index. In an example, when the reference picture index indicates the current picture, the prediction coding mode for the current block may be determined as the current picture reference. When the reference picture index indicates a picture other than the current picture, the prediction coding mode for the current block may be determined as the inter-frame prediction. In other words, the current picture reference is a prediction method using information about an already encoded / decoded area within the current picture, and the inter-frame prediction is a prediction method using information about another already encoded / decoded picture.

[0119] Combined prediction refers to a combined coding mode that combines at least two of intra prediction, inter prediction, and current picture reference. In the example, when combined prediction is applied, a first prediction block may be generated based on any one of intra prediction, inter prediction, or current picture reference, and a second prediction block may be generated based on another one of intra prediction, inter prediction, or current picture reference. When the first prediction block and the second prediction block are generated, a final prediction block may be generated by calculating an average or weighted sum of the first prediction block and the second prediction block. Information indicating whether combined prediction is applied to the current block may be signaled in a bitstream. The information may be a 1-bit flag.

[0120] Figure 4 is a diagram illustrating various partition types of a coding block.

[0121] The coding block may be partitioned into a plurality of coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. The partitioned coding block may be partitioned again into a plurality of coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0122] Quadtree partitioning represents a method of partitioning the current block into four blocks. As a result of quadtree partitioning, the current block can be partitioned into four square partitions (refer to Figure 4 of "SPLIT_QT").

[0123] Binary tree partitioning refers to a method of partitioning a current block into two blocks. An operation of partitioning a current block into two blocks along a vertical direction (i.e., using a vertical line passing through the current block) may be referred to as a vertical binary tree partitioning, and an operation of partitioning a current block into two blocks along a horizontal direction (i.e., using a horizontal line passing through the current block) may be referred to as a horizontal binary tree partitioning. As a result of the binary tree partitioning, the current block may be partitioned into two non-square partitions. Figure 4 "SPLIT_BT_VER" is a diagram showing the result of binary tree partitioning in the vertical direction, and Figure 4 “SPLIT_BT_HOR” is a diagram showing the result of binary tree partitioning in the horizontal direction.

[0124] Ternary tree partitioning means a method of partitioning a current block into three blocks. An operation of partitioning a current block into three blocks along a vertical direction (i.e., using two vertical lines passing through the current block) may be referred to as vertical ternary tree partitioning, and an operation of partitioning a current block into three blocks along a horizontal direction (i.e., using two horizontal lines passing through the current block) may be referred to as horizontal ternary tree partitioning. As a result of the ternary tree partitioning, the current block may be partitioned into three non-square partitions. In this regard, the width / height of the partition located at the center of the current block may be twice the width / height of the other partitions. Figure 4"SPLIT_TT_VER" is a diagram showing the result of the vertical ternary tree partitioning, and Figure 4 “SPLIT_TT_HOR” is a diagram showing the result of ternary tree partitioning in the horizontal direction.

[0125] The number of partitions of a CTU may be defined as a partition depth. The maximum partition depth of a CTU may be determined at a sequence level or a picture level. Thus, the maximum partition depth of a CTU may vary based on a sequence or a picture.

[0126] Optionally, the maximum partition depth may be determined independently for each partitioning method. In an example, the maximum partition depth allowed for quadtree partitioning may be different from the maximum partition depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0127] The encoder may signal information indicating at least one of a partition type and a partition depth of the current block in a bitstream, and the decoder may determine a partition type and a partition depth of the CTU based on the information obtained by parsing the bitstream.

[0128] Figure 5 is a diagram illustrating an example of aspects of partitioning a CTU.

[0129] An operation of partitioning a coding block by using quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning may be referred to as multi-tree partitioning.

[0130] A coding block generated by partitioning a coding block by applying multi-tree partitioning may be referred to as a sub coding block. When the partition depth of the coding block is k, the partition depth of the sub coding block is set to k+1.

[0131] On the contrary, for a coding block having a partition depth of k+1, a coding block having a partition depth of k may be referred to as a parent coding block.

[0132] The partition type of the current coding block may be determined based on at least one of the partition type of the parent coding block and the partition type of the adjacent coding block. In this regard, the adjacent coding block may be a block adjacent to the current coding block, and includes at least one of an upper adjacent block, a left adjacent block, or an adjacent block adjacent to the upper left corner of the current coding block. In this regard, the partition type may include whether quadtree partitioning is applied, whether binary tree partitioning is applied, the direction of binary tree partitioning, whether ternary tree partitioning is applied, or the direction of ternary tree partitioning.

[0133] In order to determine the partition type of the coding block, information indicating whether the coding block is partitioned may be signaled in the bitstream. This information is a 1-bit flag "split_cu_flag", and when the flag is true, it may indicate that the coding block is partitioned by a multi-tree partitioning method.

[0134] When split_cu_flag is true, information indicating whether the coding block is partitioned by quadtree partitioning may be signaled in the bitstream. This information is a 1-bit flag split_qt_flag, and when this flag is true, the coding block may be partitioned into four blocks.

[0135] In the example, Figure 5 In the example shown in , the CTU is partitioned by quadtree partitioning, and thus four coding blocks with a partition depth of 1 are generated. In addition, it is shown that quadtree partitioning is applied again to the first coding block and the fourth coding block of the four coding blocks generated by quadtree partitioning. As a result, four coding blocks with a partition depth of 2 can be generated.

[0136] In addition, by applying quadtree partitioning again to the coding block with a partition depth of 2, a coding block with a partition depth of 3 may be generated.

[0137] When quadtree partitioning is not applied to a coding block, it may be determined whether to perform binary tree partitioning or ternary tree partitioning for the coding block according to at least one of the size of the coding block, whether the coding block is located at a picture boundary, the maximum partition depth, or the partition type of the neighboring block. When it is determined that binary tree partitioning or ternary tree partitioning is performed for the coding block, information indicating the partition direction may be sent with a bitstream signal. The information may be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it may be determined whether the partition direction is vertical or horizontal. In addition, information indicating which of the binary tree partitioning or ternary tree partitioning is applied to the coding block may be sent with a bitstream signal. The information may be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it may be determined whether the binary tree partitioning is applied to the coding block or the ternary tree partitioning is applied to the coding block.

[0138] In the example, Figure 5 In the example shown in , vertical binary tree partitioning is applied to a coding block with a partition depth of 1, vertical ternary tree partitioning is applied to a left coding block in the coding blocks generated by partitioning, and vertical binary tree partitioning is applied to a right coding block.

[0139] Inter prediction is a prediction coding mode that predicts a current block by using information about a previous picture. In an example, a block at the same position as the current block in the previous picture (hereinafter, a co-located block) may be set as a prediction block of the current block. Hereinafter, a prediction block generated based on the co-located block of the current block may be referred to as a co-located prediction block.

[0140] In contrast, when an object existing in a previous picture has moved to another position in a current picture, the current block can be effectively predicted by using the motion of the object. For example, when the motion direction and size of the object are determined by comparing the previous picture with the current picture, a prediction block (or a prediction image) of the current block can be generated according to the motion information of the object. Hereinafter, a prediction block generated by using motion information may be referred to as a motion prediction block.

[0141] The residual block may be generated by subtracting the prediction block from the current block. In this regard, in the case where the object moves, the energy of the residual block may be reduced by using the motion prediction block instead of using the co-located prediction block, and thus the compression performance of the residual block may be improved.

[0142] As above, the operation of generating a prediction block by using motion information may be referred to as motion estimation prediction. In most inter predictions, a prediction block may be generated based on motion compensation prediction.

[0143] The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, and a bidirectional weighting factor index. The motion vector represents the motion direction and size of an object. The reference picture index specifies a reference picture of a current block among reference pictures included in a reference picture list. The prediction direction indicates any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of L0 direction motion information and L1 direction motion information may be used according to the prediction direction of the current block. The bidirectional weighting factor index specifies a weighting factor applied to an L0 prediction block and a weighting factor applied to an L1 prediction block.

[0144] Figure 6 is a flowchart of an inter-frame prediction method according to an embodiment of the present disclosure.

[0145] Reference Figure 6 The inter-frame prediction method includes: determining an inter-frame prediction mode for a current block (S601), obtaining motion information of the current block according to the determined inter-frame prediction mode (S602), and performing motion compensation prediction for the current block based on the obtained motion information (S603).

[0146] In this regard, the inter prediction mode may refer to various methods for determining motion information of the current block, and include an inter prediction mode using translational motion information, an inter prediction mode using affine motion information. In an example, the inter prediction mode using translational motion information may include a merge mode and a motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. The motion information about the current block may be determined based on a neighboring block adjacent to the current block or information obtained by parsing a bitstream.

[0147] The motion information of the current block may be derived from the motion information of another block. In this regard, the other block may be a block that is encoded / decoded by inter-frame prediction before the current block. An operation of setting the motion information of the current block to be the same as the motion information of the other block may be defined as a merge mode. In addition, an operation of setting the motion vector of the other block to a predicted value of the motion vector of the current block may be defined as a motion vector prediction mode.

[0148] Figure 7 is a flow chart of a process for deriving motion information for a current block in merge mode.

[0149] A merge candidate of a current block may be derived (S701). The merge candidate of the current block may be derived from a block encoded / decoded by inter prediction before the current block.

[0150] Figure 8 is a diagram illustrating candidate blocks for deriving merge candidates.

[0151] The candidate block may include at least one of a neighboring block or a non-neighboring block, wherein the neighboring block includes samples adjacent to the current block and the non-neighboring block includes samples not adjacent to the current block. Hereinafter, samples determining the candidate block are defined as basic samples. In addition, basic samples adjacent to the current block are referred to as neighboring basic samples, and basic samples not adjacent to the current block are referred to as non-neighboring basic samples.

[0152] The neighboring basic sample points may be included in the neighboring columns of the leftmost column of the current block or the neighboring rows of the uppermost row of the current block. In an example, when the coordinates of the upper left sample point of the current block are (0,0), at least one of the blocks including the basic sample points at positions (-1,H-1), (W-1,-1), (W,-1), (-1,H) or (-1,-1) may be used as a candidate block. Referring to the diagram, neighboring blocks of indexes 0 to 4 may be used as candidate blocks.

[0153] The non-neighboring basic sample points represent samples whose at least one of the x-axis distance or the y-axis distance from the basic sample points adjacent to the current block has a predefined value. In an example, at least one of a block including a basic sample point whose x-axis distance from the left basic sample point is a predefined value, a block including a non-neighboring sample point whose y-axis distance from the upper basic sample point is a predefined value, or a block including a non-neighboring sample point whose x-axis distance and y-axis distance from the upper left basic sample point are predefined values ​​may be used as a candidate block. The predefined value may be a natural number such as 4, 8, 12, 16, etc. Referring to the accompanying drawings, at least one of the blocks of indexes 5 to 26 may be used as a candidate block.

[0154] Sample points that are not located on the same vertical line, horizontal line, or diagonal line as adjacent basic sample points may be set as non-adjacent basic sample points.

[0155] Hereinafter, a candidate block including neighboring basic samples among the candidate blocks is referred to as a neighboring block, and a block including non-neighboring basic samples is referred to as a non-neighboring block.

[0156] When the distance between the current block and the candidate block is equal to or greater than a threshold, the candidate block may be set to be unavailable as a merge candidate. The threshold may be determined based on the size of the coding tree unit. In an example, the threshold may be set to the height of the coding tree unit (ctu_height) or to a value that adds an offset to the height of the coding tree unit or to a value that subtracts an offset from the height of the coding tree unit (e.g., ctu_height±N). As a predefined value in the encoder and decoder, the offset N may be set to 4, 8, 16, 32, or ctu_height.

[0157] When a difference between the y-axis coordinate of the current block and the y-axis coordinate of a sample included in the candidate block is greater than a threshold value, it may be determined that the candidate block is not usable as a merge candidate.

[0158] Optionally, a candidate block that does not belong to the same coding tree unit as the current block may be set to be unavailable as a merge candidate. In an example, when the base sample is outside the upper boundary of the coding tree unit to which the current block belongs, the candidate block including the base sample may be set to be unavailable as a merge candidate.

[0159] If the upper boundary of the current block is close to the upper boundary of the coding tree unit, it can be determined that multiple candidate blocks are not available as merge candidates to reduce the encoding / decoding efficiency of the current block. In order to solve such a problem, the candidate blocks can be set so that the number of candidate blocks at the left position of the current block is greater than the number of candidate blocks at the upper position of the current block.

[0160] Fig. 9 is a diagram illustrating candidate blocks for deriving merge candidates.

[0161] As in Fig. 9 In the example shown, the upper blocks of the N upper block columns belonging to the current block and the left blocks of the M left block columns belonging to the current block can be set as candidate blocks. In this case, the number of left candidate blocks can be set to be greater than the number of upper candidate blocks by setting M greater than N.

[0162] In an example, the difference between the y-axis coordinates of the basic sample points in the current block and the y-axis coordinates of the upper block that can be used as the candidate block can be set to not exceed N times the height of the current block. In addition, the difference between the x-axis coordinates of the basic sample points in the current block and the x-axis coordinates of the left block that can be used as the candidate block can be set to not exceed M times the width of the current block.

[0163] In the example, Fig. 9In the example shown, blocks belonging to two upper block columns of the current block and blocks belonging to five left block columns of the current block are set as candidate blocks.

[0164] The merge candidate may be derived from a temporally adjacent block different from the current block included in the picture. In an example, the merge candidate may be derived from a co-located block included in a co-located picture. Any one of the reference pictures included in the reference picture list may be set as the co-located picture. Index information identifying the co-located picture among the reference pictures may be signaled in a bitstream. Alternatively, a reference picture having a predefined index among the reference pictures may be determined as the co-located picture.

[0165] The motion information of the merge candidate may be set to be the same as the motion information of the candidate block. In an example, at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index of the candidate block may be set to the motion information of the merge candidate.

[0166] A merge candidate list including merge candidates may be generated ( S702 ).

[0167] The indexes of the merge candidates in the merge candidate list may be assigned according to a predetermined order. In an example, the indexes may be assigned in the order of a merge candidate derived from a left neighboring block, a merge candidate derived from an upper neighboring block, a merge candidate derived from an upper right neighboring block, a merge candidate derived from a lower left neighboring block, a merge candidate derived from an upper left neighboring block, and a merge candidate derived from a temporal neighboring block.

[0168] When multiple merge candidates are included in the merge candidate list, at least one merge candidate among the multiple merge candidates may be selected (S703). Specifically, information for specifying any one of the multiple merge candidates may be sent as a bitstream signal. In the example, information merge_idx indicating an index of any one of the merge candidates included in the merge candidate list may be sent as a bitstream signal.

[0169] When the number of merge candidates included in the merge candidate list is less than a threshold value, the motion information candidate included in the motion information table may be added to the merge candidate list as a merge candidate. In this regard, the threshold value may be the maximum number of merge candidates that may be included in the merge candidate list or a value minus an offset from the maximum number of merge candidates. The offset may be a natural number such as 1 or 2.

[0170] The motion information table includes motion information candidates derived from a block encoded / decoded based on inter-frame prediction in a current picture. In an example, motion information of the motion information candidate included in the motion information table may be set to be the same as the motion information of the block encoded / decoded based on inter-frame prediction. In this regard, the motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index.

[0171] The motion information candidates included in the motion information table may also be referred to as inter region merging candidates or prediction region merging candidates.

[0172] The maximum number of motion information candidates that can be included in the motion information table can be predefined in the encoder and the decoder. In an example, the maximum number of motion information candidates that can be included in the motion information table can be 1, 2, 3, 4, 5, 6, 7, 8 or more (e.g., 16).

[0173] Optionally, information indicating a maximum number of motion information candidates that can be included in the motion information table may be signaled in the bitstream. The information may be signaled at a sequence level, a picture level, or a slice level. The information may indicate a maximum number of motion information candidates that can be included in the motion information table. Optionally, the information may indicate a difference between a maximum number of motion information candidates that can be included in the motion information table and a maximum number of merge candidates that can be included in the merge candidate list.

[0174] Alternatively, the maximum number of motion information candidates that can be included in the motion information table may be determined according to a picture size, a slice size, or a coding tree unit size.

[0175] The motion information table may be initialized in units of a picture, a slice, a tile, a brick, a coding tree unit, or a coding tree unit line (row or column). In an example, when a slice is initialized, the motion information table is also initialized, so the motion information table may not include any motion information candidates.

[0176] Optionally, information indicating whether the motion information table is to be initialized may be signaled in the bitstream. The information may be signaled at a slice level, a tile level, a partition level, or a block level. A preconfigured motion information table may be used until the information indicates initialization of the motion information table.

[0177] Alternatively, information about the initial motion information candidate may be signaled in a picture parameter set or a slice header. Although the slice is initialized, the motion information table may include the initial motion information candidate. Therefore, the initial motion information candidate may be used for a block that is the first encoding / decoding target in the slice.

[0178] Optionally, the motion information candidate included in the motion information table of the previous coding tree unit may be set as the initial motion information candidate. In an example, the motion information candidate with the smallest index or the largest index among the motion information candidates included in the motion information table of the previous coding tree unit may be set as the initial motion information candidate.

[0179] The blocks are encoded / decoded in a coding / decoding order, and the blocks encoded / decoded based on the inter prediction may be sequentially set as motion information candidates in the coding / decoding order.

[0180] Fig.10 is a diagram for explaining the updating aspect of the motion information table.

[0181] For the current block, when inter prediction is performed (S1001), a motion information candidate may be derived based on the current block (S1002). The motion information of the motion information candidate may be set to be the same as the motion information of the current block.

[0182] When the motion information table is empty ( S1003 ), the motion information candidate derived based on the current block may be added to the motion information table ( S1004 ).

[0183] When the motion information table already includes motion information candidates (S1003), a redundancy check for the motion information of the current block (or the motion information candidates derived based on the current block) may be performed (S1005). The redundancy check is to determine whether the motion information of the pre-stored motion information candidates in the motion information table is the same as the motion information of the current block. The redundancy check may be performed for all pre-stored motion information candidates in the motion information table. Optionally, the redundancy check may be performed for motion information candidates whose indexes among the pre-stored motion information candidates in the motion information table exceed or are lower than a threshold. Optionally, the redundancy check may be performed for a predefined number of motion information candidates. In an example, two motion information candidates with the smallest index or with the largest index may be determined as targets for the redundancy check.

[0184] When a motion information candidate having the same motion information as the current block is not included, a motion information candidate derived based on the current block may be added to the motion information table (S1008). Whether the motion information candidates are the same may be determined based on whether the motion information (e.g., motion vector / reference picture index, etc.) of the motion information candidates are the same.

[0185] In this regard, when the maximum number of motion information candidates has been stored in the motion information table (S1006), the oldest motion information candidate may be deleted (S1007), and the motion information candidate derived based on the current block may be added to the motion information table (S1008). In this regard, the oldest motion information candidate may be a motion information candidate having a maximum index or a minimum index.

[0186] The motion information candidates may be identified by corresponding indices. When a motion information candidate derived from the current block is added to the motion information table, a minimum index (e.g., 0) may be assigned to the motion information candidate, and the index of the pre-stored motion information candidate may be increased by 1. In this regard, when the maximum number of motion information candidates has been stored in the motion information table, the motion information candidate having the largest index is removed.

[0187] Optionally, when adding a motion information candidate derived from the current block to the motion information table, the maximum index may be assigned to the motion information candidate. In an example, when the number of pre-stored motion information candidates in the motion information table is less than the maximum value, an index having the same value as the number of pre-stored motion information candidates may be assigned to the motion information candidate. Optionally, when the number of pre-stored motion information candidates in the motion information table is equal to the maximum value, an index obtained by subtracting 1 from the maximum value may be assigned to the motion information candidate. Optionally, the motion information candidate having the smallest index is removed, and the indexes of the remaining pre-stored motion information candidates are reduced by 1.

[0188] Fig.11 is a diagram illustrating an updating aspect of a motion information table.

[0189] It is assumed that when a motion information candidate derived from the current block is added to the motion information table, a maximum index is assigned to the motion information candidate. In addition, it is assumed that a maximum number of motion information candidates have been stored in the motion information table.

[0190] When the motion information candidate HmvpCand[n+1] derived from the current block is added to the motion information table HmvpCandList, the motion information candidate HmvpCand[0] having the smallest index among the pre-stored motion information candidates may be deleted, and the indexes of the remaining motion information candidates may be reduced by 1. In addition, the index of the motion information candidate HmvpCand[n+1] derived from the current block may be set to the maximum value (for Fig.14 The example shown in , n).

[0191] When the same motion information candidate as the motion information candidate derived based on the current block is pre-stored ( S1005 ), the motion information candidate derived based on the current block may not be added to the motion information table ( S1009 ).

[0192] Optionally, when adding a motion information candidate derived based on the current block to the motion information table, the pre-stored motion information candidate identical to the motion information candidate may be removed. In this case, the same effect as when the index of the pre-stored motion information candidate is re-updated is produced.

[0193] Fig.12 is a diagram showing an example in which indexes of pre-stored motion information candidates are updated.

[0194] When the index of the pre-stored motion information candidate that is the same as the motion information candidate mvCand derived from the current block is hIdx, the pre-stored motion information candidate may be removed, and the index of the motion information candidate with an index greater than hIdx may be reduced by 1. In an example, Fig.15 The example shown in shows that HmvpCand[2] identical to mvCand is deleted in the motion information table HvmpCandList, and the indexes from HmvpCand[3] to HmvpCand[n] are decreased by 1.

[0195] Also, the motion information candidate mvCand derived based on the current block may be added to the end of the motion information table.

[0196] Alternatively, an index assigned to the same pre-stored motion information candidate as the motion information candidate derived based on the current block may be updated. For example, the index of the pre-stored motion information candidate may be changed to a minimum value or a maximum value.

[0197] The motion information of the blocks included in the predetermined area may be set not to be added to the motion information table. In an example, the motion information candidate derived based on the motion information of the blocks included in the merge processing area may not be added to the motion information table. Since the encoding / decoding order for the plurality of blocks included in the merge processing area is not defined, it is not appropriate to use the motion information of any one of the plurality of blocks for inter-frame prediction of another block of the plurality of blocks. Therefore, the motion information candidate derived based on the blocks included in the merge processing area may not be added to the motion information table.

[0198] Optionally, the motion information of a block smaller than a preset size may be set not to be added to the motion information table. In an example, the motion information candidate derived based on the motion information of a coding block with a width or height less than 4 or 8 or the motion information of a coding block of 4×4 size may not be added to the motion information table.

[0199] When motion compensation prediction is performed in units of subblocks, a motion information candidate may be derived based on motion information of a representative subblock among a plurality of subblocks included in the current block. In an example, when a subblock merge candidate is used for the current block, a motion information candidate may be derived based on motion information of a representative subblock among the subblocks.

[0200] The motion vector of the sub-block may be derived in the following order. First, one of the merge candidates included in the merge candidate list of the current block may be selected, and an initial shift vector (shVector) may be derived based on the motion vector of the selected merge candidate. And, a shifted sub-block whose base sample position is (xColSb, yColSb) may be derived by adding the initial shift vector to the position (xSb, ySb) of the base sample (e.g., the upper left sample or the center sample) of each sub-block in the coding block. The following equation 1 shows an equation for deriving a shifted sub-block.

[0201] [Equation 1]

[0202] (xColSb, yColSb) = (xSb+shVector[0]>>4, ySb+shVector[1]>>4)

[0203] Then, the motion vector of the co-located block corresponding to the center position of the sub-block including (xColSb, yColSb) may be set as the motion vector of the sub-block including (xSb, ySb).

[0204] The representative sub-block may mean a sub-block including an upper left sample point, a center sample point, a lower right sample point, an upper right sample point, or a lower left sample point of the current block.

[0205] Fig.13 is a diagram showing the positions of representative sub-blocks.

[0206] Fig.13 (a) represents an example of setting a sub-block at the upper left position of the current block as a representative sub-block, and Fig.13 (b) represents an example of setting a subblock at the center position of the current block as a representative subblock. When motion compensation prediction is performed in units of subblocks, a motion information candidate of the current block may be derived based on a motion vector of a subblock including a top left sample of the current block or a subblock including a center sample of the current block.

[0207] Based on the inter prediction mode of the current block, it may be determined whether the current block is to be used as a motion information candidate. In an example, a block encoded / decoded based on an affine motion model may be set to be unusable as a motion information candidate. Therefore, although the current block is encoded / decoded by inter prediction, when the inter prediction mode of the current block is an affine prediction mode, the motion information table may not be updated based on the current block.

[0208] Alternatively, whether the current block is to be used as a motion information candidate may be determined based on at least one of a motion vector resolution of the current block, whether a merge offset encoding method is applied, whether a combined prediction is applied, or whether a triangular partition is applied. In an example, for at least one of a case where the motion information resolution of the current block is equal to or greater than 2 integer pixels, a case where combined prediction is applied to the current block, a case where triangular partition is applied to the current block, or a case where a merge offset encoding method is applied to the current block, the current block may be set to be unusable as a motion information candidate.

[0209] Alternatively, the motion information candidate may be derived based on a sub-block vector of at least one of the sub-blocks included in the block encoded / decoded based on the affine motion model. In an example, the motion information candidate may be derived by using a sub-block at an upper left position, a sub-block at a center position, or a sub-block at an upper right position of the current block. Alternatively, an average value of sub-block vectors of a plurality of sub-blocks may be set as a motion vector of the motion information candidate.

[0210] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, the motion information candidate included in the motion information table may be added to the merge candidate list as a merge candidate. The additional processing is performed in an order reflecting the sorting of the indexes of the motion information candidates in ascending or descending order. In an example, the motion information candidate with the largest index may be added to the merge candidate list of the current block first.

[0211] When a motion information candidate included in the motion information table is added to a merge candidate list, a redundancy check between the motion information candidate and a pre-stored merge candidate in the merge candidate list may be performed. As a result of the redundancy check, a motion information candidate having the same motion information as the pre-stored merge candidate may not be added to the merge candidate list.

[0212] A redundancy check may be performed only for a portion of the motion information candidates included in the motion information table. In an example, a redundancy check may be performed only for motion information candidates whose indexes exceed or are below a threshold. Alternatively, a redundancy check may be performed only for N motion information candidates having a maximum index or a minimum index. Alternatively, a redundancy check may be performed only for a portion of the pre-stored merge candidates in a merge candidate list. In an example, a redundancy check may be performed only for merge candidates whose indexes exceed or are below a threshold or for merge candidates derived from a block at a specific position. In this regard, the specific position may include at least one of a left neighboring block, an upper neighboring block, an upper right neighboring block, or a lower left neighboring block of the current block.

[0213] Fig.14 is a diagram showing an example in which a redundancy check is performed only on a part of merging candidates.

[0214] When the motion information candidate HmvpCand[j] is added to the merge candidate list, a redundancy check of 2 merge candidates mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1] having the largest index may be performed for the motion information candidate. In this regard, NumMerge may show the number of available spatial merge candidates and temporal merge candidates.

[0215] Unlike the example shown, when the motion information candidate HmvpCand[j] is added to the merge candidate list, a redundancy check of the two merge candidates with the smallest index may be performed for the motion information candidate. For example, it may be checked whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j].

[0216] Optionally, a redundancy check may be performed only for merge candidates derived from a specific position. In an example, a redundancy check may be performed for at least one of merge candidates derived from a neighboring block located to the left of the current block or above the current block. When there is no merge candidate derived from a specific position in the merge candidate list, the motion information candidate may be added to the merge candidate list without performing a redundancy check.

[0217] When the motion information candidate HmvpCand[j] is added to the merge candidate list, a redundancy check of 2 merge candidates mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1] having the largest index may be performed for the motion information candidate. In this regard, NumMerge may show the number of available spatial merge candidates and temporal merge candidates.

[0218] Redundancy check on merge candidates may be performed only for a portion of motion information candidates among motion information candidates. In an example, redundancy check may be performed only for N motion information candidates having a large index or a minimum index among motion information candidates included in a motion information table. In an example, redundancy check may be performed only for motion information candidates having an index whose sum difference of numbers of motion information candidates included in the motion information table is lower than a threshold. When the threshold is 2, redundancy check may be performed only for 3 motion information candidates having the largest index value among motion information candidates included in the motion information table. Redundancy check may be omitted for motion information candidates other than the above 3 motion information candidates. When redundancy check is omitted, the motion information candidate may be added to the merge candidate list regardless of whether there is motion information identical to the merge candidate.

[0219] In contrast, the redundancy check is set to be performed only for the motion information candidates having the index whose sum of the numbers of the motion information candidates included in the motion information table exceeds the threshold value.

[0220] The number of motion information candidates for performing redundancy check may be redefined in the encoder and the decoder. In an example, the threshold may be an integer such as 0, 1 or 2.

[0221] Alternatively, the threshold may be determined based on at least one of the number of merge candidates included in the merge candidate list or the number of motion information candidates included in the motion information table.

[0222] When a merge candidate identical to the first motion information candidate is found, redundant checking of the merge candidate identical to the first motion information candidate may be omitted in redundant checking for the second motion information candidate.

[0223] Fig.15 is a diagram illustrating an example of omitting a redundant check on a specific merge candidate.

[0224] When a motion information candidate HmvpCand[i] with an index of i is added to a merge candidate list, a redundancy check is performed between the motion information candidate and a pre-stored merge candidate in the merge candidate list. In this regard, when a merge candidate mergeCandlist[j] identical to the motion information candidate HmvpCand[i] is found, a redundancy check may be performed between the motion information candidate HmvpCand[i-1] with an index of i-1 and the merge candidate without adding the motion information candidate HmvpCand[i] to the merge candidate list. In this regard, a redundancy check may be omitted between the motion information candidate HmvpCand[i-1] and the merge candidate mergeCandList[j].

[0225] In the example, Fig.15 In the example shown in , it is determined that HmvpCand[i] and mergeCandList[2] are the same. Therefore, a redundancy check for HmvpCand[i-1] may be performed without adding HmvpCand[i] to the merge candidate list. In this regard, a redundancy check between HmvpCand[i-1] and mergeCandList[2] may be omitted.

[0226] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, at least one of a paired merge candidate or a zero merge candidate may be further included in addition to the motion information candidate. The paired merge candidate indicates a merge candidate having a value obtained by averaging motion vectors of more than 2 merge candidates as a motion vector, and the zero merge candidate indicates a merge candidate having a motion vector of 0.

[0227] For the merge candidate list of the current block, merge candidates may be added in the following order.

[0228] Spatial merging candidate - temporal merging candidate - motion information candidate - (affine motion information candidate) - pairwise merging candidate - zero merging candidate.

[0229] The spatial merge candidate indicates a merge candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal merge candidate indicates a merge candidate derived from a previous reference picture. The affine motion information candidate indicates a motion information candidate derived from a block encoded / decoded by an affine motion model.

[0230] The motion information table may also be used in the motion vector prediction mode. In an example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold value, the motion information candidate included in the motion information table may be set as a motion vector prediction candidate for the current block. Specifically, the motion vector of the motion information candidate may be set as a motion vector prediction candidate.

[0231] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate may be set as a motion vector predictor of the current block. Then, after decoding the motion vector residual value of the current block, the motion vector of the current block may be obtained by adding the motion vector predictor and the motion vector residual value.

[0232] The motion vector prediction candidate list of the current block may be configured in the following order.

[0233] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Motion information candidate - (affine motion information candidate) - Zero motion vector prediction candidate

[0234] The spatial motion vector prediction candidate represents a motion vector prediction candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal motion vector prediction candidate represents a motion vector prediction candidate derived from a previous reference picture. The affine motion information candidate represents a motion information candidate derived from a block encoded / decoded by an affine motion model. The zero motion vector prediction candidate represents a candidate whose motion vector value is 0.

[0235] A merge processing area larger than a coding block may be defined. The coding blocks included in the merge processing area may be processed in parallel without being encoded / decoded sequentially. In this regard, not being encoded / decoded sequentially means that the order of encoding / decoding is not defined. Therefore, the encoding / decoding processing of the blocks included in the merge processing area may be processed independently. Optionally, the blocks included in the merge processing area may share merge candidates. In this regard, merge candidates may be derived based on the merge processing area.

[0236] According to the above features, the merged processing region may be referred to as a parallel processing region, a shared merge region (SMR), or a merged estimation region (MER).

[0237] A merge candidate of the current block may be derived based on the coding block. However, when the current block is included in a merge processing region larger than the current block, a candidate block included in the same merge processing region as the current block may be set to be unavailable as a merge candidate.

[0238] Fig.16 is a diagram illustrating an example in which a candidate block included in the same merge processing region as a current block is set to be unusable as a merge candidate.

[0239] exist Fig.16 In the example shown in the left figure of , in the decoding / decoding of CU5, a block including basic samples adjacent to CU5 may be set as a candidate block. In this regard, candidate blocks x3 and x4 included in the same merge processing region as CU5 may be set to be unusable as merge candidates for CU5. However, candidate blocks x0, x1, and x2 not included in the same merge processing region as CU5 may be set to be usable as merge candidates.

[0240] exist Fig.16 In the example shown in the right figure of , in the decoding / decoding of CU8, blocks including basic samples adjacent to CU8 may be set as candidate blocks. In this regard, candidate blocks x6, x7, and x8 included in the same merge processing region as CU8 may be set to be unusable as merge candidates. However, candidate blocks x5 and x9 not included in the same merge processing region as CU8 may be set to be usable as merge candidates.

[0241] Alternatively, when the current block is included in the merge processing region, neighboring blocks adjacent to the current block and the merge processing region may be set as candidate blocks.

[0242] Fig.17 is a diagram illustrating an example of deriving a merge candidate for a current block when the current block is included in a merge processing region.

[0243] As in Fig.17In the example shown in (a), a neighboring block adjacent to the current block may be set as a candidate block for deriving a merge candidate for the current block. In this regard, a candidate block included in the same merge processing region as the current block may be set to be unusable as a merge candidate. In the example, when deriving a merge candidate for the coding block CU3, the upper neighboring block y3 and the upper right neighboring block y4 included in the same merge processing region as the coding block CU3 may be set to be unusable as a merge candidate for the coding block CU3.

[0244] By scanning neighboring blocks adjacent to the current block in a predefined order, a merge candidate may be derived. In an example, the predefined order may be the order of y1, y3, y4, y0, and y2.

[0245] When the number of merge candidates that can be derived from neighboring blocks adjacent to the current block is less than the value of the maximum number of merge candidates minus the offset or the maximum number, the Fig.17 The example shown in (b) derives a merge candidate for the current block by using a neighboring block adjacent to the merge processing region. In the example, a neighboring block adjacent to the merge processing region including the coding block CU3 may be set as a candidate block for the coding block CU3. In this regard, the neighboring block adjacent to the merge processing region may include at least one of a left neighboring block x1, an upper neighboring block x3, a lower left neighboring block x0, an upper right neighboring block x4, or an upper left neighboring block x2.

[0246] By scanning neighboring blocks adjacent to the merge processing area in a predefined order, a merge candidate can be derived. In an example, the predefined order can be the order of x1, x3, x4, x0 and x2.

[0247] In summary, a merge candidate for the coding block CU3 included in the merge processing area can be derived by scanning the candidate blocks in the following scanning order.

[0248] (y1,y3,y4,y0,y2,x1,x3,x4,x0,x2)

[0249] However, the scanning order of the candidate blocks shown above only shows an example of the present disclosure, and the candidate blocks may be scanned in an order different from the above example. Optionally, the scanning order may be adaptively determined based on at least one of the size or shape of the current block or the merge processing area.

[0250] The merged processing area may be square or non-square. Information for determining the merged processing area may be sent as a bitstream signal. The information may include at least one of information indicating the shape of the merged processing area or information indicating the size of the merged processing area. When the merged processing area is non-square, at least one of information indicating the size of the merged processing area, information indicating the width or height of the merged processing area, or information indicating the aspect ratio of the merged processing area may be sent as a bitstream signal.

[0251] The size of the merge processing area may be determined based on at least one of information signaled in a bitstream, a picture resolution, a size of a slice, or a size of a tile.

[0252] If motion compensation prediction is performed on a block included in the merge processing region, motion information candidates derived based on motion information of the block on which motion compensation prediction is performed may be added to the motion information table.

[0253] However, if the motion information candidate derived from the block included in the merge processing area is added to the motion information table, a situation may occur where the motion information candidate derived from the block is used in encoding / decoding of other blocks in the merge processing area whose encoding / decoding is actually slower than that of the block. In other words, although the dependency between blocks should be excluded in encoding / decoding of the block included in the merge processing area, a situation may occur where motion prediction compensation is performed by using motion information of other blocks included in the merge processing area. In order to solve such a problem, although encoding / decoding of the block included in the merge processing area is completed, the motion information of the block for which encoding / decoding is completed may not be added to the motion information table.

[0254] Alternatively, the motion information table may be updated using only blocks at predefined positions within the merge processing area. Examples of the predefined position may include at least one of a block located at the upper left of the merge processing area, a block located at the upper right of the merge processing area, a block located at the lower left of the merge processing area, a block located at the lower right of the merge processing area, a block located at the center of the merge processing area, a block adjacent to the right border of the merge processing area, and a block adjacent to the lower border of the merge processing area. As an example, the motion information table may be updated using only the motion information of a block adjacent to the lower right corner of the merge processing area, and the motion information table may not be updated using the motion information of other blocks.

[0255] Alternatively, after decoding of all blocks included in the merge processing area is completed, motion information candidates derived from the blocks may be added to the motion information table. That is, although the blocks included in the merge processing area are encoded / decoded, the motion information table may not be updated.

[0256] In an example, if motion compensation prediction is performed for a block included in a merge processing area, motion information candidates derived from the block may be added to a motion information table in a predefined order. In this regard, the predefined order may be determined according to a scanning order of coding blocks in a merge processing area or a coding tree unit. The scanning order may be at least one of a raster scan, a horizontal scan, a vertical scan, or a zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0257] Alternatively, a motion information candidate including unidirectional motion information may be added to the motion information table before a motion information candidate including bidirectional motion information. Conversely, a motion information candidate including bidirectional motion information may be added to the motion information table before a motion information candidate including unidirectional motion information.

[0258] Alternatively, the motion information candidates may be added to the motion information table in order of high usage frequency or low usage frequency in the merge processing region or the coding tree unit.

[0259] When the current block is included in the merge processing region and the number of merge candidates included in the merge candidate list of the current block is less than the maximum number, the motion information candidate included in the motion information table may be added to the merge candidate list. In this regard, the motion information candidate derived from the block included in the same merge processing region as the current block may be set not to be added to the merge candidate list of the current block.

[0260] Optionally, when the current block is included in the merge processing region, it may be set not to use the motion information candidates included in the motion information table. In other words, although the number of merge candidates included in the merge candidate list of the current block is less than the maximum number, the motion information candidates included in the motion information table may not be added to the merge candidate list.

[0261] In another example, a motion information table for a merge processing region or a coding tree unit may be configured. This motion information table plays a role of temporarily storing motion information of blocks included in the merge processing region. In order to distinguish a general motion information table from a motion information table for a merge processing region or a coding tree unit, the motion information table for the merge processing region or the coding tree unit is referred to as a temporary motion information table. Furthermore, the motion information candidates stored in the temporary motion information table are referred to as temporary motion information candidates.

[0262] Fig.18 is a diagram showing a temporary motion information table.

[0263] A temporary motion information table for a coding tree unit or a merge processing area may be configured. When motion compensation prediction is performed on a current block included in a coding tree unit or a merge processing area, the motion information of the block may not be added to the motion information table HmvpCandList. Alternatively, a temporary motion information candidate derived from the block may be added to the temporary motion information table HmvpMERCandList. In other words, the temporary motion information candidate added to the temporary motion information table may not be added to the motion information table. Therefore, the motion information table may not include a motion information candidate derived based on the motion information of a block included in the coding tree unit or the merge processing area including the current block.

[0264] Optionally, only the motion information of some blocks included in the merge processing area may be added to the temporary motion information table. As an example, only the blocks at predefined positions within the merge processing area may be used to update the motion information table. The predefined position may include at least one of a block located at the upper left of the merge processing area, a block located at the upper right of the merge processing area, a block located at the lower left of the merge processing area, a block located at the lower right of the merge processing area, a block located at the center of the merge processing area, a block adjacent to the right border of the merge processing area, and a block adjacent to the lower border of the merge processing area. As an example, only the motion information of the block adjacent to the lower right corner of the merge processing area may be added to the temporary motion information table, and the motion information of other blocks may not be added to the temporary motion information table.

[0265] The maximum number of temporary motion information candidates that the temporary motion information table can include may be set to be equal to the maximum number of motion information candidates that the motion information table can include. Alternatively, the maximum number of temporary motion information candidates that the temporary motion information table can include may be determined according to the size of the coding tree unit or the merge processing region. Alternatively, the maximum number of temporary motion information candidates that the temporary motion information table can include may be set to be smaller than the maximum number of motion information candidates that the motion information table can include.

[0266] The current block included in the coding tree unit or the merge processing area may be set not to use the temporary motion information table about the corresponding coding tree unit or the merge processing area. In other words, when the number of merge candidates included in the merge candidate list of the current block is less than a threshold value, the motion information candidates included in the motion information table may be added to the merge candidate list, and the temporary motion information candidates included in the temporary motion information table may not be added to the merge candidate list. Therefore, the motion information of other blocks included in the same coding tree unit or the same merge processing area as the current block may not be used for motion compensation prediction of the current block.

[0267] If encoding / decoding of all blocks included in the coding tree unit or the merge processing area is completed, the motion information table and the temporary motion information table may be made consistent.

[0268] Fig.19 is a diagram showing an example of making the motion information table and the temporary motion information table consistent.

[0269] If encoding / decoding of all blocks included in the coding tree unit or the merge processing area is completed, as in Fig.19 In the example shown in , the temporary motion information candidates included in the temporary motion information table may be updated in the motion information table.

[0270] In this regard, the temporary motion information candidates included in the temporary motion information table may be added to the motion information table in the order of insertion in the temporary motion information table (in other words, in ascending or descending order of index values).

[0271] In another example, the temporary motion information candidates included in the temporary motion information table may be added to the motion information table in a predefined order. In this regard, the predefined order may be determined according to a scanning order of coding blocks in a merge processing area or a coding tree unit. The scanning order may be at least one of a raster scan, a horizontal scan, a vertical scan, or a zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0272] Alternatively, a temporary motion information candidate including unidirectional motion information may be added to the motion information table before a temporary motion information candidate including bidirectional motion information. Conversely, a temporary motion information candidate including bidirectional motion information may be added to the motion information table before a temporary motion information candidate including unidirectional motion information.

[0273] Alternatively, the temporary motion information candidates may be added to the motion information table in the order of high usage frequency or low usage frequency in the merge processing region or the coding tree unit.

[0274] In the case where a temporary motion information candidate included in the temporary motion information table is added to the motion information table, a redundancy check for the temporary motion information candidate may be performed. In an example, when a motion information candidate identical to the temporary motion information candidate included in the temporary motion information table is pre-stored in the motion information table, the temporary motion information candidate may not be added to the motion information table. In this regard, a redundancy check may be performed for a portion of the motion information candidates included in the motion information table. In an example, a redundancy check may be performed for motion information candidates having an index exceeding or below a threshold. In an example, when the temporary motion information candidate is equal to a motion information candidate having an index higher than a predefined value, the temporary motion information candidate may not be added to the motion information table.

[0275] This may limit the use of motion information candidates derived from blocks included in the same coding tree unit or the same merge processing region as the current block as the merge candidate of the current block. To this end, address information of the block may be additionally stored for the motion information candidate. The address information of the block may include at least one of the position of the block, the address of the block, the index of the block, the position of the merge processing region including the block, the address of the merge processing region including the block, the index of the merge processing region including the block, the position of the coding tree region including the block, the address of the coding tree region including the block, or the index of the coding tree region including the block.

[0276] Intra-frame prediction predicts a current block by using reconstructed samples that have been encoded / decoded and are around the current block. In this regard, the reconstructed samples before applying the in-loop filter can be used for intra-frame prediction of the current block.

[0277] The intra prediction method includes matrix-based intra prediction and intra prediction according to the direction of adjacent reconstruction samples. Information indicating the intra prediction method of the current block can be sent as a bitstream signal. The information can be a 1-bit flag. Optionally, the intra prediction of the current block can be determined based on at least one of the position of the current block, the size of the current block, the shape of the current block, or the intra prediction method of the adjacent block. In the example, when the current block is presented as crossing the picture boundary, it can be set so that the matrix-based intra prediction method is not applied to the current block.

[0278] The matrix-based intra prediction method is a method for obtaining a prediction block of a current block based on the matrix stored in the encoder and the decoder and the matrix product of the reconstructed samples around the current block. Information for specifying any one of a plurality of pre-stored matrices may be signaled in a bitstream. The decoder may determine a matrix for performing intra prediction on the current block based on the above information and the size of the current block.

[0279] General intra prediction is a method of obtaining a prediction block of a current block based on a non-directional intra prediction mode or a directional intra prediction mode. Hereinafter, a process of intra prediction based on general intra prediction will be described in detail with reference to the accompanying drawings.

[0280] Fig. 20 is a flowchart of an intra-frame prediction method according to an embodiment of the present disclosure.

[0281] A reference sample line of a current block may be determined (S2001). The reference sample line refers to a set of reference samples included in a kth line far above and / or to the left of the current block. The reference samples may be derived from reconstructed samples encoded / decoded around the current block.

[0282] Index information identifying a reference sample line of a current block among a plurality of reference sample lines may be signaled in a bitstream. In an example, index information intra_luma_ref_idx for specifying a reference sample line of a current block may be signaled in a bitstream. Index information may be signaled per coding block.

[0283] The plurality of reference sample lines may include at least one of a first line, a second line, or a third line above and / or on the left side of the current block. A reference sample line consisting of a row adjacent to the top of the current block and a column adjacent to the left side of the current block among the plurality of reference sample lines may be referred to as an adjacent reference sample line, and other reference sample lines may be referred to as a non-adjacent reference sample line.

[0284] Table 1 shows the index assigned to each candidate reference sample line.

[0285]

Table 1

[0286] Index (intra_luma_ref_idx) Reference sample line 0 Adjacent reference sample point line 1 The first non-adjacent reference sample line 2 The second non-adjacent reference sample line

[0287] Based on at least one of the position, size, shape, or prediction coding mode of the neighboring block of the current block, a reference sample line of the current block may be determined. In one example, when the current block is adjacent to a boundary of a picture, a tile, a slice, or a coding tree unit, the neighboring reference sample line may be determined as the reference sample line of the current block. The reference sample line may include an upper reference sample located above the current block and a left reference sample located on the left side of the current block. The upper reference sample and the left reference sample may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before an in-loop filter is applied.

[0288] Next, the intra prediction mode of the current block may be determined (S2002). For the intra prediction mode of the current block, at least one of a non-directional intra prediction mode or a directional intra prediction mode may be determined as the intra prediction mode of the current block. The non-directional intra prediction mode includes planar and DC, and the directional intra prediction mode includes 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

[0289] Fig.21 is a diagram showing an intra prediction mode.

[0290] Fig.21 (a) shows 35 intra prediction modes, Fig.21 (b) shows 67 intra prediction modes.

[0291] Can define the ratio Fig.21 A greater or lesser number of intra prediction modes is shown.

[0292] Based on the intra prediction mode of the neighboring blocks adjacent to the current block, the MPM (Most Probable Mode) can be set. In this regard, the neighboring blocks may include a left neighboring block adjacent to the left side of the current block and an upper neighboring block adjacent to the upper side of the current block.

[0293] The number of MPMs included in the MPM list may be preset in the encoder and the decoder. In an example, the number of MPMs may be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs may be signaled in a bitstream. Optionally, the number of MPMs may be determined based on at least one of a prediction coding mode of a neighboring block, a size, a shape, or a reference sample line index of a current block. In an example, although N MPMs may be used when an adjacent reference sample line is determined to be a reference sample line of a current block, M MPMs may be used when a non-adjacent reference sample line is determined to be a reference sample line of a current block. Since M is a natural number less than N, in an example, N may be 6 and M may be 5, 4, or 3. Therefore, although when the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra-frame prediction mode of the current block may be determined to be any one of the 6 candidate intra-frame prediction modes, when the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra-frame prediction mode of the current block may be determined to be any one of the 5 candidate intra-frame prediction modes.

[0294] Alternatively, a fixed number (eg, 6 or 5) of MPM candidates may be used regardless of the index of the reference sample line of the current block.

[0295] When matrix-based intra prediction is applied to neighboring blocks, MPM candidates may be derived by inferring the intra prediction modes of the neighboring blocks as planes.

[0296] When intra BDPCM is applied to a neighboring block, an MPM candidate may be derived by inferring the intra prediction mode of the neighboring block as a default mode. In this case, the default mode may be at least one of DC, planar, vertical direction, or horizontal direction.

[0297] Optionally, based on the intra BDPCM application direction of the neighboring block, the intra prediction mode of the neighboring block can be determined. In the example, when the intra BDPCM in the horizontal direction is applied to the neighboring block, it can be inferred that the intra prediction mode of the neighboring block is in the horizontal direction. On the other hand, when the intra BDPCM in the vertical direction is applied to the neighboring block, it can be inferred that the intra prediction mode of the neighboring block is in the vertical direction.

[0298] An MPM list including a plurality of MPMs may be generated, and information indicating whether the MPM identical to the intra prediction mode of the current block is included in the MPM list may be signaled in a bitstream. Since the information is a 1-bit flag, it may be referred to as an MPM flag. When the MPM flag indicates that the MPM list includes the same MPM as the current block, index information identifying one of the MPMs may be signaled in a bitstream. In an example, index information mpm_idx specifying any one of a plurality of MPMs may be signaled in a bitstream. The MPM specified by the index information may be set as the intra prediction mode of the current block. When the MPM flag indicates that the MPM list does not include the same MPM as the current block, the remaining mode information indicating any one of the remaining intra prediction modes other than the MPM may be signaled in a bitstream. The remaining mode information indicates an index value corresponding to the intra prediction mode of the current block when the index is reallocated to the remaining intra prediction modes other than the MPM. The decoder may determine the intra prediction mode of the current block by arranging the MPMs in ascending order and comparing the remaining mode information with the MPM. In an example, when the remaining mode information is equal to or less than the MPM, the intra prediction mode of the current block may be derived by adding 1 to the remaining mode information.

[0299] When deriving the intra-frame prediction mode of the current block, comparing part of the MPM with the remaining mode information may be omitted. In the example, the MPM in the non-directional intra-frame prediction mode in the MPM may be excluded from the comparison target. When the non-directional intra-frame prediction mode is set to the MPM, the remaining mode information clearly indicates the directional intra-frame prediction mode, so the intra-frame prediction mode of the current block may be derived by comparing the remaining MPMs other than the non-directional intra-frame prediction mode with the remaining mode information. Instead of excluding the non-directional intra-frame prediction mode from the comparison target, the resulting value may be compared with the remaining MPM after adding the number of the non-directional intra-frame prediction mode to the remaining mode information.

[0300] Instead of setting the default mode to MPM, information indicating whether the intra-frame prediction mode of the current block is the default mode may be signaled in the bitstream. The information is a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be signaled only when the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode may include at least one of a plane, a DC, a vertical direction mode, or a horizontal direction mode. In the example, when the plane is set to the default mode, the default mode flag may indicate whether the intra-frame prediction mode of the current block is a plane. When the default mode flag indicates that the intra-frame prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra-frame prediction mode of the current block.

[0301] When the default mode flag is used, it can be set not to set the same intra prediction mode as the default mode to the MPM. In the example, when the default mode flag indicates whether the intra prediction mode of the current block is plane, the intra prediction mode of the current block can be derived by using 5 MPMs excluding the MPM corresponding to the plane.

[0302] When a plurality of intra prediction modes are set as a default mode, index information indicating any one of the default modes may be further signaled. The intra prediction mode of the current block may be set to the default mode indicated by the index information.

[0303] When the index of the reference sample line of the current block is not 0, it may be set not to use the default mode. In an example, when a non-adjacent reference sample line is determined as the reference sample line of the current block, it may be set not to use a non-directional intra prediction mode such as a DC mode or a planar mode. Therefore, when the index of the reference sample line is not 0, the default mode flag may not be signaled, and the value of the default mode flag may be inferred to be a predefined value (i.e., false).

[0304] When the intra prediction mode of the current block is determined, prediction samples of the current block may be obtained based on the determined intra prediction mode ( S2003 ).

[0305] When the DC mode is selected, the prediction sample of the current block may be generated based on the average value of the reference samples. In detail, the values ​​of all samples in the prediction block may be generated based on the average value of the reference samples. The average value may be derived using at least one of the upper reference sample adjacent to the upper side of the current block and the left reference sample adjacent to the left side of the current block.

[0306] The number or range of reference samples used when deriving the average value may vary based on the shape of the current block. In an example, when the current block is a non-square block having a width greater than a height, the average value may be calculated by using the upper reference sample. Conversely, when the current block is a non-square block having a width less than a height, the average value may be calculated by using the left reference sample. In other words, when the width and height of the current block are different, the average value may be calculated using the reference sample adjacent to the larger length. Alternatively, whether to calculate the average value by using the upper reference sample or by using the left reference sample may be determined based on the ratio between the width and height of the current block.

[0307] When the planar mode is selected, the prediction sample can be obtained by using the horizontal prediction sample and the vertical prediction sample. In this regard, the horizontal prediction sample can be obtained based on the left reference sample and the right reference sample located at the same horizontal line as the prediction sample, and the vertical prediction sample can be obtained based on the upper reference sample and the lower reference sample located at the same vertical line as the prediction sample. In this regard, the right reference sample can be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample can be generated by copying the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample can be obtained based on the weighted sum of the left reference sample and the right reference sample, and the vertical prediction sample can be obtained based on the weighted sum of the upper reference sample and the lower reference sample. In this regard, the weighting factor assigned to each reference sample can be determined according to the position of the prediction sample. The prediction sample can be obtained based on the average or weighted sum of the horizontal prediction sample and the vertical prediction sample. When the weighted sum is used, the weighting factors assigned to the horizontal direction prediction samples and the vertical direction prediction samples may be determined based on the positions of the prediction samples.

[0308] When a directional prediction mode is selected, a parameter representing a prediction direction (or prediction angle) of the selected directional prediction mode may be determined. Table 2 below represents an intra directional parameter of intraPredAng for each intra prediction mode.

[0309]

Table 2

[0310]

[0311]

[0312] Table 2 represents the intra-frame direction parameter of each intra-frame prediction mode, where when 35 intra-frame prediction modes are defined, its index is one of 2 to 34. When the directional intra-frame prediction modes are defined to be greater than 33, the intra-frame direction parameter of each intra-frame prediction mode can be set by subdividing Table 2. The upper reference sample and the left reference sample of the current block are arranged in a line, and then the prediction sample can be obtained based on the value of the intra-frame direction parameter. In this regard, when the value of the intra-frame direction parameter is a negative value, the left reference sample and the upper reference sample can be arranged in a line.

[0313] Fig. 22 and Fig.23 are diagrams each showing an example of a one-dimensional arrangement in which reference spots are arranged in a line.

[0314] Fig. 22 is a diagram showing a one-dimensional arrangement in the vertical direction in which reference samples are arranged in the vertical direction, and Fig.23is a diagram showing a horizontal one-dimensional arrangement of reference samples arranged in the horizontal direction. Assuming that 35 intra prediction modes are defined, Fig. 22 and 23 .

[0315] When the intra prediction mode index is any one of 11 to 18, a one-dimensional arrangement in the horizontal direction may be applied, wherein the upper reference sample is rotated counterclockwise, and when the intra prediction mode index is any one of 19 to 25, a one-dimensional arrangement in the vertical direction may be applied, wherein the left reference sample is rotated clockwise. When arranging the reference samples in a line, the intra prediction mode angle may be considered.

[0316] The reference sample determination parameter may be determined based on the intra direction parameter. The reference sample determination parameter may include a reference sample index for specifying a sample, and a weighting factor parameter for determining a weighting factor applied to the reference sample.

[0317] The reference sample index iIdx and the weighting factor parameter i can be obtained by the following equations 2 and 3 respectively. fact .

[0318] [Equation 2]

[0319] iIdx=(y+1)*P ang / 32

[0320] [Equation 3]

[0321] i fact =[(y+1)*P ang ]&31

[0322] In equations 2 and 3, P ang Indicates intra-frame direction parameters. The reference sample specified by the reference sample index iIdx corresponds to an integer pixel.

[0323] In order to derive the prediction sample, at least one reference sample may be specified. Specifically, according to the slope of the prediction mode, the position of the reference sample used to derive the prediction sample may be specified. In an example, the reference sample used to derive the prediction sample may be specified by using the reference sample index iIdx.

[0324] In this regard, when the slope of the intra-frame prediction mode is not represented by one reference sample, the prediction sample may be generated by performing interpolation on a plurality of reference samples. In the example, when the slope of the intra-frame prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample may be obtained by performing interpolation on the first reference sample and the second reference sample. In other words, when an angle line according to the intra-frame prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample may be obtained by performing interpolation on reference samples adjacent to the left and right sides or above and below the position where the angle line passes.

[0325] Equation 4 below represents an example of obtaining a prediction sample based on a reference sample.

[0326] [Equation 4]

[0327] P(x,y)=((32-i fact ) / 32)*Ref_1D(x+iIdx+1)+(i fact / 32)*Ref_1D(x+iIdx+2)

[0328] In Equation 4, P represents a prediction sample, and Ref_1D represents any one of the reference samples arranged in a line. In this regard, the position of the reference sample may be determined by the position (x, y) of the prediction sample and the reference sample index iIdx.

[0329] When the slope of the intra prediction mode may be represented by a reference sample, the weighting factor parameter i is set fact is 0. Therefore, Equation 4 can be simplified to the following Equation 5.

[0330] [Equation 5]

[0331] P(x, y) = Ref_1D(x+iIdx+1)

[0332] Intra prediction of the current block may be performed based on a plurality of intra prediction modes. In an example, an intra prediction mode may be derived for each prediction sample, and the prediction sample may be derived based on the intra prediction mode assigned to each prediction sample.

[0333] Optionally, an intra prediction mode may be derived for each region, and intra prediction of each region may be performed based on the intra prediction mode assigned to each region. In this regard, the region may include at least one sample. At least one of the size and shape of the region may be adaptively determined based on at least one of the size of the current block, the shape of the current block, and the intra prediction mode of the current block. Optionally, at least one of the size and shape of the region may be predefined in the encoder and the decoder independently of the size or shape of the current block.

[0334] Fig.24 is a diagram showing an angle formed between a directional intra prediction mode and a straight line parallel to the x-axis.

[0335] like Fig.24 In the example shown in , the directional prediction mode may exist between the lower left diagonal direction and the upper right diagonal direction. Describing the angle formed between the x-axis and the directional prediction mode, the directional prediction mode may exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0336] When the current block is non-square, there may be a case where a prediction sample is derived by using a reference sample located on an angle line according to an intra prediction angle and located farther than a reference sample close to a prediction sample according to an intra prediction mode of the current block.

[0337] Fig.25 is a diagram showing an aspect of obtaining prediction samples when the current block is non-square.

[0338] In the example, Fig.25 In the example shown in (a), it is assumed that the current block is a non-square with a width greater than a height, and the intra prediction mode of the current block is a directional intra prediction mode with an angle of 0 to 45 degrees. In the above case, when the prediction sample A around the right column of the current block is derived from the reference samples located in the angular mode according to the above angle, there may be a case where the left reference sample L far from the prediction sample is used instead of the upper reference sample T close to the prediction sample.

[0339] In another example, if Fig.25 In the example shown in (b), it is assumed that the current block is a non-square whose height is greater than its width, and the intra prediction mode of the current block is a directional intra prediction mode from -90 degrees to -135 degrees. In the above case, when the prediction sample A around the lower row of the current block is derived from the reference samples located in the angular mode according to the above angle, there may be a case where the upper reference sample T far from the prediction sample is used instead of the upper reference sample L close to the prediction sample.

[0340] To solve the above problem, when the current block is non-square, the intra prediction mode of the current block can be replaced with the intra prediction mode in the opposite direction. Fig.21 The directional prediction mode shown in the figure is a directional prediction mode of an angle. The above directional intra prediction mode can be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode represents a directional intra prediction mode that does not belong to the range of 45 degrees to -135 degrees.

[0341] Fig.26is a diagram illustrating a wide-angle intra prediction mode.

[0342] exist Fig.26 In the example shown in , the intra prediction modes having indices from -1 to -14 and the intra prediction modes having indices from 67 to 80 represent the wide-angle intra prediction modes.

[0343] exist Fig.26 , 14 wide-angle intra prediction modes (from -1 to -14) with angles greater than 45 degrees and 4 wide-angle intra prediction modes (from 67 to 80) with angles less than -135 degrees are shown. However, a greater or lesser number of wide-angle intra prediction modes may be defined.

[0344] When the wide-angle intra prediction mode is used, the length of the upper reference sample may be set to 2W+1, and the length of the left reference sample may be set to 2H+1.

[0345] By using the wide-angle intra prediction mode, it is possible to predict Fig.25 (a) shows the sample point A, and can be predicted by the reference sample point L Fig.25 (b) Sample point A shown.

[0346] In addition to the conventional intra prediction mode and the N wide-angle intra prediction modes, a total of 67+N intra prediction modes may be used. In an example, Table 3 represents intra direction parameters for intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0347]

Table 3

[0348]

[0349]

[0350] When the current block is non-square and the intra prediction mode of the current block obtained in S2502 belongs to the transformation range, the intra prediction mode of the current block may be transformed into a wide-angle intra prediction mode. The transformation range may be determined based on at least one of the size, shape or ratio of the current block. In this regard, the ratio may represent the ratio between the width and height of the current block. When the current block is non-square with a width greater than the height, the transformation range may be set to an intra prediction mode index in the upper right diagonal direction (e.g., 66) to (intra prediction mode index in the upper right diagonal direction - N). In this regard, N may be determined based on the ratio of the current block. When the intra prediction mode of the current block belongs to the transformation range, the intra prediction mode may be transformed into a wide-angle intra prediction mode. The transformation may be to subtract a predefined value from the intra prediction mode, and the predefined value may be the total number of intra prediction modes excluding the wide-angle intra prediction mode (e.g., 67).

[0351] In the above example, the intra prediction modes from number 66 to number 53 may be transformed into wide-angle intra prediction modes from number -1 to number -14, respectively.

[0352] When the current block is a non-square with a height greater than a width, the transformation range may be set from the intra prediction mode index in the lower left diagonal direction (e.g., 2) to (the intra prediction mode index in the lower left diagonal direction + M). In this regard, M may be determined based on the ratio of the current block. When the intra prediction mode of the current block belongs to the transformation range, the intra prediction mode may be transformed into a wide-angle intra prediction mode. The transformation may be adding a predefined value to the intra prediction mode, and the predefined value may be the total number of directional intra prediction modes excluding the wide-angle intra prediction mode (e.g., 65).

[0353] In the above example, the intra prediction modes from number 2 to number 15 may be transformed into wide-angle intra prediction modes from number 67 to number 80, respectively.

[0354] Hereinafter, the intra prediction mode belonging to the transform range is referred to as a wide-angle intra prediction alternative mode.

[0355] The transform range may be determined based on the ratio of the current block. In an example, Tables 4 and 5 show transform ranges for a case where 35 intra prediction modes excluding the wide angle intra prediction mode are defined and a case where 67 intra prediction modes excluding the wide angle intra prediction mode are defined, respectively.

[0356]

Table 4

[0357] condition Alternative intra prediction modes W / H=2 Mode 2, 3, 4 W / H>2 Mode 2, 3, 4, 5, 6 W / H=1 none H / W=1 / 2 Mode 32, 33, 34 H / W<1 / 2 Mode 30, 31, 32, 33, 34

[0358]

Table 5

[0359] condition Alternative intra prediction modes W / H=2 Mode 2, 3, 4, 5, 6, 7 W / H>2 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H=1 none H / W=1 / 2 Mode 61, 62, 63, 64, 65, 66 H / W<1 / 2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0360] As shown in the examples of Tables 4 and 5, the number of wide-angle intra prediction replacement modes included in the transform range may vary according to the ratio of the current block. The ratio of the current block may be further subdivided to set the transform range as shown in Table 6 below.

[0361]

Table 6

[0362] condition Alternative intra prediction modes W / H=16 Mode 12, 13, 14, 15 W / H=8 Mode 12, 13 W / H=4 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 H / W=2 Mode 2, 3, 4, 5, 6, 7 H / W=1 none W / H=1 / 2 Mode 61, 62, 63, 64, 65, 66 W / H=1 / 4 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66 W / H=1 / 8 Mode 55, 56 H / W=1 / 16 Modes 53, 54, 55, 56

[0363] When a non-adjacent reference sample line is determined to be a reference sample line of the current block, or when a multi-line intra prediction encoding method for selecting one of a plurality of reference sample lines is used, the prediction method may be configured not to use the wide-angle intra prediction mode. That is, although the current block has a non-square shape and the intra prediction mode of the current block belongs to the transformation range, the intra prediction mode of the current block may not be transformed into the wide-angle intra prediction mode. Optionally, when the intra prediction mode of the current block is determined to be the wide-angle intra prediction mode, the prediction method may be configured so that the non-adjacent reference sample line is not usable as a reference sample line of the current block, or may be configured not to use the multi-line intra prediction encoding method for selecting one of a plurality of reference sample lines. When the multi-line intra prediction encoding method is not used, the adjacent reference sample line may be determined to be the reference sample line of the current block.

[0364] When the wide-angle intra prediction mode is not used, each of refW and refH may be set to the sum of nTbW and nTbH. Therefore, a non-adjacent reference sample line spaced i from the current block may include (nTbW+nTbH+offsetX[i]) upper reference samples and (nTbW+nTbH+offsetY[i]) left reference samples in addition to the upper left reference sample. That is, a non-adjacent reference sample line spaced i from the current block may include (2nTbW+2nTbH+offsetX[i]+offsetY[i]+1) reference samples. For example, when the value of whRatio is greater than 1, the value of offsetX may be set to a value greater than offsetY. In one example, when the value of offsetX may be set to 1, the value of offsetY may be set to 0. Conversely, when the value of whRatio is less than 1, the value of offsetY may be set to a value greater than offsetX. In one example, the value of offsetX may be set to 0, and the value of offsetY may be set to 1.

[0365] Since the wide-angle intra-prediction mode is used in addition to the conventional intra-prediction mode, resources for encoding the wide-angle intra-prediction mode may increase, and thus, encoding efficiency may decrease. Therefore, instead of encoding the wide-angle intra-prediction mode as it is, an alternative intra-prediction mode of the wide-angle intra-prediction mode is encoded to improve encoding efficiency.

[0366] In an example, when the current block is encoded by using the wide angle intra prediction mode number 67, number 2 which is the wide angle replacement intra prediction mode number 67 may be encoded as the intra prediction mode of the current block. In addition, when the current block is encoded by using the wide angle intra prediction mode number -1, number 66 which is the wide angle replacement intra prediction mode number -1 may be encoded as the intra prediction mode of the current block.

[0367] The decoder may decode the intra prediction mode of the current block and determine whether the decoded intra prediction mode belongs to the transformation range. When the decoded intra prediction mode is the wide-angle replacement intra prediction mode, the intra prediction mode may be transformed into the wide-angle intra prediction mode.

[0368] Alternatively, when the current block is encoded by the wide-angle intra prediction mode, the wide-angle intra prediction mode may be encoded as it is.

[0369] The encoding of the intra prediction mode may be performed based on the MPM list described above. Specifically, when a neighboring block is encoded in the wide-angle intra prediction mode, the MPM may be set based on the wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode.

[0370] A coding block or a transform block may be partitioned into a plurality of sub-blocks (or sub-partitions). When a coding block or a transform block is partitioned into a plurality of sub-blocks, prediction, transformation, and quantization may be performed on each sub-block. Partitioning a coding block or a transform block into a plurality of sub-blocks may be defined as a sub-partition intra coding method.

[0371] Information indicating whether a sub-partition intra coding method is applied may be signaled in the bitstream. The information may be a 1-bit flag. In an example, a syntax factor "intra_subpartitions_mode_flag" indicating whether a coding block or a transform block is partitioned into a plurality of sub-blocks may be signaled in the bitstream.

[0372] Optionally, whether to apply the sub-partition intra-coding method may be determined based on at least one of the size, shape, or intra-prediction mode of the coding block or the transform block. In an example, when the intra-prediction mode of the coding block is a non-directional intra-prediction mode (e.g., plane or DC) or a predefined directional intra-prediction mode (e.g., an intra-prediction mode in the horizontal direction, an intra-prediction mode in the vertical direction, or an intra-prediction mode in the diagonal direction), the sub-partition intra-coding method may not be applied. Optionally, when the size of the coding block is less than a threshold, the sub-partition intra-coding method may be set to not be used.

[0373] Optionally, when intra prediction of a subblock is performed based on an intra prediction mode of a coding block, whether to apply a sub-partition intra coding method may be determined based on whether a reconstructed sample included in a neighboring subblock should be used as a reference sample in intra prediction of the subblock. In an example, when the intra prediction mode of the coding block is an intra prediction mode in a diagonal direction or a wide-angle intra prediction mode, and when intra prediction is performed on a subblock based on the intra prediction mode and a neighboring subblock may not be used as a reference sample, it may be set not to use the sub-partition intra coding method.

[0374] Optionally, when the ratio of the height and width of the coding block is equal to or greater than a threshold or equal to or less than a threshold, the sub-partition intra-coding method may be set not to be used. Optionally, when at least one of the height or width of the coding block is equal to or less than a threshold, the sub-partition intra-coding method may not be used. In the example, when the width or height of the coding block is equal to or less than a threshold, or when both the height and width of the coding block are equal to or less than a threshold, the sub-partition intra-coding method may not be used. Optionally, when the number of samples included in the coding block is equal to or less than a threshold, the sub-partition intra-coding method may not be used. The threshold may have a predefined value in the encoder and the decoder. Optionally, information for determining the threshold may be signaled in a bitstream.

[0375] Optionally, whether to signal a flag indicating whether to apply the sub-partition intra-coding method may be determined based on at least one of the size, shape, or intra-prediction mode of the coding block or the transform block. In the example, the flag indicating whether to apply the sub-partition intra-coding method may be encoded and signaled only when both the height and width of the coding block are equal to or less than a threshold value and / or when the size of the coding block is equal to or greater than a threshold value. When the flag indicating whether to apply the sub-partition intra-coding method is not encoded, the sub-partition intra-coding method may not be applied.

[0376] When the subpartition intra coding method is not used, the signaling of the syntax factor intra_subpartitions_mode_flag may be omitted. When the signaling of the flag is omitted, the flag may be considered to indicate that the subpartition intra coding method is not applied.

[0377] When the sub-partition intra encoding method is applied, the partition shape of the coding block or the transform block may be determined. In this case, the partition shape indicates the partition direction of the coding block or the transform block. In an example, partitioning in the vertical direction may refer to partitioning the coding block or the transform block by using at least one vertical line, and partitioning in the horizontal direction may refer to partitioning the coding block or the transform block by using at least one horizontal line.

[0378] Fig. 27 : is a diagram showing an example of partitioning in a vertical direction and partitioning in a horizontal direction.

[0379] Fig. 27 (a) shows an example of partitioning a coding block into two sub-blocks. Fig. 27 (b) shows an example of partitioning a coding block into four sub-blocks.

[0380] Information for determining the partition shape of a coding block or a transform block may be sent with a signal in a bitstream. In an example, information indicating whether a partition in a vertical direction is applied to a coding block or a transform block or whether a partition in a horizontal direction is applied to a coding block or a transform block may be sent with a signal. The information may be a 1-bit flag intra_subpart_type_flag. When the value of the flag is 1, it indicates that the coding block or the transform block is partitioned in the horizontal direction, and when the value of the flag is 0, it indicates that the coding block or the transform block is partitioned in the vertical direction.

[0381] Optionally, the partition shape of the coding block or transform block can be determined based on the size, shape or intra prediction mode of the coding block or transform block. In the example, the partition shape of the coding block can be determined based on the ratio of the width and height of the coding block. For example, when the value whRatio representing the aspect ratio of the coding block is equal to or greater than the first threshold, the partition in the vertical direction can be applied to the coding block. Otherwise, the partition in the horizontal direction can be applied to the coding block.

[0382] Fig.28 is a diagram illustrating an example of determining a partition shape of a coding block.

[0383] For ease of description, it is assumed that the first threshold is 2. Fig.28 In the example shown in (a), the whRatio of the coding block is 1, which is less than the first threshold. Therefore, encoding of information indicating the partition shape of the coding block can be omitted, and partitioning in the horizontal direction can be applied to the coding block.

[0384] exist Fig.28 In the example shown in (b), the whRatio of the coding block is 2, which is the same as the first threshold. Therefore, encoding of information indicating the partition shape of the coding block can be omitted, and partitioning in the vertical direction can be applied to the coding block.

[0385] The partition shape of the coding block can be determined by using a second threshold whose sign is opposite to the first threshold. In the example, when the value of whRatio is equal to or less than the second threshold, the partition in the horizontal direction can be applied to the coding block, otherwise, the partition in the vertical direction can be applied to the coding block. The absolute values ​​of the first threshold and the second threshold may be the same, and their signs may be different. In the example, when the first threshold is N (in this case, N is an integer such as 1, 2, 4, etc.), the second threshold may be -N.

[0386] Fig.29 is a diagram illustrating an example of determining a partition shape of a coding block.

[0387] For ease of description, it is assumed that the second threshold is -2. Fig.29In the example shown in (a), the whRatio of the coding block is -1, which is greater than the second threshold. Therefore, encoding of information indicating the partition shape of the coding block can be omitted, and partitioning in the vertical direction can be applied to the coding block.

[0388] exist Fig.29 In the example shown in (b), the whRatio of the coding block is -2, which is the same as the second threshold. Therefore, encoding of information indicating the partition shape of the coding block can be omitted, and partitioning in the horizontal direction can be applied to the coding block.

[0389] Optionally, the partition shape of the coding block may be determined based on a first threshold and a second threshold. In the example, when the value of whRatio is equal to or greater than the first threshold, the partition in the horizontal direction may be applied to the coding block, and when the value of whRatio is equal to or less than the second threshold, the partition in the vertical direction may be applied to the coding block. When the value of whRatio is between the first threshold and the second threshold, the partition shape of the current block may be determined by parsing information in the bitstream.

[0390] The first threshold and the second threshold may be predefined in the encoder and the decoder. Alternatively, the first threshold and the second threshold may be defined per sequence, picture or slice.

[0391] Optionally, the partition shape may be determined based on the size of the coding block or the transform block. In the example, when the size of the coding block is N×n, partitioning in the vertical direction may be applied, and when the size of the coding block is n×N, partitioning in the horizontal direction may be applied. In this case, n may be a natural number less than N. N and / or n may be predefined values ​​in the encoder and decoder. Optionally, information for determining N and / or n may be signaled in a bitstream. In the example, N may be 32, 4, 128, or 256, etc. Therefore, when the size of the coding block is 128×n (in this case, n is a natural number such as 16, 32, or 64), partitioning in the vertical direction may be applied, and when the size of the coding block is n×128, partitioning in the horizontal direction may be applied.

[0392] Optionally, the partition shape of the coding block or the transform block may be determined based on the intra prediction mode of the coding block or the transform block. In an example, when the intra prediction mode of the coding block is in the horizontal direction or in a direction similar to the horizontal direction, the partition in the vertical direction may be applied to the coding block. In this case, the intra prediction mode in the direction similar to the horizontal direction means that the partition shape is different from the intra prediction mode in the horizontal direction (e.g., Fig.21(b) INTRA_ANGULAR18) is equal to or less than the threshold intra prediction mode (for example, INTRA_ANGULAR18±N). On the other hand, when the intra prediction mode of the coding block is in the vertical direction or in a direction similar to the vertical direction, the partition in the horizontal direction may be applied to the coding block. In this case, the intra prediction mode in the direction similar to the vertical direction indicates the intra prediction mode in the vertical direction (for example, Fig.21 (b) is equal to or less than a threshold value of the intra prediction mode (e.g., INTRA_ANGULAR50±N). In this case, the threshold value N may be a predefined value in the encoder and the decoder. Alternatively, information for determining the threshold value N may be signaled at the sequence level, the picture level, or the slice level.

[0393] When both partitioning in the vertical direction and partitioning in the horizontal direction are available, the partition shape of the coding block may be determined by parsing information indicating the partition shape of the coding block.

[0394] The number of sub-blocks may be determined based on at least one of the size or shape of the coding block or the transform block. In the example, when either the width or height of the coding block is 8 and the other is 4, the coding block may be partitioned into 2 sub-blocks. On the other hand, when both the width and height of the coding block are equal to or greater than 8, or when either the width or height of the coding block is greater than 8, the coding block may be partitioned into 4 sub-blocks. In short, when the coding block has a 4×4 size, the coding block may not be partitioned into sub-blocks. When the coding block has a 4×8 or 8×4 size, the coding block may be partitioned into 2 sub-blocks. Otherwise, the coding block may be partitioned into 4 sub-blocks.

[0395] Alternatively, information indicating the size or shape of the sub-block or the number of sub-blocks may be signaled in the bitstream. The size or shape of the sub-block may be determined by the information indicating the number of sub-blocks. Alternatively, the number of sub-blocks may be determined by the information indicating the size or shape of the sub-block.

[0396] When the sub-partition intra-coding method is applied, the sub-blocks generated by partitioning the coding block or the transform block can use the same intra-prediction mode. In an example, the MPM for the coding block can be derived based on the intra-prediction mode of the neighboring blocks adjacent to the coding block, and the intra-prediction mode for the coding block can be determined based on the derived MPM. When the intra-prediction mode of the coding block is determined, each sub-block can perform intra-prediction by using the determined intra-prediction mode.

[0397] When the sub-partition intra coding method is applied, any one of the MPMs may be determined as the intra prediction mode of the coding block. In other words, when the sub-partition intra coding method is applied, the MPM flag may be considered true even though the MPM flag is not signaled.

[0398] Optionally, when the sub-partition intra-coding method is applied, any one of the predefined candidate intra-prediction modes may be determined as the intra-prediction mode of the coding block. In the example, any one of the intra-prediction modes in the horizontal direction, the intra-prediction mode in the vertical direction, the intra-prediction mode in the diagonal direction (e.g., at least one of the upper left intra-prediction mode, the upper right intra-prediction mode, or the lower left intra-prediction mode) or the non-directional intra-prediction mode (e.g., at least one of the plane or DC) may be determined as the intra-prediction mode of the coding block. Index information specifying any one of the predefined candidate intra-prediction modes may be sent with a bitstream signal. Optionally, the number and / or type of the candidate intra-prediction modes may be different depending on the partition direction of the coding block. In the example, when the partition in the horizontal direction is applied to the coding block, at least one of the non-directional intra-prediction mode, the intra-prediction mode in the vertical direction, the intra-prediction mode in the upper left diagonal direction, or the intra-prediction mode in the upper right diagonal direction may be set as the candidate intra-prediction mode. On the other hand, when partitioning in the vertical direction is applied to the coding block, at least one of a non-directional intra-frame prediction mode, an intra-frame prediction mode in the horizontal direction, an intra-frame prediction mode in the upper left diagonal direction, or an intra-frame prediction mode in the lower left diagonal direction can be set as a candidate intra-frame prediction mode.

[0399] The quantization parameters of the sub-blocks may be determined individually. Therefore, the value of the quantization parameter of each sub-block may be set differently. Information representing a difference value from a quantization parameter of a previous sub-block may be encoded to determine the quantization parameter of each sub-block. In an example, for the Nth sub-block, the different value between the quantization parameter of the Nth sub-block and the quantization parameter of the N-1th sub-block may be encoded.

[0400] Intra-frame prediction of a sub-block can be performed by using reference samples. In this case, the reference samples can be derived from the reconstructed samples of the neighboring blocks adjacent to the sub-block. When the neighboring blocks adjacent to the sub-block are other sub-blocks included in the same coding block as the sub-block, the reference samples of the sub-block can be derived based on the reconstructed samples of the other sub-blocks. In the example, when the first sub-block is located to the left or above the second sub-block, the reference samples of the second sub-block can be derived from the reconstructed samples of the first sub-block. For this reason, parallel intra-frame prediction cannot be applied between sub-blocks. In other words, encoding / decoding can be performed sequentially for the sub-blocks included in the coding block. Therefore, after the encoding / decoding of the first sub-block is completed, the intra-frame prediction of the second sub-block can be performed.

[0401] When the sub-partition intra-frame coding method is applied, a multi-line intra-frame prediction coding method that does not select any one of the multiple reference sample line candidates may be set. When the multi-line intra-frame prediction coding method is not used, the adjacent reference sample line adjacent to each sub-block may be determined as the reference sample line of each sub-block. Optionally, when the index of the reference sample line of the current block is greater than 0, the encoding of the syntax factor intra_subpartitions_mode_flag indicating whether the sub-partition intra-frame coding method is applied may be omitted. When the encoding of the syntax intra_subpartitions_mode_flag is omitted, the sub-partition intra-frame coding method may not be applied.

[0402] The filtering may be performed on samples adjacent to a boundary between subblocks. The filtering may be performed on predicted samples or reconstructed samples. In an example, when it is assumed that the second subblock is adjacent to the right side of the first subblock, the predicted samples or reconstructed samples bordering the left boundary of the second subblock may be filtered by using the reconstructed samples bordering the right boundary of the first subblock.

[0403] Whether to apply the filter to samples adjacent to a boundary between subblocks may be determined based on at least one of an intra prediction mode, a size of a subblock, or the number of subblocks.

[0404] When a prediction block is generated by performing intra prediction, the prediction samples may be updated based on the position of each prediction sample included in the prediction block. This updating method may be referred to as an intra weighted prediction method based on sample position (or position-dependent prediction combination, PDPC).

[0405] Whether PDPC will be used can be determined by considering the size or shape of the current block, the intra-frame prediction mode, the reference sample line of the current block, the size or color component of the current block. In the example, PDPC can be used when the intra-frame prediction mode of the current block is at least one of plane, DC, vertical direction, horizontal direction, a mode with an index value less than the vertical direction, or a mode with an index value greater than the horizontal direction. Optionally, PDPC can be used only when at least one of the width or height of the current block is greater than 4. Optionally, PDPC can be used only when the index of the reference picture line of the current block is 0. Optionally, PDPC can be used only when the index of the reference picture line of the current block is equal to or greater than a predefined value. Optionally, PDPC can be used only for the luminance component. Optionally, whether PDPC is used can be determined based on whether two or more of the listed conditions are met.

[0406] Alternatively, whether to use PDPC may be determined based on whether a sub-partition intra-coding method is used. In an example, when a sub-partition intra-coding method is applied to a coding block or a transform block, PDPC may be set not to be used. Optionally, when a sub-partition intra-coding method is applied to a coding block or a transform block, PDPC may be applied to at least one of a plurality of sub-blocks. In this case, a sub-block targeted by PDPC may be determined based on at least one of the size, shape, position, intra-prediction mode, or reference sample line index of the coding block or sub-block. In an example, PDPC may be applied to a sub-block adjacent to the upper and / or left boundaries of a coding block or to a sub-block adjacent to the lower and / or right boundaries of a coding block. Optionally, based on the size or shape of a sub-block, PDPC may be set to be applied to all sub-blocks included in a coding block, or PDPC may be set not to be applied to all sub-blocks included in a coding block. In an example, when at least one of the width or height of a sub-block is less than a threshold, the application of PDPC may be omitted. In another example, PDPC may be applied to all sub-blocks in a coding block.

[0407] Optionally, whether to apply PDPC may be determined for each sub-block based on whether at least one of the size, shape, intra-prediction mode, or reference picture index of a sub-block generated by partitioning a coding block or a transform block satisfies a preset condition. In the example, PDPC may be applied to the sub-block when at least one of the width or height of the sub-block is greater than 4.

[0408] In another example, information indicating whether PDPC is applied may be signaled in a bitstream.

[0409] Optionally, the area to which PDPC is applied may be determined based on at least one of the size, shape, or intra-frame prediction mode or the position of the prediction sample of the current block. In the example, when the intra-frame prediction mode of the current block has an index greater than the vertical direction, at least one of the prediction samples whose x-axis coordinates or y-axis coordinates are greater than the threshold value may not be modified, and only the prediction samples whose x-axis coordinates or y-axis coordinates are equal to or less than the threshold value may be modified. Optionally, when the intra-frame prediction mode of the current block has an index less than the horizontal direction, at least one of the prediction samples whose x-axis coordinates or y-axis coordinates are greater than the threshold value may not be modified, and only the prediction samples whose x-axis coordinates or y-axis coordinates are equal to or less than the threshold value may be modified. In this case, the threshold may be determined based on at least one of the size, shape, or intra-frame prediction mode of the current block.

[0410] When the prediction sample is obtained by intra-prediction sample, the reference sample used to modify the prediction sample may be determined based on the position of the obtained prediction sample.

[0411] The residual signal may be obtained by performing a difference operation between the predicted sample and the original sample of the current block after performing intra prediction on the current block. In this case, after deriving the difference between the residual signal at a specific position and the adjacent residual signal, the derived difference may be encoded instead of encoding the residual signal at the specific position as it is. In this case, the residual signal may represent the residual sample, the transform coefficient generated by transforming the residual sample, or the coefficient generated by skipping the transform.

[0412] For example, encoding of the difference residual value may be performed after performing a difference operation on the residual signal belonging to the first line and the residual signal belonging to the second line, rather than encoding the residual signal belonging to the second line as it is. In this case, at least one of the x-axis coordinate or the y-axis coordinate may be different for the first line and the second line.

[0413] In an example, the encoder may generate a transform coefficient by transforming the residual sample points, and encode the difference between the generated transform coefficient and the transform coefficient derived by performing a difference operation on the adjacent transform coefficients. The decoder may set the transform coefficient belonging to the first line as a transform coefficient prediction value of the second residual signal, and derive the second transform coefficient by adding the decoded difference operation transform coefficient to the transform coefficient prediction value.

[0414] As described above, when the residual difference is encoded / decoded after the difference operation is performed on the residual signal, this may be referred to as intra BDPCM (Block-based Delta Pulse Code Modulation). Intra BDPCM may be used only when the prediction coding mode of the current block is determined to be intra prediction.

[0415] When intra BDPCM is applied, the prediction samples of the current block may be set to 0. In other words, when intra BDPCM is applied, the residual samples may be set to the reconstructed samples.

[0416] Optionally, when intra BDPCM is applied, the prediction samples of the current block can be derived based on intra prediction. In this case, the intra prediction mode of the current block can be determined according to the intra BDPCM direction. In the example, when the intra BDPCM direction is horizontal, the prediction samples can be obtained based on the intra prediction mode in the horizontal direction. When the intra BDPCM direction is vertical, the prediction samples can be obtained based on the intra prediction mode in the vertical direction.

[0417] Optionally, the prediction samples of the current block may be derived by using a default intra prediction mode. The default intra prediction mode may be any one of DC, planar, horizontal, or vertical. The default intra prediction mode may be predefined in the encoder and decoder. Information specifying one of the default intra prediction modes may be encoded and signaled.

[0418] Alternatively, the intra prediction mode may be derived from one of a plurality of MPM candidates.

[0419] When intra BDPCM is applied to the current block, it is possible to compulsorily use adjacent reference sample lines. In other words, prediction samples can be obtained by omitting signaling of index information specifying one of the reference sample lines and by using the adjacent reference sample lines.

[0420] When the intra-frame BDPCM method is applied, information for determining the intra-frame BDPCM direction may be signaled in the bitstream. In an example, a flag intra_bdpcm_dir_flag indicating the intra-frame BDPCM direction may be signaled in the bitstream. When the syntax intra_bdpcm_dir_flag is 0, it indicates that BDPCM in the horizontal direction is applied, and when the syntax intra_bdpcm_dir_flag is 1, it indicates that BDPCM in the vertical direction is applied.

[0421] Alternatively, the intra-frame BDPCM direction may be determined based on the size or shape of the current block. In the example, when the current block has a non-square shape with a width greater than a height, it may be determined to apply the BDPCM in the horizontal direction. On the other hand, when the current block has a non-square shape with a height greater than a width, it may be determined to apply the BDPCM in the vertical direction.

[0422] Optionally, the intra-frame BDPCM direction can be determined by considering the intra-frame prediction mode of the neighboring block adjacent to the current block. In the example, when at least one intra-frame prediction mode of the left block and the upper block of the current block is in the horizontal direction or in a direction similar to it, the BDPCM in the horizontal direction can be applied to the current block. In this case, the direction similar to the horizontal direction refers to an intra-frame prediction mode whose difference with the intra-frame prediction mode in the horizontal direction is equal to or less than a threshold. Optionally, when at least one intra-frame prediction mode of the left block and the upper block of the current block is in the vertical direction or in a direction similar to it, the BDPCM in the vertical direction can be applied to the current block. In this case, the direction similar to the vertical direction refers to an intra-frame prediction mode whose difference with the intra-frame prediction mode in the vertical direction is equal to or less than a threshold.

[0423] When BDPCM in the horizontal direction is applied, the difference between the residual signal and the residual signal above the adjacent residual signal may be encoded. The decoder may derive the residual signal by adding the above residual signal to the decoded difference.

[0424] When BDPCM in the vertical direction is applied, the difference between the residual signal and the residual signal on the left side of the adjacent residual signal may be encoded. The decoder may derive the residual signal by adding the left residual signal to the decoded difference.

[0425] Optionally, non-directional BDPCM can be applied. In the example, DC BDPCM refers to encoding / decoding the difference between the residual signal at a predetermined position and the average value of the adjacent residual signals at the predetermined position. Planar BDPCM refers to encoding / decoding the average or weighted sum result of the difference in the horizontal direction (the difference between the residual signal at the predetermined position and the residual signal in the horizontal direction of the residual signal at the predetermined position) and the difference in the vertical direction (the difference between the residual signal at the predetermined position and the residual signal in the vertical direction of the residual signal at the predetermined position).

[0426] Information for specifying an available BDPCM mode may be signaled in a bitstream. The information may be information indicating whether non-directional BDPCM is applied or information for specifying any one of a plurality of BDPCM candidates applicable to the current block.

[0427] Information indicating whether intra BDPCM is to be applied to the current block may be signaled in the bitstream. In an example, a flag intra_bdpcm_flag may be signaled in the bitstream. When the syntax intra_bdpcm_flag is 1, this indicates that intra BDPCM is applied to the current block. When the syntax intra_bdpcm_flag is 0, this indicates that intra BDPCM is not applied to the current block.

[0428] At the picture level or sequence level, information indicating the availability of intra-frame BDPCM can be signaled. In an example, a flag sps_intra_bdpcm_flag indicating the availability of intra-frame BDPCM can be signaled through a sequence parameter set (SPS). When the syntax sps_intra_bdpcm_flag is 1, this indicates that the picture with respect to the sequence parameter set can use intra-frame BDPCM. When the syntax sps_intra_bdpcm_flag is 0, this indicates that the picture with respect to the sequence parameter set cannot use intra-frame BDPCM. The intra_bdpcm_flag indicating whether intra-frame BDPCM is applied to the current block can be signaled only when sps_intra_bdpcm_flag is 1.

[0429] When intra BDPCM is applied to the current block, PDPC may be set not to be used.

[0430] When intra BDPCM is applied to the current block, it can be set to force the application of transform skip. In other words, when intra BDPCM is applied to the current block, this value can be considered to be 1, although transform_skip_flag indicating whether transform skip is applied is not signaled.

[0431] The combined prediction mode is a method of generating a predicted image by combining two or more prediction modes. In an example, when the combined prediction mode is applied, a prediction block may be generated by averaging a first prediction block generated based on a first prediction mode and a second prediction block generated based on a second prediction mode or by using their weighted sum operation. The prediction mode may include at least one of an intra prediction mode, a merge mode, an AMVP mode, a skip mode, an intra block copy mode, or a palette mode. In an example, the first prediction mode may be a merge mode, and the second prediction mode may be an intra prediction mode.

[0432] When a combined prediction mode combining a merge mode and an intra prediction mode is used, a prediction block of a current block may be generated by weighting the predictions of a first prediction block obtained based on motion information and a second prediction block obtained based on a predetermined intra prediction mode. The motion information of the current block may be derived from a merge candidate specified by an index merge_idx signaled in a bitstream. The intra prediction mode of the current block may be set to a predefined intra prediction mode. The predefined intra prediction mode may be a plane, DC, horizontal, or vertical mode. Optionally, the intra prediction mode of a neighboring block may be set to the intra prediction mode of the current block.

[0433] A flag indicating whether the combined prediction mode is applied to the current block may be signaled in the bitstream. In an example, the syntax ciip_flag may be signaled in the bitstream. When the value ciip_flag of the syntax is 1, this indicates that the combined prediction mode is applied to the current block. When the value ciip_flag of the syntax is 0, this indicates that the combined prediction mode is not applied to the current block. When the combined prediction mode is not applied to the current block, at least one of the merged offset coding method or the triangle partitioning method may be applied.

[0434] The residual image can be derived by subtracting the predicted image from the original image. In this regard, when the residual image is converted to the frequency domain, the subjective image quality of the image will not be significantly reduced even if the high-frequency components are removed from the frequency components. Therefore, when the value of the high-frequency component is transformed to a small value, or when the value of the high-frequency component is set to 0, the compression efficiency can be improved without causing large visual distortion. Reflecting the above characteristics, a transformation can be performed on the current block so as to decompose the residual image into two-dimensional frequency components. The transformation can be performed by using a transformation method such as DCT (discrete cosine transform), DST (discrete sine transform), etc.

[0435] DCT is to decompose (or transform) the residual image into two-dimensional frequency components by using cosine transform, while DST is to synthesize (or transform) the residual image into two-dimensional frequency components by using sine transform. As a result of transforming the residual image, the frequency components can be represented as a basic image. In the example, when DCT transform is performed on a block of size N×N, N 2 The size of each basic mode component included in the N×N size block can be obtained by transformation. The size of the basic mode component can be referred to as a DCT coefficient or a DST coefficient according to the transformation method used.

[0436] The transform method DCT is mainly used to transform images with a large number of non-zero low-frequency components. The transform method DST is mainly used to transform images with a large number of high-frequency components.

[0437] The residual image may also be transformed by using a transform method other than DCT or DST.

[0438] Hereinafter, transforming the residual image into two-dimensional frequency components is referred to as two-dimensional image transform. In addition, the size of the basic mode components obtained by the transform is referred to as transform coefficients. In the example, the transform coefficients may refer to DCT coefficients or DST coefficients. When both the first transform and the second transform described below are applied, the transform coefficients may refer to the basic mode components generated according to the result of the second transform. In addition, the residual samples skipped by applying the transform are also referred to as transform coefficients.

[0439] The transformation method may be determined in units of blocks. The transformation method may be determined based on at least one of a prediction coding mode of the current block, a size of the current block, or a shape of the current block. In the example, when the current block is encoded by an intra prediction mode and the size of the current block is less than N×N, the transformation may be performed by using a DST transformation method. On the other hand, when the condition is not met, the transformation may be performed by using a DCT transformation method.

[0440] The two-dimensional image transform may not be performed on some blocks of the residual image. Not performing the two-dimensional image transform may be referred to as transform skipping. Transform skipping means that the first transform and the second transform are not applied to the current block. When transform skipping is applied, quantization may be applied to the residual values ​​for which the transform is not performed.

[0441] Whether transform skipping is allowed for the current block may be determined based on at least one of the size or shape of the current block. In an example, transform skipping may be applied only when the size of the current block is less than a threshold. The threshold is related to at least one of the width, height, or number of samples of the current block, and may be defined as 32×32, etc. Optionally, transform skipping may be allowed only for square blocks. In an example, transform skipping may be allowed for square blocks of 32×32, 16×16, 8×8, or 4×4 size. Optionally, transform skipping may be allowed only when the sub-partition intra coding method is not used.

[0442] Optionally, when the subpartition intra coding method is applied to the current block, it may be determined whether to apply transform skipping per subpartition.

[0443] Fig.30 is a diagram illustrating an example of determining whether to perform transform skip on a per sub-block basis.

[0444] The transform may be skipped and applied only to portions of multiple sub-blocks. Fig.30 In the example shown in , it may be set that the transformation is skipped and applied to the sub-block at the upper position of the current block, and the transformation is not skipped and applied to the sub-block at the lower position.

[0445] The transform type of the subblock for which transform skipping is not allowed may be determined based on information signaled in a bitstream. In an example, the transform type may be determined based on tu_mts_idx which will be described below.

[0446] Optionally, the transform type of the sub-block may be determined based on the size of the sub-block. In an example, the horizontal transform type may be determined based on whether the width of the sub-block is equal to or greater than and / or equal to or less than a threshold, and the vertical transform type may be determined based on whether the height of the sub-block is equal to or greater than and / or equal to or less than a threshold.

[0447] Alternatively, information for determining a transform type may be signaled for a coding block, and the transform type specified by the information may be commonly applied to subblocks included in the coding block. In other words, the transform types of subblocks in the coding block may be set identically.

[0448] After the current block is transformed by using DCT or DST, the transformed current block may be transformed again. In this regard, the DCT or DST-based transform may be defined as a first transform, and the block to which the first transform is applied may be transformed again as a second transform.

[0449] The first transform may be performed by using any one of a plurality of transform kernel candidates. In an example, the first transform may be performed by using any one of DCT2, DCT8, or DST7.

[0450] Different transform kernels may be used for the horizontal direction and the vertical direction. Information indicating a combination of a transform kernel in the horizontal direction and a transform kernel in the vertical direction may be signaled in a bitstream. In an example, the above tu_mts_idx may indicate one of the combinations of a transform kernel in the horizontal direction and a transform kernel in the vertical direction.

[0451] The processing unit of the first transform may be different from that of the second transform. In an example, the first transform may be performed on an 8×8 block, and the second transform may be performed on a 4×4 sized sub-block within the transformed 8×8 block. Optionally, the second transform may be performed on transform coefficients belonging to three 4×4 sized sub-blocks. The three sub-blocks may include a sub-block located at the upper left of the current block, a sub-block adjacent to the right of the sub-block, and a sub-block adjacent to the bottom of the sub-block. Optionally, the second transform may be performed on an 8×8 sized block.

[0452] The transform coefficients in the remaining region where the second transform is not performed may also be set to zero.

[0453] Alternatively, the first transform may be performed on the 4×4 block, and the second transform may be performed on a region having an 8×8 size including the transformed 4×4 block.

[0454] Information indicating a transform type of a current block may be signaled in a bitstream. The information may be index information tu_mts_idx, which indicates one of a combination of a transform type for a horizontal direction and a transform type for a vertical direction.

[0455] Based on the transform type candidates specified by the index information tu_mts_idx, a transform kernel for a vertical direction and a transform kernel for a horizontal direction may be determined. Table 7 represents transform type combinations according to tu_mts_idx.

[0456]

Table 7

[0457]

[0458] The transform type may be determined to be one of DCT2, DST7 or DCT8. Optionally, transform skip may be inserted into the transform type candidate.

[0459] When Table 7 is used, when tu_mts_idx is 0, DCT2 may be applied in the horizontal direction, and DCT2 may be applied in the vertical direction. When tu_mts_idx is 2, DCT8 may be applied in the horizontal direction, and DCT7 may be applied in the vertical direction.

[0460] When the sub-partition intra coding method is applied, the transform kernel of the sub-block can be determined independently. In an example, information for specifying transform type combination candidates can be encoded and signaled per sub-block. Therefore, the transform kernels between sub-blocks can be different.

[0461] Optionally, the subblocks may use the same transform type. In this case, tu_mts_idx specifying the transform type combination candidate may be signaled only for the first subblock. Optionally, tu_mts_idx may be signaled at a coding block level, and the transform type of the subblock may be determined by referring to the tu_mts_idx signaled at the coding block level. Optionally, the transform type may be determined based on at least one of the size, shape, or intra prediction mode of one of the subblocks, and the determined transform type may be set for all subblocks.

[0462] Fig.31 is a diagram showing an example in which sub-blocks use the same transform type.

[0463] When the coding block is partitioned in the horizontal direction, the transform type of the sub-block (Sub-CU0) at the upper position of the coding block can be set to be the same as the transform type of the sub-block (Sub-CU1) at the lower position. Fig.31 In the example shown in (a), when the horizontal transform type and the vertical transform type are determined based on tu_mts_idx signaled for the upper subblock, the determined transform type may also be applied to the lower subblock.

[0464] When the coding block is partitioned in the vertical direction, the transform type of the sub-block (Sub-CU0) at the left position of the coding block may be set to be the same as the transform type of the sub-block (Sub-CU1) at the right position. Fig.31 In the example shown in (b), when the horizontal transform type and the vertical transform type are determined based on tu_mts_idx signaled for the left subblock, the determined transform type may also be applied to the right subblock.

[0465] Whether to encode the index information may be determined based on at least one of the size or shape of the current block, the number of non-zero coefficients, whether to perform a second transform, or whether to apply a sub-partition intra encoding method. In an example, when the sub-partition intra encoding method is applied to the current block, or when the number of non-zero coefficients is equal to or less than a threshold, signaling of the index information may be omitted. When signaling of the index information is omitted, a default transform type may be applied to the current block.

[0466] The default transform type may include at least one of DCT2 or DST7. When there are multiple default transform types, one of the multiple default transform types may be selected by considering at least one of the size, shape, or intra-frame prediction mode of the current block, whether the second transform is performed, or whether the sub-partition intra-frame coding method is applied. In an example, one of the multiple transform types may be determined as a horizontal transform type based on whether the width of the current block is within a preset range, and one of the multiple transform types may be determined as a vertical transform type based on whether the height of the current block is within a preset range. Optionally, the default mode may be determined differently according to the size, shape, or intra-frame prediction mode of the current block, or whether the second transform is performed.

[0467] Optionally, when there is only a transform coefficient of a DC component in the current block, the horizontal transform type and the vertical transform type may be set to a default transform type. In an example, when there is only a transform coefficient of a DC component in the current block, the horizontal transform type and the vertical transform type may be set to DCT2.

[0468] The threshold may be determined based on the size or shape of the current block. In an example, when the size of the current block is equal to or less than 32×32, the threshold may be set to 2, and when the current block is larger than 32×32 (e.g., when the current block is a 32×64 or 64×32 size coding block), the threshold may be set to 4.

[0469] A plurality of lookup tables may be pre-stored in the encoding device / decoding device. At least one of an index value assigned to a transform type combination candidate, a type of the transform type combination candidate, or the number of transform type combination candidates may be different for each of the plurality of lookup tables.

[0470] A lookup table for the current block may be selected based on at least one of a size, a shape, or an intra prediction mode of the current block, whether a second transform is applied, or whether a transform is skipped from being applied to a neighboring block.

[0471] In an example, when the size of the current block is equal to or less than 4×4, or when the current block is encoded by inter-frame prediction, a first lookup table can be used, and when the size of the current block is greater than 4×4, or when the current block is encoded by intra-frame prediction, a second lookup table can be used.

[0472] Alternatively, information indicating one of the plurality of lookup tables may be signaled in a bitstream, and the decoding device may select a lookup table for the current block based on the information.

[0473] In another example, the index assigned to the transform type combination candidate may be adaptively determined based on at least one of the size, shape, prediction coding mode or intra prediction mode of the current block, whether the second transform is applied, or whether transform skipping is applied to a neighboring block. In the example, the index assigned to the transform skip when the size of the current block is 4×4 may be less than the index assigned to the transform skip when the size of the current block is greater than 4×4. Specifically, when the size of the current block is 4×4, index 0 may be assigned to the transform skip, and when the current block is greater than 4×4 and equal to or less than 16×16, an index greater than 0 (e.g., index 1) may be assigned to the transform skip. When the current block is greater than 16×16, a maximum value (e.g., 5) may be assigned to the index of the transform skip.

[0474] Alternatively, when the current block is encoded by inter prediction, an index of 0 may be assigned to the transform skip. When the current block is encoded by intra prediction, an index greater than 0 (eg, index 1) may be assigned to the transform skip.

[0475] Alternatively, when the current block is a 4×4 sized block encoded by inter-frame prediction, an index of 0 may be assigned to transform skip. On the other hand, when the current block is not encoded by inter-frame prediction, or when the current block is larger than 4×4, an index greater than 0 (e.g., index 1) may be assigned to transform skip.

[0476] Transform type combination candidates other than the transform type combination candidates listed in Table 7 may also be used. In an example, a transform type combination candidate including a transform skip applied to one of a horizontal direction transform or a vertical direction transform and a transform kernel (e.g., DCT2, DCT8, or DST7, etc.) applied to the other may be used. In this case, whether the transform skip is to be used as a transform type candidate in the horizontal direction or the vertical direction may be determined based on at least one of a size (e.g., width and / or height), a shape, a prediction coding mode, or an intra prediction mode of the current block.

[0477] Information indicating whether index information for determining a transform type of a current block is explicitly signaled may be signaled in a bitstream. In an example, information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for a block encoded by intra prediction and / or information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for a block encoded by inter prediction may be signaled at a sequence level.

[0478] When explicit transform type determination is allowed, the transform type of the current block may be determined based on the index information tu_mts_idx signaled in the bitstream. On the other hand, when explicit transform type determination is not allowed, the transform type may be determined based on at least one of the size or shape of the current block, whether transform is allowed to be performed in units of subblocks, the position of the subblock including the non-zero transform coefficient, whether the second transform is performed, or whether the sub-partition intra coding method is applied. In an example, the horizontal transform type of the current block may be determined based on the width of the current block, and the vertical transform type of the current block may be determined based on the height of the current block. For example, when the width of the current block is less than 4 or greater than 16, the horizontal transform type may be determined as DCT2. Otherwise, the horizontal transform type may be determined as DST7. When the height of the current block is less than 4 or greater than 16, the vertical transform type may be determined as DCT2. Otherwise, the vertical transform type may be determined as DST7. In this case, a threshold to be compared with the width and the height may be determined based on at least one of the size, shape, or intra prediction mode of the current block to determine the horizontal transform type and the vertical transform type.

[0479] Optionally, when the current block has a square shape with the same height and width, the horizontal direction transform type and the vertical direction transform type may be set to be the same, but when the current block has a non-square shape with a height and a width different from each other, the horizontal direction transform type and the vertical direction transform type may be set to be different. In the example, when the width of the current block is greater than the height, the horizontal direction transform type may be determined as DST7, and the vertical direction transform type may be determined as DCT2. When the height of the current block is greater than the width, the vertical direction transform type may be determined as DST7, and the horizontal direction transform type may be determined as DCT2.

[0480] Depending on whether explicit transform type determination is allowed, the number and / or type of transform type candidates or the number and / or type of transform type combination candidates may be different. In an example, when explicit transform type determination is allowed, DCT2, DST7, and DCT8 may be used as transform type candidates. Therefore, each of the horizontal direction transform type and the vertical direction transform type may be set to DCT2, DST8, or DCT8. When explicit transform type determination is not allowed, only DCT2 and DST7 may be used as transform type candidates. Therefore, each of the horizontal direction transform type and the vertical direction transform type may be determined to be DCT2 or DST7.

[0481] A coding block or a transform block may be partitioned into a plurality of sub-blocks, and transform may be performed only on a portion of the plurality of sub-blocks. Applying transform only to a portion of the plurality of sub-blocks may be defined as a sub-transform block encoding method.

[0482] Fig.32 and Fig.33 is a diagram illustrating application aspects of the sub-transform block coding method.

[0483] Fig.32 is a diagram showing an example in which transformation is performed on only one sub-block among four sub-blocks, Fig.33 is a diagram showing an example of performing transformation on only one of the two sub-blocks. Fig.32 and Fig.33 In , it is assumed that the transformation is performed only on the sub-blocks marked with "target".

[0484] As in Fig.32 In the example shown, after partitioning the coding block into 4 sub-blocks by using mutually orthogonal vertical and horizontal lines, transformation and quantization may be performed on only one of the sub-blocks. Transform coefficients in the sub-block where transformation is not performed may be set to 0.

[0485] Optionally, as in Fig.33 In the example shown, after the coding block is divided into 2 sub-blocks by using vertical lines or horizontal lines, transformation and quantization may be performed on only one of the sub-blocks. Transform coefficients in the sub-block where transformation is not performed may be set to 0.

[0486] Information indicating whether a sub-transform block encoding method is applied to a coding block may be signaled in a bitstream. The information may be a 1-bit flag cu_sbt_flag. When the flag is 1, it indicates that the transform is performed only on a portion of a plurality of sub-blocks generated by partitioning a coding block or a transform block, and when the flag is 0, it indicates that the transform is performed without partitioning the coding block or the transform block into sub-blocks.

[0487] Whether the sub-transform block encoding method is available for the coding block may be determined based on the size, shape or prediction coding mode of the coding block, or whether the combined prediction mode is used for at least one of the coding blocks. In an example, the sub-transform block encoding method is available for the coding block when at least one of the width or height of the coding block is equal to or greater than a threshold, the inter-frame prediction is applied to the coding block, or the combined prediction mode is not applied to the coding block. In this case, the threshold value may be a natural number such as 4, 8, or 16.

[0488] Optionally, when the ratio between the width and height of the coding block is greater than a threshold, the sub-transform block coding method may not be allowed to be applied.

[0489] When intra prediction is applied to a coding block or when an intra block copy mode is applied, it may be determined that the sub-transform block encoding method is unavailable.

[0490] Alternatively, whether the sub-transform block encoding method is available for the encoding block may be determined based on whether the sub-partition intra encoding method is applied to the encoding block. In an example, when the sub-partition intra encoding method is applied, it may be determined that the sub-transform block encoding method is available.

[0491] When it is determined that the sub-transform block coding method is available for the coding block, the syntax cu_sbt_flag may be signaled in the bitstream. According to the value of the parsed cu_sbt_flag, it may be determined whether the sub-transform block coding method is applied.

[0492] On the other hand, when it is determined that the sub-transform block encoding method is not applicable to the coding block, the signaling of the syntax cu_sbt_flag may be omitted. When the signaling of the syntax cu_sbt_flag is omitted, it may be determined that the sub-transform block encoding method is not applied to the coding block.

[0493] When the sub-transform coding method is applied to the coding block, information indicating the partition shape of the coding block can be sent by signaling in the bitstream. The information indicating the partition shape of the coding block may include at least one of information indicating whether the coding block is partitioned into sub-blocks of 1 / 4 size, information indicating the partition direction of the coding block, or information indicating the number of sub-blocks.

[0494] In an example, when the syntax cu_sbt_flag is 1, a flag cu_sbt_quadtree_flag indicating whether the coding block is partitioned into sub-blocks of ¼ size may be signaled.

[0495] When the syntax cu_sbt_quadtree_flag is 1, this indicates that the coding block is partitioned to include sub-blocks of 1 / 4 size. In an example, the coding block may be partitioned into sub-blocks of 1 / 4 size and sub-blocks of 3 / 4 size, or may be partitioned into four sub-blocks each having a 1 / 4 size. When the coding block is partitioned to include sub-blocks of 1 / 4 size, this may be referred to as quadtree partitioning. When the syntax cu_sbt_quad_tree_flag is 1, a sub-block of 1 / 4 size of the coding block may be set as a transform target.

[0496] When the syntax cu_sbt_quadtree_flag is 0, it indicates that the coding block is partitioned into sub-blocks of 1 / 2 size. In an example, the coding block can be partitioned into 2 1 / 2-sized sub-blocks by using 1 vertical line or 1 horizontal line. Splitting the coding block into 2 1 / 2-sized sub-blocks may be referred to as binary tree partitioning. When the value of the syntax cu_sbt_quad_tree_flag is 0, 1 / 2-sized sub-blocks of the coding block may be included in the coding block.

[0497] In addition, a flag indicating the partition direction of the coding block can be signaled in the bitstream. In the example, a flag cu_sbt_horizontal_flag indicating whether horizontal partitioning is applied to the coding block can be encoded and signaled. When the value of cu_sbt_horizontal_flag is 1, it indicates that horizontal partitioning using at least one partition line parallel to the top and bottom of the coding block is applied. When the value of cu_sbt_horizontal_flag is 0, it indicates that vertical partitioning using at least one partition line parallel to the left and right sides of the coding block is applied.

[0498] According to the size or shape of the coding block, the partition shape of the coding block may be determined. In an example, when at least one of the width or height of the coding block is equal to or greater than a first threshold, the quadtree partition may be available. In an example, the first threshold may be a natural number such as 4, 8, or 16. The first threshold may be referred to as a quadtree threshold.

[0499] When it is determined that quadtree partitioning is available, the syntax cu_sbt_quadtree_flag may be signaled in the bitstream. Depending on the parsed value of cu_sbt_quadtree_flag, it may be determined whether quadtree partitioning is applied to the coding block.

[0500] When it is determined that quadtree partitioning is not available, the signaling of the syntax cu_sbt_quadtree_flag may be omitted. When the signaling of the syntax cu_sbt_quadtree_flag is omitted, it may be determined that binary tree partitioning is applied to the coding block.

[0501] Table 8 shows a syntax structure for determining whether to parse the syntax cu_sbt_quadtree_flag.

[0502]

Table 8

[0503]

[0504]

[0505] In Table 8, the variable allowSbtVerQ indicates whether quadtree partitioning in the vertical direction is allowed, and the variable allowSbtHorQ indicates whether quadtree partitioning in the horizontal direction is allowed. The variables allowSbtVerQ and allowSbtHorQ may be determined based on the quadtree threshold. In an example, when the quadtree threshold is 16, allowSbtVerQ may be determined based on whether the width of the coding block is equal to or greater than 16, and allowSbtHorQ may be determined based on whether the height of the coding block is equal to or greater than 16.

[0506] As in the example shown in Table 8, when all variables allowSbtVerQ and allowSbtHorQ are true, the syntax cu_sbt_quad_flag can be parsed from the bitstream. In the example, when the coding block is 16×8, the variable allowSbtHorQ is set to false, so the parsing of the syntax cu_sbt_quad_flag can be omitted. Alternatively, when the coding block is 8×16, the variable allowSbtVerQ is set to false, so the parsing of the syntax cu_sbt_quad_flag can be omitted. When the parsing of the syntax cu_sbt_quad_flag is omitted, binary tree partitioning can be applied to the coding block.

[0507] Alternatively, unlike the example shown in Table 8, the syntax cu_sbt_quad_flag may be parsed when either the variable allowSbtVerQ or the variable allowSbtHorQ is true. In other words, quadtree partitioning may be available when only either one of the width and height of the coding block is equal to or greater than the quadtree threshold.

[0508] Optionally, although either the width or height of the coding block is equal to or greater than the first threshold, when the other of the width or height of the coding block is equal to or less than the second threshold, it can be determined that the quadtree partition of the coding block is not available. In this case, the second threshold may have a value less than the first threshold. In an example, the second threshold may be a natural number such as 2, 4, or 8.

[0509] The variable allowSbtHorH indicates whether the binary tree partition in the horizontal direction is available. When the height of the coding block is equal to or greater than the threshold, the binary tree partition in the horizontal direction can be set to be available. The variable allowSbtVerH indicates whether the binary tree partition in the vertical direction is available. When the width of the coding block is equal to or greater than the threshold, the binary tree partition in the vertical direction can be set to be available. In this case, the threshold can be a natural number such as 4, 8, or 16.

[0510] When both quadtree / binary tree partitioning in horizontal direction and quadtree / binary tree partitioning in vertical direction are available, the syntax cu_sbt_horizontal_flag may be signaled in the bitstream. Depending on the value of the syntax cu_sbt_horizontal_flag, partitioning in horizontal direction or partitioning in vertical direction may be applied to the coding block.

[0511] On the other hand, when only one of the quadtree / binary tree partition in the horizontal direction and the quadtree / binary tree partition in the vertical direction is available, the signaling of the syntax cu_sbt_horizontal_flag can be omitted. When the signaling of the syntax cu_sbt_horizontal_flag is omitted, the available one of the quadtree / binary tree partition in the horizontal direction and the quadtree / binary tree partition in the vertical direction can be applied.

[0512] When binary tree partitioning in the horizontal direction or binary tree partitioning in the vertical direction is not available, the signaling of the syntax cu_sbt_horizontal_flag can be omitted.

[0513] When the signaling of the syntax cu_sbt_horizontal_flag is omitted and the variable allowSbtHorH is true, binary tree partitioning in the horizontal direction may be applied to the coding block.

[0514] When the signaling of the syntax cu_sbt_horizontal_flag is omitted and the variable allowSbtVerH is true, binary tree partitioning in the vertical direction may be applied to the coding block.

[0515] Information for specifying a sub-block as a transform target among a plurality of sub-blocks may be signaled in a bitstream. In an example, the syntax cu_sbt_pos_flag may be signaled in a bitstream. The syntax cu_sbt_pos_flag indicates whether the transform target is the first sub-block in the coding block. In the example, when quadtree / binarytree partitioning in the horizontal direction is applied to the coding block, when cu_sbt_flag is 1, the leftmost sub-block is determined as the transform target, and when cu_sbt_flag is 0, the rightmost sub-block is determined as the transform target. When quadtree / binarytree partitioning in the vertical direction is applied to the coding block, when cu_sbt_pos_flag is 1, the topmost sub-block is determined as the transform target, and when cu_sbt_pos_flag is 0, the bottommost sub-block is determined as the transform target.

[0516] The transform type of the subblock may be determined by considering the partition direction of the coding block and the position of the subblock. In an example, when the coding block is partitioned in the vertical direction and a transform is performed on a subblock located at a left position among the subblocks, the horizontal transform type and the vertical transform type may be set to be different.

[0517] Fig.34 and Fig.35 The horizontal direction transform type and the vertical direction transform type according to the position of the subblock as the transform target are shown.

[0518] exist Fig.34In the example shown, when the sub-block as the transform target includes the upper left sample or the lower right sample of the coding block, the horizontal transform type and the vertical transform type can be set to be the same. Fig.34 In the example shown in FIG. 1 , it is shown that when the sub-block as the target of the transform includes the upper left sample of the coding block, the horizontal transform type and the vertical transform type are set to DCT8, and when the sub-block as the target of the transform includes the lower right sample of the coding block, the horizontal transform type and the vertical transform type are set to DST7.

[0519] When the sub-block as the transform target includes the upper right sample or the lower left sample of the coding block, the horizontal transform type and the vertical transform type can be set to be different. Fig.34 The example shown shows that when the sub-block as the transform target includes the upper right sample of the coding block, the horizontal transform type is set to DST7, and the vertical transform type is set to DCT8. When the sub-block as the transform target includes the lower left sample of the coding block, the horizontal transform type is set to DCT8, and the vertical transform type is set to DST7.

[0520] and Fig.34 Different from the example shown, when a sub-block including an upper left sample or a sub-block including a lower right sample in the coding block is determined as a transform target, the horizontal transform type and the vertical transform type may be set to be different, and when a sub-block including an upper right sample or a sub-block including a lower left sample in the coding block is determined as a transform target, the horizontal transform type and the vertical transform type may be set to be the same.

[0521] Fig.34 It is shown that a sub-block whose height and width are 1 / 2 of the coding block is set as a transformation target. Different from the example shown, a sub-block whose width is the same as the coding block but whose height is 1 / 4 of the coding block or a sub-block whose height is the same as the coding block but whose width is 1 / 4 of the coding block may be set as a transformation target.

[0522] exist Fig.35 In the example shown, when the sub-block as the transform target includes the upper left sample of the coding block, the horizontal transform type and the vertical transform type can be set to be different. Fig.35 In the example shown in , when binary tree partitioning in the horizontal direction is applied and the upper sub-block is determined as the transform target, the horizontal transform type may be set to DST7, and the vertical transform type may be set to DCT7. When binary tree partitioning in the vertical direction is applied and the left sub-block is determined as the transform target, the horizontal transform type may be set to DCT8, and the vertical transform type may be set to DST7.

[0523] and Fig.35 Different from the example shown, when the sub-block as the transform target includes the upper left sample of the coding block, the horizontal transform type and the vertical transform type can be set to the same, and when the sub-block as the transform target includes the lower right sample of the coding block, the horizontal transform type and the vertical transform type can be set to different.

[0524] When the sub-block as the transform target includes the lower right sample of the coding block, the horizontal transform type and the vertical transform type may be set to be the same. Fig.35 In the example shown in , when binary tree partitioning in the horizontal direction is applied and the lower sub-block is determined as the transform target, the horizontal transform type and the vertical transform type may be set to DST7. When binary tree partitioning in the vertical direction is applied and the right sub-block is determined as the transform target, the horizontal transform type and the vertical transform type may be set to DST7.

[0525] As in the above example, whether to set the horizontal transform type and the vertical transform type to be the same may be determined according to the position of the subblock as the transform target in the coding block. In addition, the horizontal transform type and the vertical transform type may be determined according to the position of the subblock as the transform target in the coding block.

[0526] For a sub-block, encoding of information indicating whether there is a non-zero coefficient (e.g., CBF) may be omitted. When encoding of the CBF is omitted, whether a non-zero residual coefficient is included in each sub-block may be determined based on the position of the block where the transform is performed. In the example, when a sub-block at the right or lower position in a coding block to which binary tree partitioning is applied is determined as a transform target, it may be inferred that the CBF value of the sub-block at the left or upper position is 0, and it may be inferred that the CBF value of the sub-block at the right or lower position is 1. Optionally, when a sub-block at the left or lower position in a coding block to which binary tree partitioning is applied is determined as a transform target, it may be inferred that the CBF value of the sub-block at the left or upper position is 1, and it may be inferred that the CBF value of the sub-block at the right or lower position is 0.

[0527] The second transform may be performed on the block to which the first transform is performed. The second transform may be performed on an upper left region in the transform block to which the first transform has been applied.

[0528] If the residual coefficients on which the first transform and the second transform have been performed are encoded, the decoding apparatus may perform the second inverse transform as the inverse process of the second transform on the transform block, and perform the first inverse transform as the inverse process of the first transform on the transform block on which the second inverse transform has been performed.

[0529] Whether to apply the second transform to the current block may be determined based on at least one of the size of the current block, the number of residual coefficients, the coding mode or the intra prediction mode, or whether a sub-partition intra coding method is applied. The encoder may encode information indicating whether the second transform is applied and signal the information to the decoder. Alternatively, the encoder and the decoder may determine whether to perform the second transform based on the same condition.

[0530] In an example, information indicating whether to perform the second transform may be signaled in a bitstream. Specifically, a flag indicating whether to perform the second transform or index information specifying whether to perform the second transform and a transform core for the second transform may be signaled.

[0531] Table 9 shows an example of signaling a flag lfnst_flag indicating whether to perform the second transform in a bitstream. When the value of the flag lfnst_flag is 0, this indicates that the second transform is not performed on the current block. On the other hand, when the value of the flag lfnst_flag is 1, this indicates that the second transform is performed on the current block.

[0532]

Table 9

[0533]

[0534] Optionally, the syntax lfnst_idx may be signaled in the bitstream. When the value of the index lfnst_idx is 0, it indicates that the second transform is not performed on the current block. On the other hand, when the index lfnst_idx is greater than 0, it indicates that the second transform is performed on the current block. When the value of the index lfnst_idx is greater than 0, lfnst_idx may be used to specify a transform core for performing the second transform.

[0535] Alternatively, whether to perform the second transformation may be determined by comparing at least one of the width or height of the current block with a threshold. In an example, when the minimum value of the width and height of the current block is less than the threshold, the second transformation may not be performed. In this case, the threshold may be a natural number such as 4, 8 or 16.

[0536] Alternatively, when the current block is encoded by inter prediction, the second transform may not be applied.

[0537] Alternatively, although the current block is encoded by intra prediction, the second transform may not be applied when matrix-based intra prediction is performed.

[0538] Alternatively, whether to perform the second transformation may be determined based on whether the transformation kernel in the horizontal direction is the same as the transformation kernel in the vertical direction. In an example, the second transformation may be performed only when the transformation kernel in the horizontal direction is the same as the transformation kernel in the vertical direction. Alternatively, the second transformation may be performed only when the transformation kernel in the horizontal direction is different from the transformation kernel in the vertical direction.

[0539] Alternatively, the second transform may be allowed only when a predefined transform kernel is used for the horizontal direction transform and the vertical direction transform. In an example, the second transform may be allowed when a DCT2 transform kernel is used for the horizontal direction transform and the vertical direction transform.

[0540] Alternatively, when the subpartition intra encoding method is applied to the current block, the second transform may be allowed only when a DCT2 transform kernel is used for transform in the horizontal direction and transform in the vertical direction.

[0541] Alternatively, whether to perform the second transform may be determined based on the number of non-zero transform coefficients of the current block. In an example, when the number of non-zero transform coefficients of the current block is less than a threshold or is the same as the threshold, the second transform may be set not to be used, and when the number of non-zero transform coefficients of the current block is greater than the threshold, the second transform may be set to be used. The second transform may be allowed to be used only when the current block is encoded by intra prediction.

[0542] Alternatively, it may be determined whether to perform the second transform based on the position of the last non-zero transform coefficient of the current block. In an example, when at least one of the x-axis coordinate or the y-axis coordinate of the last non-zero transform coefficient of the current block is greater than a threshold, or when at least one of the x-axis coordinate or the y-axis coordinate of the sub-block to which the last non-zero transform coefficient of the current block belongs is greater than a threshold, the second transform may not be performed. In this case, the threshold may be predefined in the encoder and the decoder. Alternatively, the threshold may be determined based on the size or shape of the current block.

[0543] Optionally, when there is only a transform coefficient of a DC component in the current block, it may be set not to perform the second transform. In this case, the DC component represents a transform coefficient at an upper left position in the current block.

[0544] Whether to perform the second transform may be determined based on whether the combined prediction encoding mode is applied to the current block. In an example, when the combined prediction encoding mode is applied to the current block, it may be set not to perform the second transform.

[0545] Alternatively, when the combined prediction encoding mode is applied, whether to perform the second transform may be determined based on at least one of the size or shape of the current block, the intra prediction mode, or the weight.

[0546] Optionally, when the combined prediction coding mode is applied, it may be configured to signal information indicating whether the second transformation is performed on the current block. For example, when the combined prediction coding mode is applied to the current block, the index lfnst_idx may be signaled.

[0547] Alternatively, it may be determined whether to perform the second transform regardless of whether the combined prediction encoding mode is applied.

[0548] Alternatively, whether the second transform is allowed may be determined based on whether the sub-partition intra coding method is applied to the current block. In an example, when the sub-partition intra coding method is applied to the current block, it may be set not to apply the second transform to the current block.

[0549] Optionally, when the sub-partition intra-frame coding method is applied to the current block, it can be set to apply the second transform. For example, when the vertical partition or the horizontal partition is applied to the current block, the index information indicating whether the second transform is applied to the current block can be sent by a signal. Whether to apply the second transform and / or the transform kernel for the second transform can be determined by the index information. On the other hand, when the sub-partition intra-frame coding method is not applied to the current block, the encoding of the index information can be omitted. Optionally, when the sub-partition intra-frame coding method is not applied to the current block, it can be determined whether to encode the index information indicating whether the second transform is applied based on whether a preset condition is met. In this case, the preset condition can be related to at least one of the position of the non-zero coefficient, the number of non-zero coefficients, or the size of the current block. In the example, when the number of non-zero coefficients is equal to or greater than 1, when the number of non-zero coefficients is 1 but is not included in the 4×4 upper left area of ​​the current block, or when the scanning order of the non-zero coefficients (the non-zero coefficients are not included in the 4×4 area of ​​the current block) is equal to or less than a threshold, the index information can be encoded and the index information can be sent by a signal. Otherwise, the encoding of the index information can be omitted.

[0550] When the index information is not signaled, it may be inferred that the value of the index information indicates that the second transform is not applied.

[0551] Optionally, whether to apply the second transform may be determined based on at least one of the size, width, height, or shape of the sub-block. In the example, when at least one of the size, width, or height of the sub-block is less than a threshold, it may be set not to apply the second transform. Specifically, when at least one of the size, width, or height of the sub-block is less than a threshold, encoding of the index information may be omitted. Information indicating whether to perform the second transform may be signaled at the coding block level. Whether to apply the second transform to a sub-block belonging to the coding block may be determined based on information signaled at the coding block level.

[0552] The threshold value may have a predefined value in the encoder and the decoder. In an example, the threshold value may be set to 2, 4, or 8, etc. Alternatively, the threshold value may be determined based on at least one of a partition direction of the current block or the number of subblocks included in the current block.

[0553] Alternatively, whether to encode the syntax indicating whether to apply the second transform may be determined by using parameters (listed above) related to the current block for determining whether to apply the second transform.

[0554] In an example, when at least one of the size of the current block, the number of residual coefficients, the coding mode or the intra prediction mode, or whether the sub-partition intra coding method is applied does not meet the preset condition, the encoding of the syntax indicating whether the second transform is applied may be omitted. In an example, the encoding of lfnst_flag or lfnst_idx may be omitted, and its value may be inferred to be 0. In other words, when the encoding of the syntax is omitted, the second transform may not be applied.

[0555] When intra BDPCM is applied to the current block, the second transform may not be performed. When intra BDPCM is applied to the current block, the signaling of the syntax element indicating whether the second transform is performed may be omitted, and its value may be derived as 0. Table 10 shows an example of omitting the signaling of lfnst_idx for a block to which intra BDPCM is applied.

[0556]

Table 10

[0557]

[0558] Optionally, when intra BDPCM is applied to the current block, the signaling of the syntax element (e.g., transform_skip_flag) indicating whether to perform transform skipping may be omitted. When intra BDPCM is applied to the current block, the signaling of the flag transform_skip_flag may be omitted, and its value may be inferred to be 1. In other words, when intra BDPCM is applied, transform may not be applied to the current block.

[0559] When it is determined that the transform is not applied to the current block, both the first transform and the second transform may not be applied. Therefore, when the transform of the current block is skipped, the signaling of the syntax element indicating whether the second transform is performed may be omitted, and its value may be inferred to be 0. Finally, when intra BDPCM is applied to the current block, the signaling of the syntax element lfnst_idx indicating whether the second transform is performed may be omitted.

[0560] Based on the above description, a method of performing the second transform in an encoder and a decoder will be described in detail.

[0561] The second transform may be performed for the upper left region in the current block. The target region to which the second transform is applied may have a predefined size or a predefined shape. The target region to which the second transform is applied may have a square block shape such as 4×4 or 8×8 or a non-square block shape such as 4×8 or 8×4.

[0562] Optionally, when the current block is evenly partitioned into N regions, at least one of the N regions may be set as the target region. In this case, N may be a natural number such as 2, 4, 8, or 16. The variable N may be predefined in the encoder and the decoder. Optionally, the variable N may be determined based on the size and / or shape of the current block.

[0563] Alternatively, the target region may be determined based on the number of transform coefficients. In an example, a predetermined number of transform coefficients may be determined as the target region according to a predetermined scanning order.

[0564] Optionally, information for specifying the size and / or shape of the target region may be encoded and sent in a bitstream. The information may include at least one of information indicating the size of the target region or information indicating the number of 4×4 blocks included in the target region.

[0565] Alternatively, the entire current block may be set as the target region. In an example, when the size of the current block is the same as the minimum size (eg, 4×4) of the target region, the entire current block may be set as the target of the second transform.

[0566] The second transform may be applied in an inseparable shape. Therefore, the second transform may also be referred to as a non-separable secondary transform (NSST).

[0567] Transform coefficients in the region where the second transform is applied may be arranged in a single column. In an example, when the second transform is performed on an N×N sized target region, the transform coefficients included in the target region may be converted into N 2 When a block of 4×4 size is set as the target region, the transform coefficients included in the target region may be converted into an input matrix of 16×1 size. When a block of 8×8 size is set as the target region, the transform coefficients included in the target region may be converted into an input matrix of 64×1 size.

[0568] The non-separable transform matrix may be applied to an input matrix generated by arranging transform coefficients included in a target region into rows. The size of the non-separable transform matrix may be variously determined according to the size of the input matrix.

[0569] In the example, when the input matrix size is N 2 ×1, based on N 2 ×N 2For example, when the size of the input matrix is ​​16×1, a non-separable transform matrix of 16×16 size may be used, and when the size of the input matrix is ​​64×1, a non-separable transform matrix of 64×64 size may be used.

[0570] A plurality of inseparable transform matrices may be stored in the encoder and the decoder. Information for specifying any one of the plurality of inseparable transform matrices may be signaled in a bitstream.

[0571] Alternatively, the non-separable transform matrix may be specified based on at least one of a size, a shape, a quantization parameter, an intra prediction mode, or a transform type of the current block used in the first transform.

[0572] Optionally, an inseparable transform matrix candidate that can be used by the current block may be specified based on at least one of a size, a shape, a quantization parameter, an intra prediction mode, or a transform type of the current block used in the first transform. When there are multiple inseparable transform matrix candidates that can be used by the current block, information indicating one of the multiple inseparable transform matrix candidates may be encoded and signaled.

[0573] The transformation matrix may be obtained by multiplying the non-separable transformation matrix by the input matrix. In an example, Equation 6 shows an example of obtaining the transformation matrix A'.

[0574] [Equation 6]

[0575] A′=T*A

[0576] In Equation 6, T represents the non-separable transformation matrix, and A represents the input matrix. When the size of the matrix T is N 2 ×N 2 And the size of matrix A is N 2 ×1, you can get N 2 In the example, when a 16×1 input matrix and a 16×16 non-separable transformation matrix are used, a 16×1 transformation matrix A' can be obtained. Alternatively, when a 64×1 input matrix and a 64×64 non-separable transformation matrix are used, a 64×1 transformation matrix A' can be obtained.

[0577] When the transformation matrix A' is obtained, the components in the transformation matrix A' may be set to the transformation coefficients of the N×N size block in the current block. The transformation coefficients in the remaining area excluding the N×N size block may be set to default values. In an example, the transformation coefficients in the area where the second transformation is not performed may be set to 0.

[0578] The second transformation may be performed by using an inseparable transformation matrix in which the number of rows is smaller than the number of columns. 2 )-sized non-separable transformation matrix is ​​applied to (N 2 ×1) size. In this case, k can have a size smaller than N 2 In this example, k can be N 2 / 2、N 2 / 4 or 3N 2 / 4, etc. k can be called a simplification factor.

[0579] Therefore, a transformation matrix of a (k×1) size smaller than the input matrix can be obtained. Thus, the second transform of the transformation matrix whose output size is smaller than the input matrix can be called a simplified secondary transform.

[0580] Equation 7 represents an example of applying the simplified quadratic transform.

[0581] [Equation 7]

[0582] A R =R*A

[0583] In Equation 7, R represents k×N 2 size. k (the number of rows) is less than N 2 A non-separable transformation matrix with a number of columns can be called a simplified non-separable transformation matrix. R Represents a transformation matrix of size k×1. The transformation matrix A has a size smaller than the input matrix A R It can be called a simplified transformation matrix.

[0584] When the simplified transformation matrix A is obtained R When the simplified transformation matrix A R The components in are set to the transform coefficients of at least one or more M×M-sized blocks in the current block. In this case, M may be a natural number less than N. The number of M×M-sized blocks may be determined according to the simplification factor k. The transform coefficients of the remaining area excluding at least one M×M-sized block may be set to a default value. In an example, the transform coefficients in the remaining area may be set to 0.

[0585] Fig.36 is a diagram showing aspects of encoding of transform coefficients when the simplification factor is 16.

[0586] Transform coefficients included in a target region of an 8×8 size may be transformed into an input matrix of a 64×1 size, and a transform matrix of a 16×1 size may be obtained by using a non-separable transform matrix of a 16×64 size.

[0587] A transform matrix of a 16×1 size may be set as a transform coefficient of a 4×4 block, and transform coefficients in other regions may be set to zero.

[0588] Not shown, but when the simplification factor k is 32, a transform matrix of 32×1 size may be set to transform coefficients of an 8×4 block or a 4×8 block, and transform coefficients in other regions may be set to 0.

[0589] When the simplification factor k is 48, a transform matrix of a size of 48×1 may be set to transform coefficients of three 4×4 blocks, and transform coefficients in other regions may be set to 0. Specifically, the transform matrix may be set to transform coefficients of a 4×4 block at an upper left position of the current block, a 4×4 block adjacent to the right of the upper left block, and a 4×4 block adjacent to the lower side of the upper left block.

[0590] The transform matrix may be determined based on at least one of a size or a shape of the current block or an intra prediction mode. In an example, based on the intra prediction mode of the current block, a transform matrix set may be determined, and one of a plurality of transform matrix candidates included in the transform matrix set may be selected. Index information specifying a transform matrix applied to the current block from among the plurality of transform matrix candidates may be encoded and signaled.

[0591] When the remaining transform coefficients excluding the transform coefficients generated by the second transform are set to 0, the decoder may determine whether to perform the second transform based on the position of the last non-zero residual coefficient. In an example, when the last residual coefficient is located outside the block storing the transform coefficients generated by the second transform, it may be determined that the second transform is not performed. In other words, the decoder may perform an inverse transform for the second transform only when the last residual coefficient is located in the block storing the transform coefficients generated by the second transform.

[0592] Whether to perform a simplified second transform may be determined based on at least one of the size or shape of the current block. In an example, when at least one of the width or height of the current block is greater than a threshold, a simplified second transform may be applied, otherwise, a general second transform may be applied. In this case, the threshold may be a natural number such as 4, 8, or 16.

[0593] Alternatively, whether to perform a simplified second transform may be determined according to the size of the target area. In an example, when the second transform is performed for a target area of ​​4×4 size, a general second transform may be applied. In an example, for a target area of ​​4×4 size, the second transform may be performed by using a non-separable transform matrix of 16×16 size.

[0594] On the other hand, when the second transform is performed on a target region of 8×8 size, a simplified second transform may be applied. In an example, for a target region of 8×8 size, the second transform may be performed by using a non-separable transform matrix of 48×64, 32×64 or 16×64 size.

[0595] When performing an inverse transform for the simplified second transform, the size of the output matrix has a value greater than the size of the input matrix. In an example, when the simplification factor k is 16, an output matrix of 64×1 size can be obtained by performing an inverse transform on an input matrix of 16×1 size.

[0596] When the sub-partition intra encoding method is applied to the coding block, the coding block may be partitioned into a plurality of sub-blocks. When the sub-partition intra encoding method is applied to the coding block, it may be set not to apply the second transform.

[0597] Alternatively, whether to perform the second transform may be determined based on the shape or size of the sub-partition. In the example, when the coding block is partitioned into sub-partitions with a width or height of 4, the second transform may be applied. In other words, the second transform may be applied only when the sub-partition has a 4×L or L×4 shape. In this case, L represents an integer equal to or greater than 4.

[0598] Optionally, the second transformation may be applied only when the minimum of the width and height of the sub-partition is equal to or greater than a predefined threshold. In this case, the threshold may be an integer such as 4, 8 or 16.

[0599] As in the above example, when the shape or size of the sub-partition satisfies the predefined condition, information indicating whether the second transform is applied can be signaled in the bitstream. In the example, when the sub-partition is 4×L or L×4, or when the minimum value of the width and height of the sub-partition is equal to or greater than the threshold, the syntax lfnst_idx indicating whether the second transform is performed can be signaled.

[0600] On the other hand, when the shape or size of the sub-partition does not meet the predefined condition, the encoding / decoding of the information indicating whether the second transformation is applied can be omitted. In the example, when the minimum value of the width and height of the sub-partition is less than the threshold, the encoding / decoding of the syntax lfnst_idx can be omitted. When the encoding / decoding of the syntax lfnst_idx is omitted, its value can be derived as 0.

[0601] When it is determined to apply the second transform, the second transform may be applied to a subblock at a predefined position in the coding block or a subblock whose partition index is less than a threshold. In this case, the partition index of the left subblock may be set to a value having a smaller partition index than the right subblock, or the partition index of the upper subblock may be set to a value having a smaller partition index than the lower subblock. In an example, the second transform may be applied only to the first subblock in the coding block.

[0602] Optionally, the second transform may be applied to each of all sub-blocks.

[0603] Optionally, when it is determined to perform the second transform at the coding block level, it may be adaptively determined whether to apply the second transform to each sub-block based on the attribute of each sub-block. In this case, the attribute of the sub-block may include at least one of the number of residual coefficients included in the sub-block, whether to skip the transform application to the sub-block, or whether to apply the transform kernel to the sub-block.

[0604] The size of the target region for the second transform may be determined based on the size of the sub-block. In an example, a 4×4-sized block including 16 samples, two 4×4-sized blocks including 32 samples, three 4×4-sized blocks including 48 samples, or four 4×4-sized blocks including 64 samples may be set as the target region for the second transform. Depending on the size of the target region, a general second transform may be applied, or a simplified second transform may be applied.

[0605] The size of the target area for the second transformation may be determined based on the size of the sub-block. In an example, when at least one of the width or height of the sub-block is less than a threshold value, an area including N samples may be set as the target area for the second transformation. On the other hand, when the width and height of the sub-block are equal to or greater than the threshold value, an area including M samples may be set as the target area for the second transformation. In this case, M may be a natural number greater than N. In an example, N may be 16 or 32, and M may be 48 or 64. In addition, the threshold value may be a natural number such as 2, 4, 8, or 16.

[0606] The target region for the second transform may be set not to extend beyond the boundary of the sub-block. In other words, when the target region for the second transform spans two or more sub-blocks, the second transform may be set not to be performed.

[0607] Fig.37 and Fig.38 is a diagram showing a target area for a second transformation.

[0608] When partitioning in the horizontal direction is applied to a 16×16 sized coding block, the coding block may be partitioned into 16×4 sized sub-blocks. As in the example shown, when the sub-block has an N×4 size (N is an integer greater than 4), the height of the target region for the second transform may be set to not more than 4.

[0609] In other words, as in Fig.37 In the example shown in (a), the target region for the second transform may be set to a region of 4×4 or 8×4 size.

[0610] On the other hand, as in Fig.37 In the example shown in (b), setting of a target area such as 4×8 or 8×8 for the second transform may not be allowed.

[0611] When partitioning in the vertical direction is applied to a 16×16 sized coding block, the coding block may be partitioned into 4×16 sized sub-blocks. As in the example shown, when the sub-block has a 4×N size (N is an integer greater than 4), the width of the target region for the second transform may be set to not more than 4.

[0612] In other words, as in Fig.38 In the example shown in (a), the target region for the second transform may be set to a region of 4×4 or 4×8 size.

[0613] On the other hand, as in Fig.38 In the example shown in (b), setting of a target area such as 8×4 or 8×8 for the second transform may not be allowed.

[0614] The decoder may decode the residual coefficients in the bitstream and derive transform coefficients by performing inverse quantization on the residual coefficients. When the transform coefficients are generated by the first transform and the second transform, the residual samples may be derived by performing the second inverse transform and the first inverse transform on the transform coefficients.

[0615] When it is determined to perform the second transform on the current block, a target region as a target of the second inverse transform may be determined. When a general second transform is applied, the size of the target region for the second inverse transform may be set to be the same as the size of the target region for the second transform. In an example, when the second transform is performed on a 4×4 sized region by using a 16×16 sized inseparable transform matrix, the second inverse transform may be applied to the 4×4 sized region.

[0616] On the other hand, when the simplified second transform is applied, the size of the target region for the second inverse transform may have a value smaller than the size of the target region for the second transform. In an example, when the second transform is performed on an 8×8 sized region by using a simplified transform matrix of 64×48 size, the second inverse transform may be performed on a region including 48 samples (e.g., three 4×4 sized blocks).

[0617] In the decoder, the target area for the second inverse transform may be determined based on the size of the current block. In this case, the current block may represent a coding block or a transform block as a target for performing the second inverse transform. In the example, when at least one of the width or height of the current block is less than a threshold, the target area may be configured to include 16 samples. On the other hand, when the width and height of the current block are equal to or greater than the threshold, the target area may be configured to include 48 samples.

[0618] The input matrix may be generated by arranging the transform coefficients included in the target area in rows. In this case, when the simplified second transform is applied to the current block, the input matrix may be generated based on as many transform coefficients as the simplification factor k. In the example, when the simplification factor k is 16, the input matrix may be generated based on the transform coefficients included in the upper left block of 4×4 size. When the simplification factor k is 32, the input matrix may be generated based on the transform coefficients included in the upper left block and the 4×4 size neighboring blocks adjacent to the right or below the upper left block. When the simplification factor k is 48, the input matrix may be generated based on the transform coefficients of the upper left block, the 4×4 size neighboring blocks adjacent to the right of the upper left block, and the 4×4 size neighboring blocks adjacent to the below of the upper left block.

[0619] The simplification factor k may be predefined in the encoder and the decoder. Alternatively, information for determining the simplification factor k may be signaled in the bitstream. Alternatively, the simplification factor k may be determined based on the size or shape of the current block.

[0620] The transformation matrix can be obtained by multiplying the input matrix with the non-separable inverse transformation matrix. The non-separable inverse transformation matrix can be a symmetric matrix of the non-separable transformation matrix shown in Equations 6 to 7. Equations 8 and 9 show examples of obtaining the transformation matrix by using the non-separable inverse transformation matrix.

[0621] [Equation 8]

[0622] A′=T T *A

[0623] When a general second transform is applied to the current block, it can be obtained by multiplying the input matrix A by the non-separable inverse transform matrix T T In this example, the inverse transformation matrix T of size 16×16 can be derived. T Multiplying with the 16×1 input matrix A derives a 16×1 transformation matrix.

[0624] When the transformation matrix A' is obtained, the components in the transformation matrix A' may be set as the transformation coefficients of the NxN sized block in the current block. In an example, the 16x1 sized transformation matrix may be set as the transformation coefficients of the 4x4 block.

[0625] [Equation 9]

[0626] A′=R T *A

[0627] When the simplified second transform is applied to the current block, the input matrix A can be multiplied by the simplified non-separable inverse transform matrix R T The transformation matrix A' is derived. In the example, the simplified non-separable inverse transformation matrix R of size 64×16 can be obtained byT Multiplying with the 16×1 input matrix A derives a 64×1 transformation matrix.

[0628] Alternatively, the simplified inverse transformation matrix R of size 64×32 can be T Multiplying with the 32×1 input matrix A derives a 64×1 transformation matrix.

[0629] Alternatively, the simplified inverse transformation matrix R of size 64×48 can be T Multiplying with the 48×1 input matrix A derives a 48×1 transformation matrix.

[0630] When the transformation matrix A' is obtained, the components in the transformation matrix A' may be set as the transformation coefficients of the NxN sized block in the current block. In an example, the 64x1 sized transformation matrix may be set as the transformation coefficients of the 8x8 block.

[0631] The non-separable inverse transform matrix may be determined based on at least one of the index information signaled in the bitstream or the size of the current block or the intra prediction mode. In an example, the size of the non-separable inverse transform matrix may be determined based on the size of the current block, and the non-separable inverse transform matrix set may be determined based on the intra prediction mode of the current block. When the non-separable inverse transform matrix set is determined by the intra prediction mode of the current block, at least one of the multiple non-separable inverse transform matrix candidates corresponding to the size of the current block included in the non-separable inverse transform matrix set may be specified by the index lfnst_idx.

[0632] The non-separable inverse transform matrix set may include a plurality of non-separable inverse transform matrix candidates. In this case, at least one of the type or number of inverse transform matrix candidates may be different between non-separable inverse transform matrix sets having different indices.

[0633] Table 11 shows an example of determining an inseparable inverse transform set based on an intra prediction mode. In Table 11, predModeIntra represents an index of an intra prediction mode, and lfnstTrSetIdx represents an index of the inseparable inverse transform set.

[0634]

Table 11

[0635] predModeIntra lfnstTrSetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=12 1 13<=predModeIntra<=23 2 24<=predModeIntra<=44 3 45<=predModeIntra<=55 2 56<=predModeIntra<=80 1 81<=predModeIntra<=83 0

[0636] In an example, when the intra prediction mode of the current block is planar, a non-separable inverse transform matrix set with an index of 0 may be selected. Thereafter, the non-separable inverse transform matrix may be determined based on the size of the current block and the value of lfnst_idx.

[0637] The set of indivisible inverse transforms may be determined by further simplifying Table 11. In an example, Table 11 may be simplified as in Table 12.

[0638]

Table 12

[0639] predModeIntra lfnstTrSetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<50 1 50<=predModeIntra<=83 2

[0640] Alternatively, the non-separable inverse transform matrix set may be selected by only considering whether the intra prediction mode is directional or non-directional. Table 13 shows an example of determining the non-separable inverse transform matrix set based on whether the intra prediction mode is directional or non-directional.

[0641]

Table 13

[0642] predModeIntra lfnstTrSetIdx 0<=predModeIntra<=1 0 otherwise 1

[0643] Optionally, after storing multiple lookup tables defining the mapping relationship between the intra prediction mode and the non-separable inverse transform matrix set, the non-separable inverse transform matrix set can be determined by using one of the multiple lookup tables. In an example, when determining the non-separable inverse transform matrix set, one of Tables 11 to 13 can be selectively used.

[0644] The information specifying one of the plurality of lookup tables may be signaled in the bitstream. The information may be signaled at a sequence level, a picture level, a slice level, or a block level.

[0645] Alternatively, one of the plurality of lookup tables may be selected based on at least one of a size or a shape of the current block or a transform kernel applied in the first transform.

[0646] When an inverse transform matrix of a predefined size is used, it can be set not to use the method of determining an inseparable inverse transform matrix set based on the intra prediction mode. In an example, when an inverse transform matrix of a 48×16 size is used, the process of determining an inseparable inverse transform matrix set based on the intra prediction mode can be omitted.

[0647] As described above, one of a plurality of non-separable transform matrix candidates may be applied to the current block. In this case, each of the non-separable transform matrix candidates may have at least one different size or coefficient.

[0648] Fig.39 is an example showing various inseparable transformation matrix candidates.

[0649] It is assumed that a 4×4 sub-block is a target of the second inverse transform.

[0650] As in Fig.39 In the example shown in (a), a transformation matrix of 64×1 size can be derived by using a non-separable inverse transformation matrix of 64×16 size. The derived transformation matrix can be set as a transformation coefficient of a block of 8×8 size.

[0651] As in Fig.39 In the example shown in (b), a transform matrix of 32×1 size can be derived by using a non-separable inverse transform matrix of 32×16 size. The derived transform matrix can be set as a transform coefficient of a block of 4×8 or 8×4 size.

[0652] As in Fig.39 In the example shown in (c), a transform matrix of 48×1 size can be derived by using a non-separable inverse transform matrix of 48×16 size. The derived transform matrix can be set as the transform coefficients of three 4×4 size blocks. There is a non-zero transform coefficient in the upper left block of the three 4×4 size blocks, and in other blocks, the value of the transform coefficient in the block can be set to 0.

[0653] The non-separable inverse transform matrix to be applied to the current block may be determined based on whether the sub-partition intra encoding method is applied to the current block. In an example, optionally, when the sub-partition intra encoding method is applied to the current block, a non-separable inverse transform matrix of size 32×16 may be applied.

[0654] Alternatively, the type or number of non-separable inverse transform matrix candidates may be determined differently based on whether the subpartition intra encoding method is applied to the current block. In an example, only when the subpartition intra encoding method is applied to the current block, a non-separable inverse transform matrix of 32×16 size may be used as a candidate.

[0655] When the subpartition intra encoding method is applied to the current block and a non-separable inverse transform matrix of a 32×16 size is applied, the size or shape of a block derived by performing the second inverse transform may differ according to the size or shape of a subpartition.

[0656] In an example, when a partition in a vertical direction is applied to a current block (for example, when a sub-partition has a 4×L shape), a transform matrix of a size of 32×1 obtained by performing the second inverse transform may be set as a transform coefficient of a block of a size of 4×8. On the other hand, when a partition in a horizontal direction is applied to a current block (for example, when a sub-partition has an L×4 shape), a transform matrix of a size of 32×1 obtained by performing the second inverse transform may be set as a transform coefficient of a block of a size of 8×4.

[0657] Although the subpartition intra prediction encoding method is not applied, when the current block is encoded by intra prediction and its size is 4×L or L×4, a non-separable matrix of a size of 32×16 may be applied.

[0658] When the subpartition intra encoding method is applied to the current block, the second inverse transform and the first inverse transform may be applied to each of a plurality of subblocks included by the current block.

[0659] Optionally, the second inverse transform and the first inverse transform may be applied to a sub-block at a predefined position in the plurality of sub-blocks or a sub-block whose partition index is less than a threshold value. In an example, the second transform may be applied only to a sub-partition at the top position of the current block or a sub-partition at the leftmost position of the current block, and may not be applied to the remaining sub-partitions.

[0660] When the sub-partition intra-coding method is applied, the target area for the second inverse transform may be determined as 4×4 in size. Alternatively, the size of the target area may be adaptively determined according to the size of the sub-block. In the example, when the minimum value of the width and height of the sub-partition is 4, a 4×4 sized area may be set as the target area.

[0661] Alternatively, for a sub-block of a 4×N or N×4 shape, an area of ​​a size of 8×4 may be set as a target area. In this case, N represents an integer equal to or greater than 8.

[0662] The input matrix may be generated by arranging the transform coefficients in the target region included in the sub-block into rows. In an example, when the target region is set to a 4×4 size, the transform coefficients included in the target region may be transformed into an input matrix of a 16×1 shape. Alternatively, when the target region is set to a 4×8 or 8×4 size, the transform coefficients included in the target region may be transformed into an input matrix of a 32×1 shape.

[0663] The transformation matrix can be derived by multiplying the input matrix by the non-separable transformation matrix. In an example, a 16×1 sized transformation matrix can be obtained by multiplying a 16×16 sized non-separable transformation matrix with a 16×1 sized input matrix. Alternatively, a 32×1 sized transformation matrix can be obtained by multiplying a 32×32 sized non-separable transformation matrix with a 32×1 sized input matrix.

[0664] When the transformation matrix is ​​obtained, the components in the transformation matrix can be set as the transformation coefficients of the sub-block. In the example, the transformation matrix of 16×1 size can be set as the transformation coefficients of the block of 4×4 size. Alternatively, the transformation matrix of 32×1 size can be set as the transformation coefficients of the upper left block of 4×8 or 8×4 size.

[0665] When the second inverse transform is applied to the subblock, a predefined transform kernel may be applied to the horizontal transform and the vertical transform of the subblock. In an example, the horizontal transform kernel and the vertical transform kernel of the subblock to which the second inverse transform is applied may be set to DCT2.

[0666] Whether the second transform is allowed may be determined based on whether the sub-transform block encoding method is applied to the coding block. In an example, when the sub-transform block encoding method is applied to the coding block, it may be set not to apply the second transform.

[0667] Optionally, when the sub-transform block coding method is applied to the coding block, it can be set to apply the second transform only to at least one available sub-block among the multiple sub-blocks. In this case, the available sub-block may represent a block among the multiple sub-blocks on which the first transform is performed.

[0668] The size of the target area for the second transformation in the sub-block may be determined according to the size or shape of the sub-block. In an example, when at least one of the height or width of the sub-block is less than a threshold, the second transformation may be performed on a 4×4 area. On the other hand, when at least one of the height or width of the sub-block is equal to or greater than the threshold, the second transformation may be performed on an 8×8 area.

[0669] Whether to apply the second transform to the sub-block may be determined based on at least one of the size, shape, position, or partition index of the sub-block. In an example, the second transform may be applied only to the sub-block including the upper left sample of the coding block. Optionally, the second transform may be applied only when at least one of the height or width of the sub-block is greater than a threshold.

[0670] Alternatively, information indicating whether the second transform is applied to the sub-block may be signaled in a bitstream.

[0671] When the sub-transform block coding method is applied, it may be set not to allow the simplified second transform.Alternatively, although the sub-transform block coding method is applied, whether to perform the simplified second transform may be determined based on at least one of the size or shape of the sub-block.

[0672] The reconstructed samples of the sub-block on which the transformation is performed may be derived by the sum of the prediction samples and the residual samples. On the other hand, the prediction samples may be set to omit the reconstructed samples in the sub-block on which the transformation is performed. Quantization is to reduce the energy of the block, and the quantization process includes a process of dividing the transformation coefficients by a specific constant value. The constant value may be derived from a quantization parameter, and the quantization parameter may be defined as a value from 1 to 63.

[0673] When obtaining a reconstructed block of the current block, information loss occurring in the quantization and encoding process can be reduced via in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter is referred to as a first reconstructed block, and the reconstructed block after applying the in-loop filter is referred to as a second reconstructed block.

[0674] The second reconstructed block may be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block. In this regard, the SAO or ALF may be applied after the deblocking filter is applied.

[0675] The operation of applying the embodiments described for the decoding process or the encoding process to the encoding process or the decoding process, respectively, may be included in the scope of the present disclosure. Within the scope of the present disclosure, the embodiments in which the operations occur in a predetermined order may be modified to embodiments in which the operations occur in an order different from the predetermined order.

[0676] Although the above embodiments are described based on a series of operations or flow charts, the embodiments do not limit the time series order of the operations of the method to this. In another example, the operations can be performed simultaneously or in a different order from them as needed. In addition, in the above embodiments, each component (e.g., unit, module, etc.) in the components constituting the block diagram can be implemented in the form of hardware devices or software. Multiple components can be combined with each other as a single component that can be implemented using a single hardware device or software. The above embodiments can be implemented using program instructions that can be executed via various computer components. Instructions can be recorded in a computer-readable storage medium. The computer-readable storage medium can include program instructions, data files, data structures, etc., individually or in combination with each other. Examples of computer-readable storage media include magnetic media (such as hard disks, floppy disks, and tapes), optical storage media (such as CD-ROMs, DVDs), and magneto-optical media (such as floppy disks) and hardware devices (such as ROMs, RAMs, flash memories, etc.) that are specifically configured to store and execute program instructions therein. The hardware device can be configured to operate as one or more software modules to perform processing according to the present disclosure, or vice versa.

[0677] Industrial Applicability

[0678] The present disclosure may be applied to an electronic device that encodes / decodes a video.

Claims

1. A method for decoding a video, the method comprising: Determine whether to partition the coding block based on the intra-frame sub-partition flag; When the intra-frame sub-partition flag indicates that the coding block is to be partitioned, partitioning the coding block into a plurality of sub-blocks in a horizontal direction or a vertical direction; performing intra prediction on each of the plurality of sub-blocks; determining whether to decode first index information for the coding block from a bitstream; determining whether to apply a second inverse transform to one of the plurality of sub-blocks based on the first index information; When it is determined that the second inverse transform is applied to the sub-block, performing the second inverse transform on the sub-block; Performing a first inverse transform on the sub-block on which the second inverse transform is performed, the first inverse transform including a horizontal inverse transform and a vertical inverse transform; as well as Obtaining reconstructed samples based on the prediction samples generated by the intra prediction and the residual samples generated by the first inverse transform, wherein determining whether to decode the first index information from the bitstream is based on a comparison of a width or a height of the sub-block with a threshold value, Wherein, when the first index information is 0, it is determined that the second inverse transform is not used for the coding block, Wherein, when the first index information is greater than 0, it is determined that a second inverse transform is used for the coding block and one of a plurality of inverse transform matrices is specified by the first index information, wherein only when the width or height of the sub-block is equal to or greater than the threshold, first index information for the coding block is decoded from the bitstream, and Wherein, when decoding of the first index information for the coding block is omitted, it is determined that the second inverse transform is not applied to any of the sub-blocks.

2. The method according to claim 1, wherein: An inverse vertical transform is performed based on the inverse vertical transform matrix, and an inverse horizontal transform is performed based on the inverse horizontal transform matrix, and Wherein, when the coding block is partitioned into the plurality of sub-blocks, decoding of the second index information specifying one of the plurality of inverse transform matrix sets is omitted, each of the plurality of inverse transform matrix sets represents a pair of horizontal inverse transform matrices and vertical inverse transform matrices, and the horizontal inverse transform matrix and the vertical inverse transform matrix are determined as default transform matrices.

3. The method according to claim 1, wherein: selecting one of a set of inverse transform matrices based on an intra prediction mode of the coding block, each of the inverse transform matrix sets comprising a plurality of inverse transform matrices, and The first index information specifies one of the inverse transformation matrices included in the selected inverse transformation matrix set.

4. The method according to claim 3, wherein: The inverse transform matrix set selected when the intra prediction mode of the coding block is a non-directional mode is different from the inverse transform matrix set selected when the intra prediction mode of the coding block is a directional mode.

5. The method according to claim 1, wherein: When a block-based Delta Pulse Code Modulation (BDPCM) scheme is applied to the coding block, decoding of first index information for the coding block is omitted, and a second inverse transform is not performed on any of the subblocks.

6. A method for encoding a video, the method comprising: Partitioning the coding block into a plurality of sub-blocks in a horizontal direction or a vertical direction; performing intra prediction on each of the plurality of sub-blocks; Obtaining residual samples by subtracting predicted samples generated by intra prediction from original samples; Performing a first transform on residual samples of one of the multiple sub-blocks, where the first transform includes a horizontal transform and a vertical transform; determining whether to perform a second transform on the sub-block on which the first transform is performed; as well as determining whether to encode first index information for the coding block, wherein an intra-frame sub-partition flag indicating whether the coding block is partitioned into the plurality of sub-blocks is encoded into the bitstream, The determining whether to encode the first index information for the coding block is based on a comparison of a width or a height of the sub-block with a threshold value, When the second transformation is not performed on the sub-block, the first index information is set to 0. Wherein, when a second transformation is performed on the sub-block, the first index information is set to be greater than 0 and indicates one of the transformation matrices used for the second transformation, wherein the first index information for the coding block is encoded into the bitstream only when the width or height of the sub-block is equal to or greater than the threshold, and When the width or height of a sub-block is smaller than the threshold, the second transformation is not applied to any of the sub-blocks.

7. The method according to claim 6, wherein: A vertical transformation is performed based on the vertical transformation matrix, and a horizontal transformation is performed based on the horizontal transformation matrix, and In which, when the coding block is partitioned into the multiple sub-blocks, encoding of the second index information specifying one of the multiple transformation matrix sets is omitted, each of the multiple transformation matrix sets represents a pair of horizontal transformation matrices and vertical transformation matrices, and the horizontal transformation matrix and the vertical transformation matrix are determined as default transformation matrices.

8. The method according to claim 6, wherein: selecting one of a set of transform matrices based on an intra prediction mode of the coding block, each of the set of transform matrices comprising a plurality of transform matrices, and The first index information specifies one of the transformation matrices included in the selected transformation matrix set.

9. The method according to claim 8, wherein: The transform matrix set selected when the intra prediction mode of the coding block is a non-directional mode is different from the transform matrix set selected when the intra prediction mode of the coding block is a directional mode.

10. The method according to claim 6, wherein: When a block-based delta pulse code modulation (BDPCM) scheme is applied to the coding block, encoding of first index information for the coding block is omitted, and a second transform is not performed on any of the subblocks.

11. An apparatus for transmitting compressed video data, comprising: a processor configured to obtain compressed video data; as well as a sending unit configured to send compressed video data, The compressed video data is obtained, including: Partitioning the coding block into a plurality of sub-blocks in a horizontal direction or a vertical direction; performing intra prediction on each of the plurality of sub-blocks; Obtaining residual samples by subtracting predicted samples generated by intra prediction from original samples; Performing a first transform on residual samples of one of the multiple sub-blocks, where the first transform includes a horizontal transform and a vertical transform; determining whether to perform a second transform on the subblock on which the first transform was performed; and determining whether to encode first index information for the coding block, wherein an intra-frame sub-partition flag indicating whether the coding block is partitioned into the plurality of sub-blocks is encoded into the bitstream, The determining whether to encode the first index information for the coding block is based on a comparison of a width or a height of the sub-block with a threshold value, When the second transformation is not performed on the sub-block, the first index information is set to 0. Wherein, when a second transformation is performed on the sub-block, the first index information is set to be greater than 0 and indicates one of the transformation matrices used for the second transformation, wherein the first index information for the coding block is encoded into the bitstream only when the width or height of the sub-block is equal to or greater than the threshold, and When the width or height of a sub-block is smaller than the threshold, the second transformation is not applied to any of the sub-blocks.

Images