Method for encoding / decoding video signal and device for the same

Abstract

To provide a combined prediction method for combining a plurality of prediction methods when encoding / decoding a video signal, and a device for executing the method.SOLUTION: A method for encoding / decoding a video signal includes: determining whether a combined prediction mode is applied to a current block; when the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block; and obtaining a third prediction block of the current block by performing a weighted sum operation on the first prediction block and the second prediction block. In this case, the first prediction block may be obtained based on the motion information of a merge candidate of the current block, and the second prediction block may be obtained based on an intra prediction mode of the current block.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method for encoding / decoding a video signal and to an apparatus for said method. [Background technology]

[0002] As display panels get larger, video services with higher picture quality are becoming necessary. The biggest problem with high-definition video services is the large increase in data volume. To solve this problem, research is being actively conducted to improve video compression rates. As a representative example, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC) in 2009. JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013, and its compression performance is about twice that of H.264 / AVC. With the rapid development of high-definition video services, the performance limitations of HEVC are gradually becoming apparent. Summary of the Invention [Problem to be solved by the invention]

[0003] It is an object of the present invention to provide a combined prediction method for combining multiple prediction methods when encoding / decoding a video signal, and an apparatus for implementing said method.

[0004] An object of the present invention is to provide a method for dividing a coding block into a number of prediction units when encoding / decoding a video signal, and an apparatus for implementing said method.

[0005] The technical problems that the present invention aims to achieve are not limited to those mentioned above, and those skilled in the art will be able to clearly understand other technical problems not mentioned from the following description. [Means for solving the problem]

[0006] The video signal decoding / encoding method according to the present invention includes determining whether to apply a combined prediction mode to a current block, obtaining a first predicted block and a second predicted block of the current block when the combined prediction mode is applied to the current block, and obtaining a third predicted block of the current block by performing a weighted sum operation of the first predicted block and the second predicted block. In this case, the first predicted block may be obtained based on motion information of a merge candidate of the current block, and the second predicted block may be obtained based on an intra prediction mode of the current block.

[0007] In the video signal decoding / encoding method according to the present invention, when the combined prediction mode is applied to the current block, triangulation may be set not to be applied to the current block.

[0008] In the video signal decoding / encoding method according to the present invention, an intra prediction mode of the current block may be set as a planar mode.

[0009] In the video signal decoding / encoding method according to the present invention, the second prediction block may be obtained based on a reference sample line included in an adjacent reference sample line.

[0010] In the video signal decoding / encoding method according to the present invention, when performing the weighted sum operation, weighting values ​​to be applied to the first prediction block and the second prediction block may be determined based on the prediction encoding mode of an adjacent block adjacent to the current block.

[0011] In the video signal decoding / encoding method according to the present invention, if at least one of the width and the height of the current block is greater than a threshold, the combined prediction mode may not be applied to the current block.

[0012] In the video signal decoding / encoding method according to the present invention, if a flag indicating that a merge mode is to be applied to the current block is true, the combined prediction mode may be set to be applicable to the current block.

[0013] The brief features of the invention described above are illustrative embodiments of the detailed description of the invention that follows, and are not intended to limit the scope of the invention. Effect of the Invention

[0014] According to the present invention, by providing a combined prediction method that combines a plurality of prediction methods, it is possible to improve inter prediction efficiency.

[0015] According to the present invention, by providing a method for dividing a coding block into a plurality of predictive blocks and deriving motion information for each of the plurality of predictive blocks, it is possible to improve inter-prediction efficiency.

[0016] The effects that can be achieved by the present invention are not limited to the above effects, and those skilled in the art will be able to clearly understand other effects not mentioned from the following description. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram of a video encoder according to an embodiment of the present invention. [Diagram 2]FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing a number of division types of a coding block. [Diagram 5] FIG. 5 is a diagram showing an example of division of a coding tree unit. [Figure 6] FIG. 6 is a diagram showing the shape of a basic data unit. [Figure 7] FIG. 7 is a diagram showing an example of dividing a coding block into a plurality of sub-blocks. [Figure 8] FIG. 8 is a diagram showing an example of dividing a coding block into a plurality of sub-blocks. [Figure 9] FIG. 9 is a flowchart of an inter prediction method according to an embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart of a process for deriving motion information of a current block in the merge mode. [Figure 11] FIG. 11 is a diagram showing examples of candidate blocks for deriving merging candidates. [Figure 12] FIG. 12 is a diagram showing an example of dividing a coding block into a plurality of prediction units using diagonals. [Figure 13] FIG. 13 is a diagram showing an example of dividing a coding block into two prediction units. [Figure 14] FIG. 14 is a diagram showing an example of dividing a coding block into a plurality of prediction blocks of various sizes. [Figure 15] FIG. 15 is a diagram showing an example in which diagonal division is applied to a transform unit. [Figure 16] FIG. 16 is a diagram showing adjacent blocks for deriving triangle merging candidates. [Figure 17] FIG. 17 is a diagram for explaining an example of determining the availability of adjacent blocks for each triangular prediction unit. [Figure 18]18 and 19 are diagrams illustrating an example of deriving a predicted sample by performing a weighted sum operation on a first predicted sample and a second predicted sample. [Figure 19] FIG. 19 is a diagram illustrating an example of deriving a predicted sample by performing a weighted sum operation on a first predicted sample and a second predicted sample. [Figure 20] FIG. 20 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention. [Figure 21] FIG. 21 is a diagram showing intra prediction modes. [Figure 22] FIG. 22 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Diagram 23] FIG. 23 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Figure 24] FIG. 24 is a diagram showing angular intra prediction modes and angles formed by a line parallel to the x-axis. [Diagram 25] FIG. 25 illustrates an embodiment of obtaining predicted samples when the current block is non-square. [Figure 26] FIG. 26 is a diagram showing the wide-angle intra prediction mode. [Figure 27] FIG. 27 is a diagram showing an embodiment in which the PDPC is applied. [Figure 28] FIG. 28 illustrates an embodiment for applying weighting values. [Figure 29] FIG. 29 illustrates an example of assigning different weighting values ​​to sub-blocks depending on whether the merge candidate has bidirectional motion information. [Diagram 30] FIG. 30 is a diagram illustrating an example of assigning different weighting values ​​to sub-blocks based on the intra-prediction mode of the current block. [Diagram 31] FIG. 31 is a diagram showing an example of specifying the second merging candidate in consideration of the search order of the candidate blocks. [Diagram 32] FIG. 32 is a diagram showing an example of selecting a first merging candidate and a second merging candidate from merging candidates derived from non-adjacent blocks. [Diagram 33]FIG. 33 is a diagram illustrating an example of determining the second merging candidate. [Diagram 34] FIG. 34 is a diagram showing an example of determining the weighting value to be applied to a prediction block based on the shape of a candidate block. [Diagram 35] FIG. 35 shows examples of replaced merging candidates. [Diagram 36] FIG. 36 shows the sub-blocks that perform the secondary transformation. [Figure 37] FIG. 37 shows the sub-blocks that perform the secondary transformation. [Figure 38] FIG. 38 is a diagram for explaining an example of determining the transformation type of the current block. [Figure 39] FIG. 39 is a flow chart showing the process of determining the block strength. [Diagram 40] FIG. 40 is a diagram showing predefined filter candidates. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0019] Video encoding and decoding is performed on a block-by-block basis, for example, encoding / decoding operations such as transform, quantization, prediction, in-loop filtering, or reconstruction may be performed on a coding block, a transform block, or a prediction block.

[0020] Hereinafter, the block to be coded / decoded is referred to as a “current block.” For example, depending on the current coding / decoding process step, the current block may indicate a coding block, a transformation block, or a prediction block.

[0021] Additionally, the term "unit" as used herein may be understood to refer to a basic unit for performing a particular encoding / decoding process, and "block" to refer to a sample array of a given size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described below, a coding block and a coding unit may be understood to have the same meaning.

[0022] FIG. 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0023] As shown in FIG. 1, the video encoding device 100 may include a picture partitioning unit 110, a prediction unit 120, 125, a transform unit 130, a quantization unit 135, a reordering unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150 and a memory 155.

[0024] Each component shown in Fig. 1 is shown independently, which does not indicate that each component is composed of a separate hardware or a single software component, but indicates various characteristic functions of the video encoding device. That is, for convenience of explanation, at least two of the components are combined into one component in the arrangement manner of the components, or one component is divided into multiple components to perform the function. As long as it does not deviate from the essence of the present invention, such an embodiment in which each component is combined and an embodiment in which each component is separated are also within the scope of the present invention.

[0025] Some structural elements are not necessary for performing essential functions in the present invention, but are optional structural elements used only to improve performance. The present invention may be implemented by including only the components necessary to realize the essence of the present invention other than the structural elements used only to improve performance, and a structure including only the necessary structural elements other than the optional structural elements used only to improve performance also falls within the scope of the present invention.

[0026] The picture division unit 110 can divide an input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU, Prediction Unit), a transform unit (TU, Transform Unit), or a coding unit (CU, Coding Unit). The picture division unit 110 can divide one picture into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined standard (e.g., a cost function) to code the picture.

[0027] For example, a picture may be divided into multiple coding units. To divide a picture into coding units, a recursive tree structure, for example a Quad Tree Structure, may be used, with one video or largest coding unit as the root, but the coding unit may be divided into other coding units that may have child nodes corresponding to the number of coding units into which it is divided. Due to some restrictions, coding units that are not divided again become leaf nodes. That is, assuming that a coding unit can only realize a square division, one coding unit can be divided into at most four other coding units.

[0028] Hereinafter, in the embodiments of the present invention, the encoding unit may refer to a unit that performs encoding, and may also refer to a unit that performs decoding.

[0029] The prediction units in one coding unit may be divided into at least one of the following shapes: square or rectangle of the same size, and one prediction unit in one coding unit may be divided into a different shape and / or size than the other prediction units.

[0030] When a prediction unit for performing intra prediction using a coding unit is not the smallest coding unit, intra prediction may be performed without dividing the prediction unit into a plurality of N×N prediction units.

[0031] The prediction units 120 and 125 may include an inter prediction unit 120 for performing inter prediction and an intra prediction unit 125 for performing intra prediction. It may be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) may be determined based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific contents. For example, the prediction method and prediction mode may be determined by the prediction unit, and the prediction may be performed by the transformation unit. Residual values ​​(residual blocks) of the generated prediction block and original block may be input to the transformation unit 130. In addition, prediction mode information, motion vector information, etc. for prediction may be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When a specific encoding mode is used, the prediction unit 120 and 125 may not generate a prediction block, but the original block may be directly encoded and transmitted to the decoder.

[0032] The inter prediction unit 120 may predict the prediction unit based on information of at least one of a picture before or after the current picture. In some cases, the prediction unit may be further predicted based on information of a part of an encoded region in the current picture. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0033] The reference picture interpolator receives reference picture information from the memory 155 and can generate integer or fractional pixel information based on the reference picture. For luma pixels, a DCT-based 8-tap interpolation filter with different filter coefficients may be used to generate fractional pixel information in 1 / 4 pixel units. For chroma signals, a DCT-based 4-tap interpolation filter with different filter coefficients may be used to generate fractional pixel information in 1 / 8 pixel units.

[0034] The motion prediction unit can perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. A motion vector can be calculated using a number of methods, such as a full search-based block matching algorithm (FBMA), a three step search method (TSS), a new three step search algorithm (NTS), etc. The motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixel. The motion prediction unit may predict the current prediction unit using various motion prediction methods, such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, and the like.

[0035] The intra prediction unit 125 may generate a prediction unit based on reference pixel information around a current block as pixel information in a current picture. When a neighboring block of a current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed may be used as reference pixel information of the neighboring block on which intra prediction has been performed. That is, when a reference pixel is unavailable, at least one of the available reference pixels may be substituted for the unavailable reference pixel information.

[0036] In intra prediction, the prediction mode may include an angular prediction mode that uses reference pixel information based on a prediction direction, and a non-angular mode that does not use direction information when performing prediction. The prediction mode of luma information and the prediction mode of chroma information may be different, and intra prediction mode information used to predict luma information or predicted luma signal information may be applied to predict chroma information.

[0037] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction may be performed on the prediction unit based on the pixels on the left side, the pixels on the top left, and the pixels on the top. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction may be performed using reference pixels based on the transform unit. And intra prediction using N×N division may be applied to only the smallest coding unit.

[0038] The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels based on a prediction mode. The type of the adaptive intra smoothing filter for the reference pixels may be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted according to the intra prediction mode of a prediction unit existing in the vicinity of the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted by a surrounding prediction unit, if the intra prediction modes of the current prediction unit and the surrounding prediction unit are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction unit are the same may be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction unit are different, the prediction mode information of the current block may be encoded by performing entropy encoding.

[0039] In addition, a residual block including residual information can be generated, the residual information being a difference between a prediction unit that is predicted by the prediction unit generated in the prediction unit 120, 125 and the original block of the prediction unit. The generated residual block can be input to the conversion unit 130.

[0040] The transform unit 130 may transform the residual block using a transform method such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST). The residual block includes residual information between the original block and the prediction unit generated by the prediction unit 120, 125. The DCT transform core includes at least one of DCT2 or DCT8, and the DST transform core includes DST7. It may be determined whether to apply DCT or DST to transform the residual block based on intra prediction mode information of the prediction unit used to generate the residual block. Furthermore, the transform of the residual block may be skipped. A flag indicating whether to skip the transform of the residual block may be coded. For residual blocks whose size is equal to or smaller than a threshold, luma component, or chroma component (4:4:4 format or less), the transform may be allowed to be skipped.

[0041] The quantization unit 135 can quantize the values ​​transformed into the frequency domain by the transformation unit 130. The quantization coefficients may vary depending on the importance of the block or video. The values ​​calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the reordering unit 160.

[0042] Reordering unit 160 may perform reordering of coefficient values ​​on the quantized residual values.

[0043] The rearrangement unit 160 may convert two-dimensional block shape coefficients into one-dimensional vector format by a coefficient scanning method. For example, the rearrangement unit 160 may scan DC coefficients to high frequency region coefficients using a zig-zag scan method and convert them into one-dimensional vector format. Depending on the size of the transform unit and the intra prediction mode, instead of the zig-zag scan, vertical scanning for scanning two-dimensional block shape coefficients along a column direction and horizontal scanning for scanning two-dimensional block shape coefficients along a row direction may be used. That is, depending on the size of the transform unit and the intra prediction mode, it may be determined which scanning method to use among zig-zag scan, vertical scan, and horizontal scan.

[0044] The entropy coding unit 165 may perform entropy coding based on the values ​​calculated by the rearrangement unit 160. For example, the entropy coding may use a number of coding methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0045] The entropy coding unit 165 can code multiple pieces of information such as residual value coefficient information of the coding unit from the rearrangement unit 160 and the prediction units 120 and 125, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0046] The entropy coding unit 165 may perform entropy coding on the coefficient values ​​of the coding unit input from the rearrangement unit 160 .

[0047] The inverse quantization unit 140 and the inverse transform unit 145 inverse quantize a plurality of values ​​quantized by the quantization unit 135 and inverse transform the values ​​transformed by the transform unit 130. The residual values ​​generated by the inverse quantization unit 140 and the inverse transform unit 145 can be merged with prediction units predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0048] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0049] The deblocking filter can remove block artifacts generated in a reconstructed picture due to boundaries between blocks. To determine whether to perform deblocking, it can be determined whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter may be applied based on a required deblocking filtering strength. In addition, when performing vertical filtering and horizontal filtering in the process of using the deblocking filter, horizontal filtering and vertical filtering may be processed synchronously.

[0050] The offset correction unit can correct the offset between the original video and the video on which deblocking is performed on a pixel-by-pixel basis. In order to perform offset correction on a specified picture, a method may be used in which pixels included in the video are divided into a predetermined number of regions, a region to be offset is determined, and an offset is applied to the corresponding region, or an offset is applied after taking into account edge information of each pixel.

[0051] Adaptive loop filtering (ALF) may be performed based on a comparison between the reconstructed picture after filtering and the original video. After dividing the pixels included in the video into predetermined sets, filtering can be performed differentially for each set by determining one filter to be used for the corresponding set. Information related to whether adaptive loop filtering is applied and a luminance signal may be transmitted by a coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may differ for each block. Moreover, the same type (fixed type) of adaptive loop filter may be applied regardless of the characteristics of the block to which it is applied.

[0052] The memory 155 may store reconstructed blocks or pictures calculated by the filter unit 150 and may provide stored reconstructed blocks or pictures to the prediction units 120, 125 when performing inter prediction.

[0053] FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0054] As shown in FIG. 2, the video decoder 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0055] When a video stream is input from a video encoder, the input stream may be decoded according to the reverse steps of the video encoder.

[0056] The entropy decoding unit 210 may perform entropy decoding according to steps opposite to those of the entropy encoding performed in the entropy encoding unit of the video encoder. For example, multiple methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc. may be applied in accordance with the method performed in the video encoder.

[0057] The entropy decoding unit 210 may decode information related to intra-prediction and inter-prediction performed by the encoder.

[0058] The reordering unit 215 can perform reordering based on the method for reordering the stream entropy decoded by the entropy decoding unit 210 in the encoding unit. The reordering unit 215 may reconstruct and reorder coefficients in a two-dimensional block shape, based on coefficients represented in a one-dimensional vector format. The reordering unit 215 may receive information related to coefficient scanning performed in the encoding unit, and perform the reordering by a method of performing a backward scan according to the scanning order performed in the corresponding encoding unit.

[0059] The inverse quantization unit 220 may perform inverse quantization based on a quantization parameter provided by the encoder and the coefficient values ​​of the block to be reordered.

[0060] The inverse transform unit 225 may perform an inverse discrete cosine transform and an inverse discrete sine transform on the quantization result performed by the video encoder. The inverse discrete cosine transform and the inverse discrete sine transform belong to the inverse transform of the transform performed in the transform unit, that is, the inverse transform of the discrete cosine transform and the discrete sine transform. The DCT transform core may include at least one of DCT2 and DCT8, and the DST transform core may include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform may not be performed in the inverse transform unit 225. The inverse transform may be performed by a transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder may selectively perform a transform method (e.g., DCT or DST) based on a plurality of information such as a prediction method, a size of a current block, and a prediction direction.

[0061] The prediction units 230 and 235 may generate a prediction block based on information related to the generation of the prediction block provided by the entropy decoding unit 210 and previously decoded block or picture information provided by the memory 245 .

[0062] As described above, when performing intra prediction in the same manner as the operation in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, perform intra prediction on the prediction unit based on the pixel on the left side, the pixel on the top left, and the pixel on the top; if the size of the prediction unit when performing intra prediction is different from the size of the transform unit, perform intra prediction using reference pixels based on the transform unit. And further, apply intra prediction using N×N division to only the smallest coding unit.

[0063] The prediction unit 230, 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives multiple types of information, such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, classifies the prediction unit based on the current encoding unit, and can determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may use information necessary for inter prediction of the current prediction unit provided by the video encoder, and perform inter prediction on the current prediction unit based on information included in at least one picture of a picture before or after the current picture to which the current prediction unit belongs. Alternatively, the inter prediction may be further performed based on information of a reconstructed part of the current picture to which the current prediction unit belongs.

[0064] To perform inter prediction, an encoding unit can determine whether the motion prediction method of a prediction unit included in a corresponding encoding unit is skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), or intra block copy mode.

[0065] The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit on which intra prediction has been performed, the intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra smoothing filter is a part that performs filtering on reference pixels of the current block, and may determine whether to apply a filter based on the prediction mode of the current prediction unit. Adaptive intra smoothing filtering may be performed on reference pixels of the current block using the prediction mode of the prediction unit and adaptive intra smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode in which adaptive intra smoothing filtering is not performed, the adaptive intra smoothing filter may not be applied.

[0066] Regarding the reference pixel interpolation unit, when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values ​​to be interpolated for reference pixels, the reference pixels may be interpolated to generate reference pixels in pixel units of integer or fractional values. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating reference pixels, the reference pixels may not need to be interpolated. When the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0067] The reconstructed block or picture may be provided to a filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0068] From the video encoder, information related to whether to apply a deblocking filter to a corresponding block or picture, and information related to whether to apply a strong filter or a weak filter when applying the deblocking filter may be received. From a deblocking filter of the video decoder, information related to a deblocking filter provided by the video encoder may be received, and deblocking filtering may be performed on the corresponding block in the video decoder.

[0069] The offset correction unit can perform offset correction on the reconstructed video based on the type and offset amount information for offset correcting the video during encoding.

[0070] The ALF may be applied to the coding unit based on encoder-provided information related to whether to apply the ALF, ALF coefficient information, etc. Such ALF information may be provided by being included in a particular parameter set.

[0071] The memory 245 stores the reconstructed pictures or blocks so that they can be used as reference pictures or blocks and provide the reconstructed pictures to an output.

[0072] FIG. 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0073] A coding block of a maximum size may be defined as a coding tree block. A picture may be divided into multiple coding tree units (CTUs). A coding tree unit is a coding unit of a maximum size, and may be further referred to as a largest coding unit (LCU). Figure 3 shows an example of dividing a picture into multiple coding tree units.

[0074] The size of the coding tree unit may be defined at the picture level or at the sequence level, so a signal may be used by the picture parameter set or the sequence parameter set to transmit information indicating the size of the coding tree unit.

[0075] For example, the size of the coding tree unit for the entire picture in the sequence may be set as 128 x 128. Alternatively, the size of the coding tree unit may be determined to be either 128 x 128 or 256 x 256 at the picture level. For example, the size of the coding tree unit in the first picture may be set as 128×128, and the size of the coding tree unit in the second picture may be set as 256×256.

[0076] Coding blocks can be generated by dividing the coding tree units. A coding block indicates a basic unit for performing coding / decoding processing. For example, prediction or transformation may be performed by different coding blocks, or a predictive coding mode may be determined by different coding blocks. A predictive coding mode indicates a method for generating a predicted picture. For example, the predictive coding mode may include intra prediction (Intra Prediction), inter prediction (Inter Prediction), current picture referencing (CPR), or intra block copy (IBC) or combined prediction. For a coding block, at least one predictive coding mode among intra prediction, inter prediction, current picture referencing, or combined prediction may be used to generate a predictive block related to the coding block.

[0077] Information indicating the predictive coding mode of the current block may be transmitted by a signal through the stream. For example, the information may be a one-bit flag indicating whether the predictive coding mode is an intra mode or an inter mode. Only when it is determined that the predictive coding mode of the current block is an inter mode, the current picture reference or combined prediction may be used.

[0078] The current picture reference is used to set the current picture as a reference picture and obtain a prediction block of the current block from the coded / decoded area in the current picture. The current picture means a picture including the current block. Information indicating whether the current picture reference is applied to the current block may be transmitted by a signal via the stream. For example, the information may be a 1-bit flag, and if the flag is true, the predictive coding mode of the current block may be determined as current picture reference, and if the flag is false, the predictive coding mode of the current block may be determined as inter prediction.

[0079] Alternatively, the predictive coding mode of the current block may be determined based on the reference picture index. For example, if the reference picture index points to the current picture, the predictive coding mode of the current block may be determined as current picture reference. If the reference picture index points to another picture other than the current picture, the predictive coding mode of the current block may be determined as inter prediction. That is, the current picture reference is a prediction method that uses information of an area in the current picture after encoding / decoding, and the inter prediction is a prediction method that uses information of another picture after encoding / decoding.

[0080] The combined prediction indicates a coding mode in which two or more of intra prediction, inter prediction, and current picture reference are combined. For example, when the combined prediction is applied, the first predicted block may be generated by one of intra prediction, inter prediction, or current picture reference, and the second predicted block may be generated by another of intra prediction, inter prediction, or current picture reference. When the first predicted block and the second predicted block are generated, the final predicted block may be generated by an average operation or a weighted sum operation of the first predicted block and the second predicted block. Information indicating whether the combined prediction is applied may be transmitted by a signal via the stream. The information may be a 1-bit flag.

[0081] FIG. 4 is a diagram showing a number of division types of a coding block.

[0082] A coding block may be divided into a plurality of coding blocks by quad-tree partitioning, binary tree partitioning, or ternary tree partitioning, and the divided coding block may be further divided into a plurality of coding blocks by quad-tree partitioning, binary tree partitioning, or ternary tree partitioning.

[0083] Quadtree partitioning refers to a partitioning technique that splits the current block into four blocks. As a result of the quadtree partitioning, the current block may be split into four square segments (see “SPLIT_QT” in part (a) of Figure 4).

[0084] Binary tree splitting refers to a splitting technique that splits the current block into two blocks. The process of splitting the current block into two blocks along the vertical direction (i.e., using a vertical line across the current block) may be referred to as vertical binary tree splitting, and the process of splitting the current block into two blocks along the horizontal direction (i.e., using a horizontal line across the current block) may be referred to as horizontal binary tree splitting. After binary tree splitting, the current block may be split into two non-square segments. "SPLIT_BT_VER" in part (b) of FIG. 4 indicates the vertical binary tree split result, and "SPLIT_BT_HOR" in part (c) of FIG. 4 indicates the horizontal binary tree split result.

[0085] Ternary tree partitioning refers to a partitioning technique that partitions the current block into three blocks. The process of partitioning the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) may be referred to as vertical ternary tree partitioning, and the process of partitioning the current block into three blocks along the horizontal direction (i.e., using two horizontal lines crossing the current block) may be referred to as horizontal ternary tree partitioning. After ternary tree partitioning, the current block may be partitioned into three non-square segments. In such a case, the width / height of the segment located at the center of the current block may be twice the width / height of the other segments. "SPLIT_TT_VER" in part (d) of FIG. 4 indicates the vertical ternary tree partitioning result, and "SPLIT_TT_HOR" in part (e) of FIG. 4 indicates the horizontal ternary tree partitioning result.

[0086] The number of times a coding tree unit is divided may be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit may be determined at a sequence or picture level, so that the maximum partitioning depth of a coding tree unit may be different for different sequences or pictures.

[0087] Alternatively, the maximum partitioning depth for each of multiple partitioning techniques may be determined independently, for example, the maximum partitioning depth allowed for quadtree partitioning may be different from the maximum partitioning depth allowed for binary and / or ternary tree partitioning.

[0088] The encoder may transmit information indicating at least one of the partition shape or partition depth of the current block by a signal via the stream, and the decoder may determine the partition shape and partition depth of the coding tree unit based on the information parsed from the stream.

[0089] FIG. 5 is a diagram showing an example of division of a coding tree unit.

[0090] The process of partitioning a coding block using partitioning techniques such as quad-tree partitioning, binary-tree partitioning, and / or ternary-tree partitioning may be referred to as multi-tree partitioning.

[0091] The coding blocks generated by applying multi-tree partitioning to the coding block may be referred to as downstream coding blocks. When the partitioning depth of the coding block is k, the partitioning depth of the downstream coding blocks is set to k+1.

[0092] On the other hand, for multiple coding blocks with a division depth of k+1, the coding block with a division depth of k may be referred to as an upstream coding block.

[0093] The partition type of the current coding block may be determined based on at least one of the partition shape of the upstream coding block or the partition type of the adjacent coding block. The adjacent coding block may be adjacent to the current coding block and may include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the current coding block. The partition type may include at least one of whether to partition into a quad tree, whether to partition into a binary tree, the partition direction of the binary tree, whether to partition into a ternary tree, or the partition direction of the ternary tree.

[0094] To determine the partition shape of the coding block, information indicating whether the coding block is partitioned may be transmitted by a signal via the stream, the information being a one-bit flag "split_cu_flag", which indicates that the coding block is partitioned by a multi-tree partitioning technique if the flag is true.

[0095] If "split_cu_flag" is true, a signal may be sent over the stream indicating whether the coding block has been quadtree split, the information being a 1-bit flag "split_qt_flag", which, if true, indicates that the coding block may be split into four blocks.

[0096] 5 shows an example in which a coding tree unit is quadtree-divided to generate four coding blocks with a division depth of 1. It also shows that quadtree division is applied again to the first and fourth coding blocks among the four coding blocks generated as a result of the quadtree division. Finally, four coding blocks with a division depth of 2 can be generated.

[0097] In addition, quadtree partitioning may be applied again to the coding block with partition depth 2 to generate a coding block with partition depth 3.

[0098] When quadtree partitioning is not applied to a coding block, it may be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block, taking into consideration at least one of the following: the size of the coding block, whether the coding block is located on a picture boundary, the maximum partition depth, or the partition shape of an adjacent block. When it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating a partitioning direction may be transmitted by a signal via a stream. The information may be a 1-bit flag "mtt_split_cu_vertical_flag". It may be determined whether the partitioning direction is vertical or horizontal based on the flag. Also, information indicating which partition of binary tree partitioning or ternary tree partitioning is applied to the coding block may be transmitted by a signal via a stream. The information may be a 1-bit flag "mtt_split_cu_binary_flag". It may be determined whether to apply binary tree partitioning or ternary tree partitioning to the coding block based on the flag.

[0099] For example, the example shown in Figure 5 shows that vertical binary tree partitioning is applied to a coding block with a partitioning depth of 1, vertical ternary tree partitioning is applied to the left coding block among the coding blocks generated as a result of the partitioning, and vertical binary tree partitioning is applied to the right coding block.

[0100] When implementing a video encoding or decoding device, there is a problem that it is difficult to process an area exceeding a threshold due to hardware performance. For example, if the hardware performance allows for simultaneous processing of at most 4096 samples, a data unit of size 64×64 should be accessed and processed redundantly, and there may be a problem that data in an area of ​​more than 4096 samples cannot be processed simultaneously. As described above, a basic unit of data processing may be defined as a pipeline-based basic data unit (virtual processing data unit, VPDU, hereinafter referred to as a basic data unit).

[0101] The elementary data units may be divided into square, non-square or non-rectangular types.

[0102] FIG. 6 is a diagram showing the shape of a basic data unit.

[0103] A basic data unit may include samples up to the maximum number of samples that can be processed simultaneously. For example, in the example shown in FIG. 6(a), a square block of 64×64 size may be set as the basic data unit. Alternatively, a non-square block may be set as the basic data unit. For example, in the example shown in FIG. 6(b) or FIG. 6(c), a block of 32×128 size or a block of 64×32 size may be set as the basic data unit.

[0104] Although not shown, basic data units of triangles, L-shapes or polygons may be defined.

[0105] Information for determining the elementary data units may be transmitted by a signal over the stream, said information may be used to determine at least one of a size or a shape of the elementary data units, and based on said information it may be determined whether non-square elementary data units or non-rectangular elementary data units are allowed.

[0106] Alternatively, at least one of the size or shape of the elementary data unit may be predefined in the encoder and decoder.

[0107] It may be determined whether a division type of a coding block is allowed by considering the size of the basic data unit. For example, if a coding block generated as a result of the division of the coding block is larger than the basic data unit, such division may not be allowed. Alternatively, if a non-square coding block generated as a result of the division of the coding block is larger than the basic data unit, such division may not be allowed. For example, if the width or height of the coding block is larger than a threshold or the number of samples included in the coding block is larger than a threshold, a binary tree or ternary tree division may not be allowed. Therefore, encoding of information related to the binary tree or ternary tree division may be omitted.

[0108] Alternatively, it may be set that a coding block larger than the basic data unit must be split. Alternatively, it may be set that a binary tree partition or a ternary tree partition must be performed on a coding block larger than the basic data unit. Therefore, for a coding block larger than the basic data unit, a flag split_flag indicating whether the coding block has been split may not be coded, but the value of the flag may be derived as 1.

[0109] As another example, a coding block larger than a base data unit may be divided into multiple sub-blocks, where the sub-blocks may be set as prediction units as base units for prediction or as transform units as base units for transformation and / or quantization. In such a case, dividing the coding block into multiple prediction units may be defined as VPDU prediction unit division, and dividing the coding block into multiple transform units may be defined as VPDU transform unit division.

[0110] At least one of VPDU prediction unit splitting and VPDU transform unit splitting may be applied to the coding block. A split type of the coding block to which the VPDU prediction unit splitting is applied may be set to be the same as a split type of the coding block to which the VPDU transform unit splitting is applied.

[0111] When only VPDU prediction unit splitting is applied to a coding block, prediction is performed for each sub-block, but transformation and / or quantization may be performed for the coding block. In such a case, a prediction mode, such as a predictive coding mode, an intra-prediction mode, or an inter-prediction mode, may be determined for the coding block.

[0112] When only VPDU transform unit splitting is applied to a coding block, prediction is performed on sub-blocks, but transform and / or quantization may be performed on each sub-block.

[0113] 7 and 8 are diagrams showing an example of dividing a coding block into a plurality of sub-blocks.

[0114] FIG. 7 is a diagram showing the division modes when only square basic data units are allowed, and FIG. 8 is a diagram showing the division modes when square basic data units and non-square basic data units are allowed.

[0115] Assuming that only square basic data units are allowed, in Figures 7(a) and 7(b), CU0 and CU2 are defined as two different VPDUs, and CU1 is defined as four different VPDUs, so CU0 and CU2 may be split into two sub-blocks, and CU1 may be split into four sub-blocks.

[0116] Assuming that square and non-square basic data units are allowed, in Figures 8(a) and 8(b), CU0 and CU2 may be defined as one VPDU, and CU1 may be defined as two different VPDUs, so CU0 and CU2 are not divided into sub-blocks, but CU1 may be divided into two sub-blocks.

[0117] In such a case, CU1 may be divided into square or non-square sub-blocks. For example, CU1 may be divided into two square sub-blocks based on a horizontal line dividing CU1 from top to bottom. Alternatively, CU1 may be divided into two non-square sub-blocks based on a vertical line dividing CU1 from left to right.

[0118] If there are multiple partition type candidates to be applied to a coding block, information of any one of the multiple partition type candidates may be transmitted by a signal via the stream, for example, the information may indicate whether the coding block is partitioned into square sub-blocks or whether the coding block is partitioned into non-square sub-blocks.

[0119] Alternatively, splitting a coding block into square sub-blocks may be set to have a higher priority than splitting a coding block into non-square sub-blocks, e.g., if a coding block cannot be split into square sub-blocks, splitting the coding block into non-square sub-blocks may be allowed.

[0120] Alternatively, the division type of the coding block may be determined based on the division type of the coding block as the father node. For example, when the father node coding block is divided by a ternary tree, the coding block may be set to be divided into square sub-blocks. On the other hand, when the father node coding block is divided by a binary tree or a ternary tree, the coding block may be set to be divided into non-square sub-blocks.

[0121] Inter prediction refers to predicting the predictive coding mode of the current block using information of the previous picture. For example, a block in the previous picture that is located at the same position as the current block (hereinafter referred to as a collocated block) may be set as the prediction block of the current block. Hereinafter, a prediction block generated based on a block that is located at the same position as the current block is referred to as a collocated prediction block.

[0122] Meanwhile, if an object existing in a previous picture has already moved to another position in the current picture, the current block may be effectively predicted according to the motion of the object. For example, if the moving direction and size of an object can be known by comparing the previous picture with the current picture, a predicted block (or predicted picture) of the current block may be generated taking into account the motion information of the object. Hereinafter, the predicted block generated according to the motion information may be referred to as a motion predicted block.

[0123] A residual block can be generated by removing the prediction block from the current block. In this case, if an object moves, the motion prediction block can be substituted for the co-located prediction block. This can reduce the energy of the residual block and improve the compression performance of the residual block.

[0124] The process of generating a prediction block using motion information as described above may be referred to as motion compensated prediction. In most inter predictions, a prediction block may be generated by motion compensated prediction.

[0125] The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weighting value index. The motion vector indicates the moving direction and size of an object. The reference picture index specifies a reference picture of the current block among multiple reference pictures included in a reference picture list. The prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of the motion information in the L0 direction or the motion information in the L1 direction may be used based on the prediction direction of the current block. The bidirectional weighting value index specifies a weighting value for the L0 prediction block and a weighting value applied to the L1 prediction block.

[0126] FIG. 9 is a flowchart of an inter prediction method according to an embodiment of the present invention.

[0127] As shown in FIG. 9, the inter prediction method includes determining an inter prediction mode of a current block (S901), obtaining motion information of the current block based on the determined inter prediction mode (S902), and performing motion compensated prediction of the current block based on the obtained motion information (S903).

[0128] The inter prediction mode indicates a number of techniques for determining motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information may include a merge mode and an advanced motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. Depending on the inter prediction mode, the motion information of the current block may be determined based on information analyzed from a neighboring block or stream adjacent to the current block.

[0129] The motion information of the current block may be derived from the motion information of another block of the current block. The other block may be a block that is preferentially encoded / decoded by inter prediction rather than the current block. When the motion information of the current block is set to be the same as the motion information of the other block, it may be defined as a merge mode. And when the motion vector of the other block is set as a predicted value of the motion vector of the current block, it may be defined as a motion vector prediction mode.

[0130] FIG. 10 is a flowchart of a process for deriving motion information of a current block in the merge mode.

[0131] Merge candidates for a current block may be derived (S1001). The merging candidates for the current block may be derived from blocks that are encoded / decoded using inter prediction before the current block.

[0132] FIG. 11 is a diagram showing examples of candidate blocks for deriving merging candidates.

[0133] The candidate blocks may include at least one of adjacent blocks containing samples that are adjacent to the current block, or non-adjacent blocks containing samples that are not adjacent to the current block. Hereinafter, the samples for determining the candidate block are designated as reference samples, and the reference samples that are adjacent to the current block are referred to as adjacent reference samples, and the reference samples that are not adjacent to the current block are referred to as non-adjacent reference samples.

[0134] The adjacent reference samples may be included in adjacent columns of the leftmost column of the current block or adjacent rows of the topmost row of the current block. For example, if the coordinates of the top left sample of the current block are (0,0), at least one of a block including a reference sample at a (-1,H-1) position, a block including a reference sample at a (W-1,-1) position, a block including a reference sample at a (W,-1) position, a block including a reference sample at a (-1,H) position, or a block including a reference sample at a (-1,-1) position may be used as a candidate block. Referring to the drawing, adjacent blocks with indexes 0 to 4 may be used as candidate blocks.

[0135] The non-adjacent reference sample indicates a sample having at least one of the x-axis distance or y-axis distance from a reference sample adjacent to the current block having a predefined value. For example, at least one of a block including a reference sample having a predefined x-axis distance from a left reference sample, a block including a non-adjacent sample having a predefined y-axis distance from an upper reference sample, or a block including a non-adjacent sample having predefined x-axis and y-axis distances from an upper left reference sample may be used as a candidate block. The predefined value may be an integer such as 4, 8, 12, 16, etc. Referring to the drawing, at least one of blocks of indexes 5 to 26 may be used as a candidate block.

[0136] Alternatively, a candidate block that does not belong to the same coding tree unit as the current block may be set as ineligible for use as a merging candidate, for example, a candidate block containing a reference sample may be set as ineligible for use as a merging candidate if the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs.

[0137] Furthermore, merging candidates may be derived from temporally adjacent blocks included in a picture different from the current block. For example, merging candidates may be derived from co-located blocks included in a co-located picture. Any one of the multiple reference pictures included in the reference picture list may be set as the co-located picture. Index information for indicating the co-located picture among the multiple reference pictures may be transmitted by a signal via the stream. Alternatively, a reference picture having a predefined index among the multiple reference pictures may be determined as the co-located picture.

[0138] The motion information of the merging candidate may be set to be the same as the motion information of the candidate block. For example, at least one of the motion vector, the reference picture index, the prediction direction, or the bidirectional weighting value index of the candidate block may be set as the motion information of the merge candidate.

[0139] A merge candidate list may be generated that includes the merge candidates (S1002).

[0140] The indices of the multiple merging candidates in the merging candidate list may be assigned according to a predetermined order, for example, in the following order: merging candidates derived from the left neighboring block, merging candidates derived from the upper neighboring block, merging candidates derived from the upper right neighboring block, merging candidates derived from the lower left neighboring block, merging candidates derived from the upper left neighboring block, and merging candidates derived from the temporal neighboring blocks.

[0141] When the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates may be selected (S1003). Specifically, information for designating any one of the multiple merge candidates may be transmitted by a signal via the stream. For example, index information merge_idx indicating any one of the multiple merge candidates included in the merge candidate list may be transmitted by a signal via the stream.

[0142] If the number of merging candidates included in the merging candidate list is less than a threshold, merging candidates included in the inter motion information list may be added to the merging candidate list. The threshold may be the maximum number of merging candidates that can be included in the merging candidate list or the maximum number of merging candidates minus an offset amount. The offset amount may be an integer such as 1 or 2. The inter motion information list may include merging candidates derived based on blocks that are encoded / decoded before the current block.

[0143] The inter motion information list includes merge candidates derived from blocks in the current picture that are coded / decoded by inter prediction. For example, the motion information of the merge candidates included in the inter motion information list may be set to be the same as the motion information of the blocks that are coded / decoded by inter prediction. The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weighting value index.

[0144] A coding block may be divided into multiple prediction units, and prediction may be performed for each of the divided prediction units. A prediction unit refers to a basic unit for prediction.

[0145] The coding block may be divided using at least one of vertical lines, horizontal lines, diagonal lines, or diagonal lines. Information for determining at least one of the number, angle, or position of lines dividing the coding block may be transmitted by a signal via the stream. For example, information indicating any one of candidates for division type of the coding block may be transmitted by a signal via the stream, or information specifying any one of a plurality of line candidates dividing the coding block may be transmitted by a signal via the stream. Alternatively, information for determining the number or type of line candidates dividing the coding block may be transmitted by a signal via the stream. For example, a 1-bit flag may be used to determine whether a diagonal line with an angle larger than a diagonal and / or a diagonal line with an angle smaller than a diagonal can be used as a line candidate.

[0146] Alternatively, at least one of the number, angle or position of lines dividing the coding block may be adaptively determined based on at least one of the intra prediction mode, inter prediction mode, position of available merging candidates or division type of adjacent blocks of the coding block.

[0147] When a coding block is divided into multiple prediction units, intra prediction or inter prediction may be performed on each divided prediction unit.

[0148] FIG. 12 is a diagram showing an example of dividing a coding block into a plurality of prediction units using diagonals.

[0149] In the example shown in Figures 12(a) and 12(b), a diagonal may be used to divide the coding block into two triangular prediction units.

[0150] 12(a) and 12(b) show the division of a coding block into two prediction units using a diagonal line connecting two vertices of the coding block. However, a coding block may also be divided into two prediction units using a diagonal line where at least one end of the line does not pass through a vertex of the coding block.

[0151] FIG. 13 is a diagram showing an example of dividing a coding block into two prediction units.

[0152] In the examples shown in Figures 13(a) and 13(b), the coding block may be divided into two prediction units using a diagonal line whose ends touch the upper and lower boundaries of the coding block, respectively.

[0153] Alternatively, in the examples shown in Figures 13(c) and 13(d), the coding block may be divided into two prediction units using a diagonal line whose ends touch the left and right boundaries of the coding block, respectively.

[0154] Alternatively, the coding block may be divided into two prediction blocks of different sizes, for example, the diagonal line dividing the coding block is set to touch two boundary surfaces that form one vertex, thereby dividing the coding block into two prediction units of different sizes.

[0155] FIG. 14 is a diagram showing an example of dividing a coding block into a plurality of prediction blocks of different sizes.

[0156] In the examples shown in Figures 14(a) and 14(b), the coding block can be divided into two prediction units of different sizes by setting the diagonal line connecting the upper left or lower right corner of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block without passing through the upper left or lower right corner of the coding block.

[0157] Alternatively, in the examples shown in Figures 14(c) and 14(d), the coding block can be divided into two prediction units of different sizes by setting the diagonal line connecting at the upper right or lower left corner of the coding block to pass through the left, right, upper or lower boundary of the coding block without passing through the upper left or lower right corner of the coding block.

[0158] Each prediction unit generated by dividing the coding block is called an "Nth prediction unit". For example, in the examples shown in Figs. 12 to 14, PU1 may be defined as the first prediction unit, and PU2 may be defined as the second prediction unit. The first prediction unit may refer to a prediction unit including a lower left sample or an upper left sample in the coding block, and the second prediction unit may refer to a prediction unit including a upper right sample or a lower right sample in the coding block.

[0159] Conversely, a prediction unit including the top right sample or the bottom right sample in a coding block may be defined as the first prediction unit, and a prediction unit including the bottom left sample or the top left sample in a coding block may be defined as the second prediction unit.

[0160] The following embodiment mainly describes a diagonal division example. In particular, the process of dividing a coding block into two prediction units using a diagonal is called diagonal division or triangular division, and the prediction unit generated by diagonal division is called a triangular prediction unit. However, it is of course possible to use vertical, horizontal or oblique divisions at angles other than diagonal in the embodiments described below.

[0161] Whether to apply a diagonal split to a coding block may be determined based on at least one of the slice type, the maximum number of merge candidates that may be included in a merge candidate list, the size of the coding block, the shape of the coding block, the predictive coding mode of the coding block, or the split type of the parent node.

[0162] For example, whether to apply diagonal splitting to a coding block may be determined depending on whether the current slice is a B-slice, and diagonal splitting is only allowed if the current slice is a B-slice.

[0163] Alternatively, whether to apply a diagonal split to a coding block may be determined depending on whether the maximum number of merging candidates included in the merging candidate list is greater than or equal to 1. A diagonal split may be allowed only if the maximum number of merging candidates included in the merging candidate list is greater than or equal to 1.

[0164] Alternatively, if at least one of the width and height of the hardware is greater than 64, a data processing unit of 64×64 size will be accessed redundantly. This may not allow the coding block to be divided into multiple prediction blocks if at least one of the width and height of the coding block is greater than a threshold. For example, if at least one of the height and width of the coding block is greater than 64 (e.g., if at least one of the width and height is 128), diagonal division may not be used.

[0165] Alternatively, diagonal splitting may not be allowed for coding blocks with a number of samples greater than a threshold, taking into account the maximum number of samples that can be processed simultaneously in a hardware embodiment, e.g., diagonal splitting may not be allowed for coding tree blocks with a number of samples greater than 4096.

[0166] Alternatively, diagonal partitioning may not be allowed for coding blocks that contain fewer samples than a threshold value. For example, if the number of samples in a coding block is fewer than 64, diagonal partitioning may be set not to be applied to the coding block.

[0167] Alternatively, whether to apply diagonal division to the coding block may be determined according to whether the ratio of the width to the height of the coding block is smaller than a first threshold or whether the ratio of the width to the height of the coding block is greater than a second threshold, where the ratio of the width to the height of the coding block, whRatio, may be determined as the ratio of the width CbW and the height CbH of the coding block, as shown in Equation 1 below.

[0168] equation 1 JPEG2024138541000002.jpg544

[0169] The second threshold may be the reciprocal of the first threshold, for example, if the first threshold is k, the second threshold may be 1 / k.

[0170] Diagonal splitting can be applied to a coding block only if the ratio of the width to the height of the coding block is between a first threshold and a second threshold.

[0171] Alternatively, triangular partitioning can be used only if the width-to-height ratio of the coding block is less than a first threshold or greater than a second threshold, e.g., if the first threshold is 16, then diagonal partitioning is not allowed for coding blocks of size 64x4 or 4x64.

[0172] Alternatively, it may be determined whether diagonal splitting is allowed based on the split type of the parent node. For example, if the coding block as the parent node is split by quadtree splitting, diagonal splitting can be applied to the coding block as the leaf node. On the other hand, if the coding block as the parent node is split by binary or ternary tree splitting, the coding block as the leaf node is set so that diagonal splitting is not allowed.

[0173] Alternatively, it may be determined whether diagonal splitting is allowed based on the predictive coding mode of the coding block, for example, diagonal splitting may be allowed only if the coding block is coded by intra prediction, if the coding block is coded by inter prediction, or if the coding block is coded in a predefined inter prediction mode. The predefined inter prediction mode may indicate at least one of a merge mode, an advanced motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0174] Alternatively, it may be determined whether diagonal splitting is allowed based on the size of the parallel processing region, for example, if the size of the coding block is larger than the size of the parallel processing region, diagonal splitting may not be used.

[0175] Furthermore, it may be possible to determine whether to apply diagonal division to a coding block by taking into consideration two or more of the conditions listed above.

[0176] As another example, information indicating whether diagonal partitioning is applied to a coding block may be transmitted by a signal via the stream, and the information may be transmitted by a signal at a sequence level, a picture level, a slice level, or a block level. For example, at the coding block level, a signal may transmit a flag triangle_partition_flag indicating whether triangulation is applied to the coding block.

[0177] When deciding to apply a diagonal split to a coding block, the signal may transmit information via the stream indicating the number of lines or the position of the lines that split the coding block.

[0178] For example, when a coding block is divided by a diagonal, information indicating the direction of the diagonal dividing the coding block may be transmitted by a signal via a stream. For example, a flag triangle_partition_type_flag indicating the direction of the diagonal may be transmitted by a signal via a stream. The flag indicates whether the coding block is divided by a diagonal connecting the upper left and lower right, or whether the coding block is divided by a diagonal connecting the upper right and lower left. Dividing the coding block by a diagonal connecting the upper left and lower right is called a left triangular partition type, while dividing the coding block by a diagonal connecting the upper right and lower left is called a right triangular partition type. For example, when the value of the flag is 0, it indicates that the division type of the coding block is a left triangular partition type, and when the value of the flag is 1, it indicates that the division type of the coding block is a right triangular partition type.

[0179] Also, information indicating whether the prediction units have the same size or information indicating the position of a diagonal for dividing the coding block may be transmitted by a signal via the stream. For example, if the information indicating the size of the prediction units indicates that the sizes of the prediction units are the same, coding of the information indicating the diagonal position may be omitted, and the coding block may be divided into two prediction units using a diagonal passing through two vertices of the coding block. On the other hand, if the information indicating the size of the prediction units indicates that the sizes of the prediction units are different, the position of the diagonal for dividing the coding block may be determined based on the information indicating the position of the diagonal. For example, when a left triangular division type is applied to the coding block, the position information may indicate whether the diagonal touches the left and bottom boundaries or the top and right boundaries of the coding block. Alternatively, when a right triangular division type is applied to the coding block, the position information may indicate whether the diagonal touches the right and bottom boundaries or the top and left boundaries of the coding block.

[0180] Information indicating the partition type of a coding block may be signaled at the coding block level, which may determine the partition type for different coding blocks to which diagonal partitioning is applied.

[0181] As another example, information indicating the partition type may be transmitted by a signal for a sequence, a picture, a slice, a segment, or a coding tree unit. In such a case, the partition type of the coding block to which the diagonal partition is applied may be set to be the same in the sequence, the picture, the slice, the segment, or the coding tree unit.

[0182] Alternatively, for a first coding unit that applies a diagonal partition in a coding tree unit, information for determining the partition type is coded and signaled, and a second and subsequent coding units that apply diagonal partition are configured to use the same partition type as the first coding unit.

[0183] As another example, the partition type of the coding block may be determined based on the partition type of the neighboring blocks. The neighboring blocks may include at least one of the neighboring blocks adjacent to the upper left corner of the coding block, the neighboring block adjacent to the upper right corner, the neighboring block adjacent to the lower left corner, the neighboring block located above, or the neighboring block located to the left. For example, the partition type of the current block may be set to be the same as the partition type of the neighboring blocks. Alternatively, the partition type of the current block may be determined according to whether the upper left neighboring block applies a left triangular partition type, and whether the upper right neighboring block or the lower left neighboring block applies a right triangular partition type.

[0184] The size of the prediction unit may be set to be equal to or smaller than the size of the transform unit, and one transform unit may be set to have to include one prediction unit. Thus, when applying VPDU transform unit partitioning to the coding block, instead of applying diagonal partitioning to the coding block, diagonal partitioning may be applied to the transform unit.

[0185] FIG. 15 is a diagram showing an example of applying diagonal division to a transform unit.

[0186] In the example shown in Figure 15, when a coding block is divided into multiple transform units, it may be determined whether to apply diagonal division to each transform unit. Alternatively, it may be determined whether to apply diagonal division to each block whose size is smaller than the size of the transform unit.

[0187] Alternatively, when applying VPDU transform unit splitting to a coding block, it may be determined whether to apply diagonal splitting to the coding block, and all transform units included in the coding block may be set to comply with the determination. For example, when it is determined to apply diagonal splitting to the coding block, all transform units included in the coding block may be split into two prediction units.

[0188] In order to perform motion prediction compensation for the first triangular prediction unit and the second triangular prediction unit, motion information of each of the first triangular prediction unit and the second triangular prediction unit may be derived. In this case, the motion information of the first triangular prediction unit and the second triangular prediction unit may be derived from merge candidates included in the merge candidate list. In order to distinguish between a general merge candidate list and a merge candidate list used in deriving motion information of a triangular prediction unit, the merge candidate list for deriving motion information of a triangular prediction unit is referred to as a triangle merge candidate list, and the merge candidates included in the triangle merge candidate list are referred to as triangle merge candidates. However, it is also within the spirit of the present invention to use the above merge candidate derivation method and merge candidate list construction method for a triangle merge candidate and a triangle merge candidate list construction method.

[0189] The signal may transmit information over the stream to determine a maximum number of triangle merge candidates that may be included in the triangle merge candidate list, and the information may indicate a difference between the maximum number of merge candidates that may be included in the merge candidate list and the maximum number of triangle merge candidates that may be included in the triangle merge candidate list.

[0190] Triangle merging candidates may be derived from the spatial and temporal neighboring blocks of a coding block.

[0191] FIG. 16 is a diagram showing adjacent blocks for deriving triangle merging candidates.

[0192] At least one of a neighboring block located above the coding block, a neighboring block located to the left of the coding block, or a co-located block included in a different picture from the coding block may be used to derive triangle merging candidates. The upper neighboring block may include at least one of a block including samples (xCb+CbW-1, yCb-1) located above the coding block, a block including samples (xCb+CbW, yCb-1) located above the coding block, or a block including samples (xCb-1, yCb-1) located above the coding block. The left neighboring block may include at least one of a block including samples (xCb-1, yCb+CbH-1) located to the left of the coding block, or a block including samples (xCb-1, yCb+CbH) located to the left of the coding block. The collocated block may be determined as either the block containing samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coding block in the collocated picture, or the block containing samples (xCb / 2, yCb / 2) located at the center of the coding block.

[0193] The neighboring blocks may be searched according to a predefined order, and the triangle merge candidates may be constructed into a triangle merge candidate list according to a predefined order, for example, the triangle merge candidates may be searched according to the order of B1, A1, B0, A0, C0, B2, and C1 to construct the triangle merge candidate list.

[0194] The motion information of the triangular prediction units may be derived based on the triangle merging candidate list, i.e., the triangular prediction units may share one triangle merging candidate list.

[0195] To derive the motion information of the triangle merging unit, information for specifying at least one of the triangle merging candidates included in the triangle merging candidate list may be transmitted by a signal via the stream, for example, index information merge_triangle_idx for specifying at least one of the triangle merging candidates may be transmitted by a signal via the stream.

[0196] The index information may specify a combination of a merging candidate of the first triangular prediction unit and a merging candidate of the second triangular prediction unit. For example, Table 1 below shows an example of a combination of merging candidates based on the index information merge_triangle_idx.

[0197] [Table 1]

[0198] When the value of the index information merge_triangle_idx is 1, it indicates that the motion information of the first triangular prediction unit is derived from the merge candidate with index 1, and the motion information of the second triangular prediction unit is derived from the merge candidate with index 0. The index information merge_triangle_idx may determine the triangle merge candidate for deriving the motion information of the first triangular prediction unit and the triangle merge candidate for deriving the motion information of the second triangular prediction unit.

[0199] Furthermore, the partition type of the coding block to which the diagonal partition is applied may be determined based on the index information. That is, the index information may specify a combination of merge candidates of the first triangular prediction unit, merge candidates of the second triangular prediction unit, and a partition direction of the coding block. When the partition type of the coding block is determined based on the index information, it is not necessary to code the information triangle_partition_type_flag indicating the direction of the diagonal line that divides the coding block. Table 2 shows the partition types of the coding block with respect to the index information merge_triangle_idx.

[0200] [Table 2]

[0201] When the variable TriangleDir is 0, it indicates that the coding block applies a left triangular division type, and when the variable TriangleDir is 1, it indicates that the coding block applies a right triangular division type. By combining Table 1 and Table 2, a combination of merging candidates for the first triangular prediction unit, merging candidates for the second triangular prediction unit, and division direction of the coding block may be specified based on the index information merge_triangle_idx.

[0202] As another example, index information used for only one of the first triangular prediction unit and the second triangular prediction unit may be transmitted by a signal, and an index of a triangle merging candidate used for the other of the first triangular prediction unit and the second triangular prediction unit may be determined based on the index information. For example, a triangle merging candidate for the first triangular prediction unit may be determined based on index information merge_triangle_idx indicating an index of any one of the triangle merging candidates. In addition, a triangle merge candidate of the second triangular prediction unit may be specified based on the merge_triangle_idx. For example, a triangle merge candidate of the second triangular prediction unit may be derived by adding an offset amount to the index information merge_triangle_idx or subtracting an offset amount from the index information merge_triangle_idx. The offset amount may be an integer such as 1 or 2. For example, a triangle merge candidate of the second triangular prediction unit may be determined as a triangle merge candidate having an index obtained by adding 1 to merge_traingle_idx. When merge_triangle_idx indicates a triangle merge candidate with the largest index value among the triangle merge candidates, motion information of the second triangular prediction unit may be derived from a triangle merge candidate with index 0 or a triangle merge candidate having an index obtained by subtracting 1 from merge_triangle_idx.

[0203] Alternatively, the motion information of the second triangular prediction unit may be derived from a triangle merge candidate having the same reference picture as the triangle merge candidate of the first triangular prediction unit specified based on the index information. The triangle merge candidate having the same reference picture as the triangle merge candidate of the first triangular prediction unit may indicate at least one of an L0 reference picture or an L1 reference picture having the same triangle merge candidate as the triangle merge candidate of the first triangular prediction unit. When there are multiple triangle merge candidates whose reference picture is the same as the triangle merge candidate of the first triangular prediction unit, any one of them may be selected according to at least one of whether the merge candidate includes bidirectional motion information or a difference between the index of the merge candidate and the index information.

[0204] As another example, index information may be transmitted by a signal to each of the first triangular prediction unit and the second triangular prediction unit. For example, first index information 1st_merge_idx for determining triangle merge candidates for the first triangular prediction unit and second index information 2nd_merge_idx for determining triangle merge candidates for the second triangular prediction unit may be transmitted by a signal via a stream. Motion information of the first triangular prediction unit may be derived from the triangle merge candidates determined based on the first index information 1st_merge_idx, and motion information of the second triangular prediction unit may be derived from the triangle merge candidates determined based on the second index information 2nd_merge_idx.

[0205] The first index information 1st_merge_idx may indicate an index of any one of the triangle merge candidates included in the triangle merge candidate list. The triangle merge candidate of the first triangular prediction unit may be determined as the triangle merge candidate pointed to by the first index information 1st_merge_idx.

[0206] The triangle merge candidate pointed to by the first index information 1st_merge_idx is set not to be used as a triangle merge candidate of the second triangular prediction unit. Thus, the second index information 2nd_merge_idx of the second triangular prediction unit may indicate an index of any one of the remaining triangle merge candidates other than the triangle merge candidate pointed to by the first index information. If the value of the second index information 2nd_merge_idx is smaller than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangular prediction unit may be determined as the triangle merge candidate having index information indicated by the second index information 2nd_merge_idx. On the other hand, if the value of the second index information 2nd_merge_idx is the same as or larger than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangular prediction unit may be determined as the triangle merge candidate having an index value obtained by adding 1 to the value of the second index information 2nd_merge_idx.

[0207] Alternatively, it may be determined whether to transmit the second index information by a signal based on the number of triangle merge candidates included in the triangle merge candidate list. For example, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is less than or equal to 2, transmitting the second index information by a signal may be omitted. If transmitting the second index information by a signal is omitted, the second triangle merge candidate may be derived by adding an offset amount to or subtracting an offset amount from the first index information. For example, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is 2 and the first index information is index 0, the second triangle merge candidate may be derived by adding 1 to the first index information. Alternatively, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is 2 and the first index information is 1, the second triangle merge candidate may be derived by subtracting 1 from the first index information.

[0208] Alternatively, when the second index information is omitted from being transmitted by the signal, the second index information may be set as a default value. The default value may be 0. The second triangle merging candidate may be derived by comparing the first index information with the second index information. For example, if the second index information is smaller than the first index information, the merging candidate with index 0 is set as the second triangle merging candidate, and if the second index information is equal to or larger than the first index information, the merging candidate with index 1 is set as the second triangle merging candidate.

[0209] If the triangle merge candidate has unidirectional motion information, the unidirectional motion information of the triangle merge candidate is set as the motion information of the triangle prediction unit. On the other hand, if the triangle merge candidate has bidirectional motion information, only one of the L0 motion information or the L1 motion information is set as the motion information of the triangle prediction unit. It may be determined which of the L0 motion information or the L1 motion information to obtain based on the index of the triangle merge candidate or the motion information of other triangle prediction units.

[0210] For example, when the index of the triangle merge candidate is an even number, the L0 motion information of the triangle prediction unit is set as 0, and the L1 motion information of the triangle merge candidate is set as the L1 motion information of the triangle prediction unit. On the other hand, when the index of the triangle merge candidate is an odd number, the L1 motion information of the triangle prediction unit is set as 0, and the L0 motion information of the triangle merge candidate is set as 0. Conversely, when the index of the triangle merge candidate is an even number, the L0 motion information of the triangle merge candidate may be set as the L0 motion information of the triangle prediction unit, and when the index of the triangle merge candidate is an odd number, the L1 motion information of the triangle merge candidate may be further set as the L1 motion information of the triangle prediction unit. Alternatively, when the triangle merge candidate is an even number for the first triangle prediction unit, the L0 motion information of the triangle merge candidate is set as the L0 motion information of the first triangle prediction unit, while when the triangle merge candidate is an odd number for the second triangle prediction unit, the L1 motion information of the triangle merge candidate is set as the L1 motion information of the second triangle prediction unit.

[0211] Alternatively, if the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit is set as 0, and the L1 motion information of the triangle merging candidate is set as the L1 information of the second triangle prediction unit; on the other hand, if the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit is set as 0, and the L0 motion information of the triangle merging candidate is set as the L0 motion signal of the second triangle prediction unit.

[0212] Furthermore, the triangle merging candidate list for deriving motion information of the first triangular prediction unit and the triangle merging candidate list for deriving motion information of the second triangular prediction unit may be set to be different from each other.

[0213] For example, when a triangle merge candidate for deriving motion information of the first triangular prediction unit in a triangle merge candidate list is specified based on index information related to the first triangular prediction unit, the motion information of the second triangular prediction unit may be derived using a triangle merge list including remaining triangle merge candidates other than the triangle merge candidate indicated by the index information. Specifically, the motion information of the second triangular prediction unit may be derived from any one of the remaining triangle merge candidates.

[0214] As a result, the number of maximum triangle merge candidates included in the triangle merge candidate list of the first triangular prediction unit is different from the number of maximum triangle merge candidates included in the triangle merge candidate list of the second triangular prediction unit. For example, if the triangle merge candidate list of the first triangular prediction unit includes M merge candidates, the triangle merge candidate list of the second triangular prediction unit may include M-1 merge candidates other than the triangle merge candidate pointed to by the index information of the first triangular prediction unit.

[0215] As another example, merging candidates for each triangular prediction unit may be derived based on neighboring blocks adjacent to the coding block, and the availability of neighboring blocks may be determined taking into account the shape or position of the triangular prediction unit.

[0216] FIG. 17 is a diagram for explaining an example of determining the availability of adjacent blocks for each triangular prediction unit.

[0217] An adjacent block that is not adjacent to the first triangular prediction unit may be set as an unavailable adjacent block for the first triangular prediction unit, and an adjacent block that is not adjacent to the second triangular prediction unit may be set as an unavailable adjacent block for the second triangular prediction unit.

[0218] 17(a), when a left triangulation type is applied to a coding block, it can be determined that blocks A1, A0, and A2 adjacent to the first triangular prediction unit among adjacent blocks adjacent to the coding block can be used for the first triangular prediction unit, but blocks B0 and B1 cannot be used for the first triangular prediction unit, so that a triangle merge candidate list related to the first triangular prediction unit includes triangle merge candidates derived from blocks A1, A0, and A2, but does not include triangle merge candidates derived from blocks B0 and B1.

[0219] 17(b), when applying a left triangulation type to a coding block, it can be determined that blocks B0 and B1 adjacent to the second triangular prediction unit can be used for the second triangular prediction unit, but blocks A1, A0, and A2 cannot be used for the second triangular prediction unit, so that a triangle merge candidate list related to the second triangular prediction unit includes triangle merge candidates derived from blocks B0 and B1, but does not include triangle merge candidates derived from blocks A1, A0, and A2.

[0220] Thereby, the number of triangle merging candidates or the range of triangle merging candidates that the triangular prediction unit can use may be determined based on at least one of the position of the triangular prediction unit or the division type of the coding block.

[0221] As another example, a merge mode may be applied to only one of the first triangular prediction unit and the second triangular prediction unit, and motion information of the other of the first triangular prediction unit and the second triangular prediction unit may be set to be the same as the motion information of the triangular prediction unit to which the merge mode is applied, or may be derived by subdividing the motion information of the triangular prediction unit to which the merge mode is applied.

[0222] For example, the motion vector and reference picture index of the first triangular prediction unit may be derived based on the triangle merging candidate, and the motion vector of the second triangular prediction unit may be derived by subdividing the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding the fine motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit, or by subtracting the fine motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0223] The signal may transmit information for determining a fine motion vector indicating a difference between a motion vector of the first triangular prediction unit and a motion vector of the second triangular prediction unit via the stream, the information including at least one of information indicating a size of the fine motion vector or information indicating a symbol of the fine motion vector.

[0224] Alternatively, the symbols of the fine motion vector may be derived based on at least one of the position, index, or partition type applied to the coding block of the triangular prediction unit.

[0225] As another example, a motion vector and a reference picture index of one of the first triangular prediction unit and the second triangular prediction unit may be transmitted by a signal, and the motion vector of the other of the first triangular prediction unit and the second triangular prediction unit may be derived by subdividing the motion vector transmitted by the signal.

[0226] For example, the motion vector and the reference picture index of the first triangular prediction unit may be determined based on information transmitted by a signal from the stream. And the motion vector of the second triangular prediction unit may be derived by subdividing the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding the fine motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit, or by subtracting the fine motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0227] Based on the motion information of the first triangular prediction unit and the motion information of the second triangular prediction unit, motion prediction and compensation prediction may be performed on the coding blocks, respectively. In this case, image quality may be degraded at the boundary between the first triangular prediction unit and the second triangular prediction unit. For example, image quality may be continuously degraded near the edge on the boundary between the first triangular prediction unit and the second triangular prediction unit. In order to reduce image quality degradation at the boundary, predicted samples may be derived by smoothing filtering or weighted prediction.

[0228] A prediction sample to which diagonal division is applied in the coding block may be derived by performing a weighted sum operation of a first prediction sample obtained based on the motion information of the first triangular prediction unit and a second prediction sample obtained based on the motion information of the second triangular prediction unit. Alternatively, a prediction sample of the first triangular prediction unit may be derived from a first prediction block determined based on the motion information of the first triangular prediction unit, a prediction sample of the second triangular prediction unit may be derived from a second prediction block determined based on the motion information of the second triangular prediction unit, and a prediction sample located in a boundary area between the first triangular prediction unit and the second triangular prediction unit may be derived by performing a weighted sum operation of the first prediction sample included in the first prediction block and the second prediction sample included in the second prediction block. For example, the following equation 2 shows an example of deriving prediction samples of the first triangular prediction unit and the second triangular prediction unit.

[0229] equation 2 JPEG2024138541000005.jpg572

[0230] In Equation 2, P1 denotes the first predicted sample, P2 denotes the second predicted sample, w1 denotes the weighting value applied to the first predicted sample, and (1-w1) denotes the weighting value applied to the second predicted sample. In the example shown in Equation 2, the weighting value applied to the second predicted sample can be derived by subtracting the weighting value applied to the first predicted sample from a constant.

[0231] When a left triangulation type is applied to a coding block, the boundary region may include prediction samples having the same x-axis and y-axis coordinates, whereas when a right triangulation type is applied to a coding block, the boundary region may include prediction samples whose sum of the x-axis and y-axis coordinates is greater than or equal to a first threshold and less than or equal to a second threshold.

[0232] The size of the boundary area may be determined based on at least one of the size of the coding block, the shape of the coding block, the motion information of the triangular prediction unit, the motion vector difference of the triangular prediction unit, the output order of the reference picture, or the difference between the first prediction sample and the second prediction sample at the diagonal boundary.

[0233] Figures 18 and 19 are diagrams showing an example of deriving a predicted sample by performing a weighted sum operation on a first predicted sample and a second predicted sample. Figure 18 shows an example of applying a left triangular division type to a coding block, and Figure 19 shows an example of applying a right triangular division type to a coding block. Also, Figures 18(a) and 19(a) show prediction patterns for the luma component, and Figures 18(b) and 19(b) show prediction patterns for the chroma component.

[0234] In the drawing, a number written in a prediction sample near the boundary between the first prediction unit and the second prediction unit indicates a weighting value applied to the first prediction sample. For example, if the number written in a prediction sample is N, a weighting value of N / 8 is applied to the first prediction sample, and a weighting value of (1-(N / 8)) is applied to the second prediction sample, thereby deriving the prediction sample.

[0235] In the non-boundary region, the first prediction sample or the second prediction sample may be determined as the prediction sample. In the example of Fig. 18, in the region where the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold and belongs to the first triangular prediction unit, the first prediction sample derived based on the motion information of the first triangular prediction unit may be determined as the prediction sample. On the other hand, in an area belonging to the second triangular prediction unit among areas where the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, a second prediction sample derived based on motion information of the second triangular prediction unit may be determined as the prediction sample.

[0236] 19, in a region where the sum of the x-axis coordinate and the y-axis coordinate is smaller than a first threshold, a first prediction sample derived based on the motion information of a first triangular prediction unit may be determined as a prediction sample, while in a region where the sum of the x-axis coordinate and the y-axis coordinate is larger than a second threshold, a second prediction sample derived based on the motion information of a second triangular prediction unit may be determined as a prediction sample.

[0237] The threshold for determining the non-border region may be determined based on at least one of the size of the coding block, the shape of the coding block, or the color component. For example, if the threshold associated with the luma component is set as N, the threshold associated with the chroma component may be set as N / 2.

[0238] A predicted sample included in the boundary region may be derived by performing a weighted sum operation on the first predicted sample and the second predicted sample. In such a case, weighting values ​​to be applied to the first predicted sample and the second predicted sample may be determined based on at least one of a position of the predicted sample, a size of the coding block, a shape of the coding block, or a color component.

[0239] For example, in the example shown in Fig. 18(a), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample at a position having the same x-axis coordinate and y-axis coordinate can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (3:1) or (1:3), a predicted sample whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived. And by setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 2 can be derived.

[0240] Alternatively, in the example shown in Figure 18(b), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample at a position having the same x-axis coordinate and y-axis coordinate can be derived, and by setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived.

[0241] For example, in the example shown in FIG. 19(a), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (3:1) or (1:3), a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as the width or height of the coding block or 2 less than the width or height of the coding block can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 more than the width or height of the coding block or 3 less than the width or height of the coding block can be derived.

[0242] Alternatively, in the example shown in Fig. 19(b), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as the width or height of the coding block or 2 less than the width or height of the coding block can be derived.

[0243] As another example, the weighting values ​​may be determined taking into account the position of the prediction sample or the shape of the coding block. Equations 3 to 5 show examples of deriving the weighting values ​​when a left triangulation type is applied to the coding block. Equation 3 shows an example of deriving the weighting value to be applied to the first prediction sample when the coding block is square.

[0244] equation 3 JPEG2024138541000006.jpg530

[0245] In Equation 3, x and y indicate the location of the predicted sample. If the coding block is non-square, the weighting value to be applied to the first predicted sample may be derived as shown in Equation 4 or 5. Equation 4 illustrates the case where the width of the coding block is greater than the height, and Equation 5 illustrates the case where the width of the coding block is less than the height.

[0246] equation 4 JPEG2024138541000007.jpg551

[0247] Equation 5 JPEG2024138541000008.jpg551

[0248] When a right triangulation type is applied to the coding block, the weighting values ​​to be applied to the first predicted samples may be determined as shown in Equations 6 to 8. Equation 6 shows an example of deriving the weighting values ​​to be applied to the first predicted samples when the coding block is square.

[0249] Equation 6 JPEG2024138541000009.jpg547

[0250] In Equation 6, CbW denotes the width of the coding block. If the coding block is non-square, we may derive the weighting value to be applied to the first predicted sample as shown in Equation 7 or 8. Equation 7 illustrates the case where the coding block width is greater than the height, and Equation 8 illustrates the case where the coding block width is less than the height.

[0251] Equation 7 JPEG2024138541000010.jpg566

[0252] equation 8 JPEG2024138541000011.jpg567

[0253] In Equation 7, CbH denotes the height of the coding block.

[0254] In the illustrated example, for prediction samples in a boundary region, a first prediction sample is given a higher weighting value than the second prediction sample to derive a sample included in a first triangular prediction unit, and a second prediction sample is given a higher weighting value than the first prediction sample to derive a sample included in a second triangular prediction unit.

[0255] When diagonal division is applied to a coding block, the coding block may be set not to apply a combined prediction mode that combines an intra prediction mode and a merge mode.

[0256] Intra prediction is to predict the current block using reconstructed samples after encoding / decoding of the surrounding block. In this case, the intra prediction of the current block may use reconstructed samples before applying the in-loop filter.

[0257] The intra prediction techniques include matrix-based intra prediction and general intra prediction considering the directionality of surrounding reconstructed samples. Information indicating the intra prediction technique of the current block may be transmitted by a signal through the stream. The information may be a 1-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, shape, or intra prediction techniques of the neighboring blocks of the current block. For example, if the current block is located across a picture boundary, the current block is set not to apply matrix-based intra prediction.

[0258] Matrix-based intra prediction is a method of obtaining a prediction block of a current block by matrix multiplication of a matrix stored in an encoder and a decoder and a reconstructed sample around the current block. Information for specifying one of a plurality of stored matrices may be transmitted by a signal via a stream. The decoder may determine a matrix for intra prediction of the current block based on the information and the size of the current block.

[0259] General intra prediction is a method of obtaining a predicted block related to a current block based on a non-angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in more detail with reference to the accompanying drawings.

[0260] FIG. 20 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention.

[0261] A reference sample line of the current block may be determined (S2001). The reference sample line refers to a set of reference samples included in the Kth row from the top and / or left side of the current block. The reference samples may be derived from reconstructed samples after encoding / decoding around the current block.

[0262] Index information indicating a reference sample line of a current block among a plurality of reference sample lines may be transmitted by a signal via a stream. For example, index information intra_luma_ref_idx for specifying a reference sample line of a current block may be transmitted by a signal via a stream. The index information may be transmitted by a signal in units of coding blocks.

[0263] The plurality of reference sample lines may include at least one of a first line, a second line, a third line, or a fourth line located above and / or to the left of the current block. Of the plurality of reference sample lines, a reference sample line consisting of a row adjacent to the upper side of the current block and a column adjacent to the left side of the current block may be referred to as an adjacent reference sample line, and the other reference sample lines may be further referred to as non-adjacent reference sample lines.

[0264] Some of the multiple reference sample lines may be selected as the reference sample lines of the current block. For example, the remaining reference sample lines other than the third non-adjacent reference sample line among the multiple reference sample lines may be set as candidate reference sample lines. Table 3 shows the indexes assigned to each candidate reference sample line.

[0265] [Table 3]

[0266] Candidate reference sample lines may be set more than the described sample lines, or less than the described sample lines. And the number or positions of non-adjacent reference sample lines set as candidate reference sample lines are not limited to the described examples. For example, the first non-adjacent reference sample line and the third non-adjacent reference sample line may be set as candidate reference sample lines, or the second non-adjacent reference sample line and the third non-adjacent reference sample line may be set as candidate reference sample lines. Alternatively, the first non-adjacent reference sample line, the second non-adjacent reference sample line, and the third non-adjacent reference sample line may all be set as candidate reference sample lines.

[0267] The number or type of candidate reference sample lines may be determined based on at least one of the size, shape, position of the current block, whether the sub-blocks are split, or the intra-prediction mode.

[0268] The reference sample line of the current block may further be determined based on at least one of the position, size, shape, or predictive coding mode of the neighboring block of the current block. For example, when the current block contacts the boundary of a picture, a segment, a slice, or a coding tree unit, the first reference sample line may be determined as the reference sample line of the current block.

[0269] Alternatively, if the current block is non-square, the adjacent reference sample line may be set as the reference sample line of the current block. Alternatively, if the width-to-height ratio of the current block is greater than or equal to a threshold, the adjacent reference sample line may be determined as the reference sample line of the current block.

[0270] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper reference sample and the left reference sample may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before applying an in-loop filter.

[0271] Next, an intra prediction mode of the current block may be determined (S2002). For the intra prediction mode of the current block, at least one of a non-angular intra prediction mode or an angular intra prediction mode may be determined as the intra prediction mode of the current block. The non-angular intra prediction modes include planar and DC, and the angular intra prediction modes include 33 or 65 modes from the lower left diagonal to the upper right diagonal.

[0272] FIG. 21 is a diagram showing intra prediction modes.

[0273] FIG. 21(a) shows 35 intra prediction modes, and FIG. 21(b) shows 67 intra prediction modes.

[0274] More or less intra-prediction modes than those shown in FIG. 21 may be defined.

[0275] A Most Probable Mode (MPM) may be set based on intra-prediction modes of neighboring blocks adjacent to the current block. The neighboring blocks may include a left neighboring block adjacent to the left of the current block and an upper neighboring block adjacent above the current block.

[0276] The number of MPMs included in the MPM list in the encoder and decoder may be predefined. For example, the number of MPMs may be three, four, five, or six. Alternatively, information indicating the number of MPMs may be transmitted by a signal through the stream. Alternatively, the number of MPMs may be determined based on at least one of the predictive coding mode of the neighboring block, the size or shape of the current block. For example, when determining a neighboring reference sample line as the reference sample line of the current block, N MPMs may be used, but when determining a non-neighboring reference sample line as the reference sample line of the current block, M MPMs may be used. M is an integer smaller than N, for example, N may be 6, and M may be 5, 4, or 3. Thus, when the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra prediction mode of the current block may be determined as any one of six candidate intra prediction modes, but when the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra prediction mode of the current block may be determined as any one of five candidate intra prediction modes.

[0277] Alternatively, a fixed number of MPM candidates (eg, six or five) may be used regardless of the index of the reference sample line of the current block.

[0278] An MPM list including a plurality of MPMs may be generated, and information indicating whether the same MPM as the intra prediction mode of the current block is included in the MPM list may be transmitted by a signal via the stream. The information may be a 1-bit flag and may be referred to as an MPM flag. If the MPM flag indicates that the same MPM as the current block is included in the MPM list, index information indicating one of the MPMs may be transmitted by a signal via the stream. For example, index information mpm_idx specifying any one of the plurality of MPMs may be transmitted by a signal via the stream. The MPM specified by the index information may be set as the intra prediction mode of the current block. If the MPM flag indicates that the same MPM as the current block is not included in the MPM list, remaining mode information indicating any one of the remaining intra prediction modes other than the MPM may be transmitted by a signal via the stream. The remaining mode information indicates an index value corresponding to the intra prediction mode of the current block when reassigning an index to the remaining intra prediction mode other than the MPM. The decoder may determine the intra prediction mode of the current block by arranging the MPMs in ascending order and comparing the remaining mode information with the MPM. For example, if the residual mode information is equal to or less than the MPM, the intra prediction mode of the current block may be derived by adding 1 to the residual mode information.

[0279] When deriving the intra prediction mode of the current block, comparison of some of the MPMs with the remaining mode information may be omitted. For example, the MPM of the non-angular intra prediction mode may be excluded from the comparison object. When the non-angular intra prediction mode is set as the MPM, the remaining mode information clearly indicates the angular intra prediction mode, so that the intra prediction mode of the current block may be derived by comparing the remaining MPMs other than the non-angular intra prediction mode with the remaining mode information. Instead of excluding the non-angular intra prediction mode from the comparison object, a result value obtained by adding the number of non-angular intra prediction modes to the remaining mode information may be compared with the MPM.

[0280] Instead of setting the default mode as the MPM, information indicating whether the intra prediction mode of the current block is the default mode may be transmitted by a signal via the stream. The information may be a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be transmitted by a signal only if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode may include at least one of planar, DC, vertical mode, or horizontal mode. For example, if planar is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is planar. If the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

[0281] When the default mode flag is used, the intra prediction mode that is the same as the default mode may not be set as the MPM. For example, if the default mode flag indicates whether the intra prediction mode of the current block is planar, the intra prediction mode of the current block may be derived using five MPMs other than the MPM corresponding to planar.

[0282] When multiple intra prediction modes are set as default modes, index information indicating one of the default modes may be further transmitted by a signal, and the intra prediction mode of the current block may be set as the default mode indicated by the index information.

[0283] The default mode is set not to be used if the index of the reference sample line of the current block is not 0. For example, if a non-adjacent reference sample line is determined as the reference sample line of the current block, a non-angular intra prediction mode such as DC mode or planar mode may be set not to be used. Thus, if the index of the reference sample line is not 0, the default mode flag may not be transmitted by a signal, and the value of the default mode flag may be set to a predefined value (i.e., false).

[0284] When the intra-prediction mode of the current block is determined, a prediction sample of the current block may be obtained based on the determined intra-prediction mode (S2003).

[0285] When the DC mode is selected, a predicted sample related to the current block may be generated based on an average value of reference samples. Specifically, values ​​of all samples in the predicted block may be generated based on the average value of reference samples. The average value may be derived using at least one of an upper reference sample located above the current block and a left reference sample located to the left of the current block.

[0286] The number or range of reference samples used in deriving the average value varies depending on the shape of the current block. For example, if the current block is a non-square block with a width greater than its height, only the upper reference samples may be used to calculate the average value. On the other hand, if the current block is a non-square block with a width less than its height, only the left reference samples may be used to calculate the average value. That is, if the width and height of the current block are different, only the reference samples adjacent to the longer side may be used to calculate the average value. Alternatively, it may be determined whether to calculate the average value using only the upper reference samples or only the left reference samples based on the ratio of the width and height of the current block.

[0287] When the planar mode is selected, the prediction sample may be obtained using the horizontal prediction sample and the vertical prediction sample. The horizontal prediction sample is obtained based on the left reference sample and the right reference sample located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on the upper reference sample and the lower reference sample located on the same vertical line as the prediction sample. The right reference sample may be generated by copying the reference sample adjacent to the upper right corner of the current block, and the lower reference sample may be generated by copying the reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample may be obtained by a weighted sum operation of the left reference sample and the right reference sample, and the vertical prediction sample may be obtained by a weighted sum operation of the upper reference sample and the lower reference sample. In this case, a weighting value to be given to each reference sample may be determined based on the position of the prediction sample. The prediction sample may be obtained by an average operation or a weighted sum operation of the horizontal prediction sample and the vertical prediction sample. When performing the weighted sum operation, a weighting value to be given to the horizontal prediction sample and the vertical prediction sample may be determined based on the position of the prediction sample.

[0288] When an angular prediction mode is selected, a parameter indicating a prediction direction (or a prediction angle) of the selected angular prediction mode may be determined. Table 4 below shows an intra prediction parameter intraPredAng for each intra prediction mode.

[0289] [Table 4]

[0290] Table 4 shows, when 35 intra prediction modes are defined, the intra direction parameters of each intra prediction mode having one of indexes 2 to 34. When 33 or more angular intra prediction modes are defined, Table 4 is further subdivided to set the intra direction parameters of each angular intra prediction mode.

[0291] The upper reference sample and the left reference sample of the current block may be arranged in a row, and then a predicted sample may be obtained based on the value of the intra direction parameter. In this case, if the value of the intra direction parameter is a negative number, the left reference sample and the upper reference sample may be arranged in a row.

[0292] 22 and 23 are diagrams showing examples of a one-dimensional array in which reference samples are arranged in one row.

[0293] Figure 22 shows an example of a one-dimensional vertical array in which reference samples are arranged in the vertical direction, and Figure 23 shows an example of a one-dimensional horizontal array in which reference samples are arranged in the horizontal direction. The embodiments of Figures 22 and 23 will be described assuming that 35 intra prediction modes are defined.

[0294] If the intra prediction mode index is one of 11 to 18, a horizontal one-dimensional array in which the upper reference sample is rotated counterclockwise may be applied. If the intra prediction mode index is one of 19 to 25, a vertical one-dimensional array in which the left reference sample is rotated clockwise may be applied. When arranging the reference samples in a row, the intra prediction mode angle may be taken into consideration.

[0295] Reference sample decision parameters may be determined based on the intra direction parameters, and may include a reference sample index for specifying the reference sample, and a weighting value parameter for determining a weighting value to be applied to the reference sample.

[0296] The reference sample index iIdx and the weighting parameter ifact may be obtained by the following Equations 9 and 10, respectively.

[0297] equation 9 JPEG2024138541000014.jpg739

[0298] Equation 10 JPEG2024138541000015.jpg743

[0299] In equations 9 and 10, P ang indicates the intra direction parameter. The reference sample specified based on the reference sample index iIdx is an integer pixel (Integer pel). is equivalent to.

[0300] One or more reference samples may be specified to derive the predicted sample. Specifically, the position of the reference sample used to derive the predicted sample may be specified in consideration of the gradient of the prediction mode. For example, the reference sample index iIdx may be used to specify the reference sample used to derive the predicted sample.

[0301] In this case, if the gradient of the intra prediction mode is not represented by one reference sample, a prediction sample may be generated by interpolating multiple reference samples. For example, if the gradient of the intra prediction mode is a value between the gradient between the prediction sample and a first reference sample and the gradient between the prediction sample and a second reference sample, the prediction sample may be obtained by interpolating the first reference sample and the second reference sample. That is, if an angular line according to the intra prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample may be obtained by interpolating adjacent reference samples to the left and right or above and below the position where the angular line passes.

[0302] Equation 11 below shows an example of obtaining a predicted sample based on a reference sample.

[0303] Equation 11 JPEG2024138541000016.jpg7137

[0304] In Equation 11, P denotes a predicted sample, and Ref_1D denotes any one of the reference samples in the one-dimensional array. In such a case, the position of the reference sample may be determined based on the position (x, y) of the predicted sample and the reference sample index iIdx.

[0305] If the gradient of the intra prediction mode may be represented as one reference sample, the weighting value parameter ifact may be set as 0. Thus, Equation 11 may be simplified to the following Equation 12.

[0306] Equation 12 JPEG2024138541000017.jpg552

[0307] Furthermore, intra prediction may be performed on the current block based on multiple intra prediction modes, for example, intra prediction modes may be derived for different prediction samples, and prediction samples may be derived based on the intra prediction modes assigned to each prediction sample.

[0308] Alternatively, intra prediction modes may be derived for different regions, and intra prediction may be performed for each region based on the intra prediction mode assigned to each region. The regions may include at least one sample. At least one of the sizes or shapes of the regions may be adaptively determined based on at least one of the size, shape, or intra prediction mode of the current block. Alternatively, at the encoder and decoder, at least one of the sizes or shapes of the regions may be predefined, regardless of the size or shape of the current block.

[0309] Alternatively, intra prediction may be performed based on multiple intra predictions, and a final prediction sample may be derived by averaging or weighted summing of multiple prediction samples obtained by multiple intra predictions. For example, intra prediction may be performed based on a first intra prediction mode to obtain a first prediction sample, and intra prediction may be performed based on a second intra prediction mode to obtain a second prediction sample. Then, a final prediction sample may be obtained by averaging or weighting summing of the first prediction sample and the second prediction sample. In this case, the weighting values ​​assigned to the first prediction sample and the second prediction sample may be determined by considering at least one of whether the first intra prediction mode is a non-angle / angle prediction mode, whether the second intra prediction mode is a non-angle / angle prediction mode, or the intra prediction mode of a neighboring block.

[0310] The multiple intra prediction modes may be a combination of non-angular intra prediction modes and angular prediction modes, a combination of angular prediction modes, or a combination of non-angular prediction modes.

[0311] FIG. 24 is a diagram showing angular intra prediction modes and angles formed by a line parallel to the x-axis.

[0312] In the example shown in Figure 24, the angular prediction modes may lie between the bottom left diagonal and the top right diagonal. When described as the angle between the x-axis and the angular prediction modes, the angular prediction modes may lie between 45 degrees (bottom left diagonal) and -135 degrees (top right diagonal).

[0313] If the current block is non-square in shape, a situation may occur in which a prediction sample is derived based on the intra prediction mode of the current block using a reference sample that is away from the prediction sample among the reference samples located on a corner line according to the intra prediction angle, rather than a reference sample that is close to the prediction sample.

[0314] FIG. 25 illustrates an embodiment of obtaining predicted samples when the current block is non-square.

[0315] For example, in the example shown in Figure 25(a), it is assumed that the current block is a non-square block whose width is greater than its height, and the intra prediction mode of the current block is an angular intra prediction mode whose angle is 0 degrees to 45 degrees. In this case, when deriving a prediction sample A near the right column of the current block, a situation occurs in which a left reference sample L that is far from the prediction sample among reference samples in the angular mode located at the angle is substituted for an upper reference sample T that is close to the prediction sample.

[0316] As another example, in the example shown in Figure 25(b), assume that the current block is a non-square block with its height greater than its width, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between -90 degrees and -135 degrees. In this case, when deriving a prediction sample A in the vicinity of the lower row of the current block, a situation occurs in which an upper reference sample T away from the prediction sample among reference samples in the angular mode located at the angle is substituted for a left reference sample L close to the prediction sample.

[0317] To solve the above problem, if the current block is non-square, the intra prediction mode of the current block may be replaced with an intra prediction mode of the opposite direction. Thus, for a non-square block, an angular prediction mode having an angle larger or smaller than the angle of the angular prediction mode shown in FIG. 21 may be used. Such an angular intra prediction mode may be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode refers to an angular intra prediction mode that is not located in the range of 45 degrees to -135 degrees.

[0318] FIG. 26 is a diagram showing the wide-angle intra prediction mode.

[0319] In the example shown in FIG. 26, the intra prediction modes of indexes -1 to -14 and the intra prediction modes of indexes 67 to 80 indicate wide-angle intra prediction modes.

[0320] FIG. 26 shows 14 wide-angle intra prediction modes (-1 to -14) having angles greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) having angles less than -135 degrees, however, a greater or lesser number of wide-angle intra prediction modes may be defined.

[0321] When using a wide-angle intra prediction mode, the length of the upper reference sample is set as 2W+1, and the length of the left reference sample is set as 2H+1.

[0322] When using a wide-angle intra prediction mode, reference sample T may be used to predict sample A shown in Figure 25(a), and reference sample L may be used to predict sample A shown in Figure 25(b).

[0323] By adding the existing intra prediction modes and the N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 5 shows the intra direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0324] [Table 5]

[0325] If the current block is non-square and the intra prediction mode of the current block obtained in step S2002 is within a transformation range, the intra prediction mode of the current block may be transformed to a wide-angle intra prediction mode. The transformation range may be determined based on at least one of the size, shape, or ratio of the current block. The ratio may indicate a ratio between the width and height of the current block.

[0326] If the current block is a non-square block with width greater than height, a transformation range may be set to an upper right diagonal intra prediction mode index (e.g., 66) to (upper right diagonal intra prediction mode index -N). N may be determined based on the ratio of the current block. If the intra prediction mode of the current block is within the transformation range, the intra prediction mode may be transformed to a wide-angle intra prediction mode. The transformation may be performed by subtracting a predefined value from the intra prediction mode, and the predefined value may be a total number of intra prediction modes other than the wide-angle intra prediction mode (e.g., 67).

[0327] Based on the above embodiment, the 66th to 53rd intra prediction modes may be converted into the −1st to −14th wide-angle intra prediction modes, respectively.

[0328] If the current block is a non-square block with height greater than width, a transform range may be set to the lower left diagonal intra prediction mode index (e.g., 2) to (lower left diagonal intra prediction mode index + M). M may be determined based on the ratio of the current block. If the intra prediction mode of the current block is within the transform range, the intra prediction mode may be transformed to a wide-angle intra prediction mode. The transform may be performed by adding a predefined value to the intra prediction mode, and the predefined value may be the total number of angular intra prediction modes other than the wide-angle intra prediction mode (e.g., 65).

[0329] Based on the above embodiment, the 2nd to 15th intra prediction modes are converted into the 67th to 80th wide-angle intra prediction modes, respectively.

[0330] Hereinafter, the intra prediction modes that are within the transform range are referred to as wide-angle intra substitution prediction modes.

[0331] The transform range may be determined based on the ratio of the current block. For example, Tables 6 and 7 show transform ranges when 35 intra prediction modes and 67 intra prediction modes other than the wide-angle intra prediction mode are defined, respectively.

[0332] [Table 6]

[0333] [Table 7]

[0334] In the examples shown in Tables 6 and 7, the number of wide-angle intra-permutation prediction modes in the transform range may vary according to the proportion of the current block.

[0335] By subdividing the ratio of the current block, the conversion range can be set as shown in Table 8.

[0336] [Table 8]

[0337] When using a multi-line intra prediction coding method for determining a non-adjacent reference sample line as a reference sample line of a current block or selecting any one of a plurality of reference sample lines, it may be set not to use a wide-angle intra prediction mode. That is, even if the current block is non-square and the intra prediction mode of the current block is within the conversion range, the intra prediction mode of the current block may not be converted to the wide-angle intra prediction mode.

[0338] Alternatively, when the intra prediction mode of the current block is determined as a wide-angle intra prediction mode, the non-adjacent reference sample line may be set not to be used as the reference sample line of the current block, or the multi-line intra prediction coding method for selecting any one of the multiple reference sample lines may be set not to be used. When the multi-line intra prediction coding method is not used, the adjacent reference sample line may be set as the reference sample line of the current block.

[0339] When the wide-angle intra prediction mode is not used, refW and refH are set as the sum of nTbW and nTbH. Thus, in addition to the top left reference sample, non-adjacent reference samples with a distance i from the current block may include (nTbW+nTbH+offsetX[i]) upper reference samples and (nTbW+nTbH+offsetY[i]) left reference samples. That is, non-adjacent reference samples with a distance i from the current block may include (2nTbW+2nTbH+offsetX[i]+offsetY[i]+1) reference samples. For example, if the value of whRatio is greater than 1, the value of offsetX is set to a value greater than offsetY. For example, the value of offsetX is set to 1, and the value of offsetY is set to 0. On the other hand, if the value of whRatio is less than 1, the value of offsetY is set to a value greater than offsetX. For example, set the value of offsetX as 0 and the value of offsetY as 1.

[0340] As wide-angle intra-prediction modes are used in addition to existing intra-prediction modes, the resources required to encode the wide-angle intra-prediction modes may increase, which may reduce coding efficiency. Therefore, instead of directly encoding the wide-angle intra-prediction modes, coding efficiency can be improved by encoding a replacement intra-prediction mode related to the wide-angle intra-prediction mode.

[0341] For example, when the 67th wide-angle intra prediction mode is used to encode the current block, the intra prediction mode of the current block may be coded as the 67th wide-angle replacement intra prediction mode, i.e., number 2. When the current block is encoded using the −1th wide-angle intra prediction mode, the intra prediction mode of the current block may be coded as the −1th wide-angle replacement intra prediction mode, i.e., number 66.

[0342] The decoder may decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode is included in the transform range. If the decoded intra-prediction mode is a wide-angle substitute intra-prediction mode, the decoder may transform the intra-prediction mode to a wide-angle intra-prediction mode.

[0343] Alternatively, if the current block is coded in a wide-angle intra-prediction mode, a wider-angle intra-prediction mode may be coded directly.

[0344] Encoding of the intra prediction mode may be realized based on the MPM list. Specifically, when encoding a neighboring block in a wide-angle intra prediction mode, the MPM may be set based on a wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode.

[0345] When generating a prediction block according to the result of performing intra prediction, the prediction samples may be updated based on the position of each prediction sample included in the prediction block. The above-described update method may be referred to as a sample position-based intra weighted prediction method (or Position Dependent Prediction Combination (PDPC)).

[0346] Whether to use PDPC may be determined by taking into account the intra prediction mode of the current block, the reference sample line of the current block, the size or color components of the current block. For example, PDPC may be used if the intra prediction mode of the current block is at least one of planar mode, DC mode, vertical mode, horizontal mode, mode with index value smaller than vertical, or mode with index value larger than horizontal. Alternatively, PDPC may be used only if at least one of the width and height of the current block is greater than 4. Alternatively, PDPC may be used only if the index of the reference picture line of the current block is 0. Alternatively, PDPC may be used only if the index of the reference picture line of the current block is equal to or greater than a predefined value. Alternatively, PDPC may be used only for the luminance component. Alternatively, whether to use PDPC may be determined depending on whether two or more of the above listed conditions are satisfied.

[0347] As another example, information indicating whether the PDPC applies may be transmitted by a signal via the stream.

[0348] When a prediction sample is obtained by an intra prediction sample, a reference sample for correcting the prediction sample may be determined based on the position of the obtained prediction sample. For convenience of description, in the following embodiments, the reference sample for correcting the prediction sample is called the PDPC reference sample, the prediction sample obtained by intra prediction is called the first prediction sample, and the prediction sample obtained by correcting the first prediction sample is called the second prediction sample.

[0349] FIG. 27 is a diagram showing an embodiment in which the PDPC is applied.

[0350] At least one PDPC reference sample may be used to correct the first predicted sample, which may include at least one of a reference sample adjacent to an upper left corner of the current block, an upper reference sample located above the current block, or a left reference sample located to the left of the current block.

[0351] At least one of the reference samples belonging to the reference sample line of the current block may be set as the PDPC reference sample. Alternatively, regardless of the reference sample line of the current block, at least one of the reference samples belonging to the reference sample line of index 0 may be set as the PDPC reference sample. For example, even if the first predicted sample is obtained using the reference sample included in the reference sample line of index 1 or index 2, the second predicted sample may be obtained using the reference sample included in the reference sample line of index 0.

[0352] The number or positions of PDPC reference samples for correcting the first predicted sample may be determined by taking into account at least one of the intra prediction mode of the current block, the size of the current block, the shape of the current block, or the position of the first predicted sample.

[0353] For example, when the intra prediction mode of the current block is the planar mode or the DC mode, the second predicted sample may be obtained using an upper reference sample and a left reference sample, where the upper reference sample may be a reference sample perpendicular to the first predicted sample (e.g., a reference sample having the same x coordinate), and the left reference sample may be a reference sample parallel to the first predicted sample (e.g., a reference sample having the same y coordinate).

[0354] If the intra prediction mode of the current block is a horizontal intra prediction mode, the second prediction sample may be obtained using an upper reference sample, where the upper reference sample may be a reference sample perpendicular to the first prediction sample.

[0355] If the intra prediction mode of the current block is a vertical intra prediction mode, a second predicted sample may be obtained using a left reference sample, where the left reference sample may be a reference sample parallel to the first predicted sample.

[0356] If the intra prediction mode of the current block is a left-bottom diagonal intra prediction mode or an right-top diagonal intra prediction mode, the second predicted sample may be obtained based on the top-left reference sample, the top reference sample, and the left reference sample. The top-left reference sample may be a reference sample adjacent to the top-left corner of the current block (e.g., a reference sample at a (-1,-1) position). The top reference sample may be a reference sample located diagonally to the top right of the first predicted sample, and the left reference sample may be a reference sample located diagonally to the bottom left of the first predicted sample.

[0357] In short, if the position of the first predicted sample is (x, y), R(-1,-1) may be set as the top-left reference sample, R(x+y+1,-1) or R(x,-1) may be set as the top reference sample, and R(-1,x+y+1) or R(-1,y) may be set as the left reference sample.

[0358] A single prediction mode may be applied multiple times to the current block, or multiple prediction modes may be repeatedly applied to the current block. In this manner, a prediction method using the same or different prediction modes may be referred to as a combined prediction mode (or a multi-hypothesis prediction mode).

[0359] The combined prediction modes may include at least one of a mode that combines a merge mode with a merge mode, a mode that combines inter prediction with intra prediction, a mode that combines a merge mode with an advanced motion vector prediction mode, and a mode that combines a merge mode with intra prediction.

[0360] In the combined prediction mode, a first prediction block may be generated according to a first prediction mode, a second prediction block may be generated according to a second prediction mode, and then a third prediction block may be generated by performing a weighted sum operation of the first prediction block and the second prediction block. The third predicted block may be set as the final predicted block of the current block.

[0361] Whether to use the combined prediction mode may be determined based on the size or shape of the current block. For example, whether to use the combined prediction mode may be determined based on at least one of the size of the coding block, the number of samples included in the coding block, the width of the coding block, and the height of the coding block. For example, if at least one of the width and height of the coding block is 128 or more, or the number of samples included in the coding block is 64 or less, the combined prediction mode may not be applied to the coding block.

[0362] Also, information indicating whether a combined prediction mode has been applied to the current block may be transmitted by a signal via the stream. For example, the information may be a 1-bit flag. For example, a flag mh_intra_flag indicating whether a combined prediction mode combining merge mode and intra prediction is used may be transmitted by a signal via the stream. When mh_intra_flag is 1, it indicates that a combined prediction mode combining merge mode and intra prediction is used, and when mh_intra_flag is 0, it indicates that a combined prediction mode combining merge mode and intra prediction is not used.

[0363] A combined prediction mode that combines the merge mode and intra prediction may be applied to the current block only when the prediction coding mode of the current block is an inter prediction mode and the merge mode is applied to the current block. That is, when the value of the merge_flag flag indicating whether the merge mode is applied to the current block is 1, mh_intra_flag may be transmitted by a signal.

[0364] In a combined prediction mode that combines the merge mode and intra prediction, the intra prediction mode of the current block may be set to a predefined intra prediction mode. For example, when using the combined prediction mode, the intra prediction mode of the current block may be set to a planar mode.

[0365] As another example, when a combined prediction mode that combines merge mode and intra prediction is used, one of the candidate intra prediction modes may be determined as the intra prediction mode of the current block. Here, the candidate intra prediction modes may include at least one of a non-angular intra prediction mode and a specific direction angular intra prediction mode. Here, the non-angular intra prediction mode includes at least one of a DC mode and a planar mode, and the angular intra prediction mode includes at least one of a horizontal intra prediction mode, a vertical intra prediction mode, and a diagonal intra prediction mode. For example, when a combined prediction mode is used, only a DC mode, a planar mode, a horizontal intra prediction mode, or a vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when a combined prediction mode is used, only a planar mode, a horizontal intra prediction mode, or a vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when a combined prediction mode is used, only a DC mode, a planar mode, or a vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when using combined prediction modes, only DC mode, planar mode or horizontal intra prediction mode may be set as the intra prediction mode of the current block.

[0366] Index information for specifying one of the candidate intra prediction modes may be transmitted by a signal via the stream. For example, an index mh_intra_idx for specifying one of the candidate intra prediction modes may be transmitted by a signal via the stream. Tables 9 and 10 show intra prediction modes based on the value of mh_intra_idx. The intra prediction mode indicated by mh_intra_idx may be determined as the intra prediction mode of the current block.

[0367] [Table 9]

[0368] [Table 10]

[0369] In the encoder and decoder, the number of candidate intra-prediction modes may have a fixed value. Alternatively, the number or type of candidate intra-prediction modes varies depending on at least one of the size of the current block, the shape of the current block, or whether to code neighboring blocks by intra-prediction.

[0370] As another example, when a combined prediction mode that combines a merge mode and intra prediction is applied, the intra prediction mode of the current block may be determined based on the MPM. In this case, when general intra prediction is applied, the number of MPMs included in the MPM list may be different from the number of MPMs included in the MPM list when the combined prediction mode is applied. For example, when the combined prediction mode is not applied, the MPM list may include six or five MPMs, but when the combined prediction mode is applied, the MPM list may include four or three MPMs. That is, when general intra prediction is performed, N MPMs are used, but when the combined prediction mode is applied, less than N MPMs may be used.

[0371] For example, when applying a common intra prediction mode, it is assumed that the MPMs derived based on the intra prediction modes of adjacent blocks adjacent to the current block are planar mode, DC mode, INTRA_MODE32, INTRA_MODE31, INTRA_MODE33, and INTRA_MODE30. When applying a combined prediction mode to the current block, only some of the six MPMs may be used to determine the intra prediction mode of the current block. For example, the intra prediction mode of the current block may be determined using an MPM list including three MPMs (i.e., planar mode, DC mode, and INTRA_MODE32) having the smallest indexes among the MPMs. Alternatively, any one of the MPMs having a predetermined index value may be determined as the intra prediction mode of the current block.

[0372] As another example, the number of MPMs for determining the intra prediction mode of the current block when applying general intra prediction may be set to be the same as the number of MPMs for determining the intra prediction mode of the current block when applying a combined prediction mode.

[0373] When a combined prediction mode is applied, encoding of the MPM flag may be omitted, and the MPM flag may be regarded as true. That is, when a combined prediction mode is applied, any one of the MPMs may be determined as the intra prediction mode of the current block.

[0374] When a combined prediction mode that combines merge mode and intra prediction is applied to the current block, a multi-line intra prediction coding method may be set not to be used. When the multi-line intra prediction coding method is not used, a neighboring reference sample line may be used for intra prediction of the current block.

[0375] Alternatively, when a combined prediction mode combining a merge mode and intra prediction is applied to a current block, a multi-line intra prediction coding method may be set to be used. Information for determining a reference sample line of the current block from a plurality of reference sample lines may be transmitted by a signal through a stream. Alternatively, a reference sample line of the current block may be determined by considering at least one of a size of the current block, a shape of the current block, an intra prediction mode of the current block, or a reference sample line of a neighboring block.

[0376] The number of available candidate intra prediction modes may be set according to the difference in the reference sample line of the current block. For example, when adjacent reference sample lines are used, the candidate intra prediction modes may include at least one of a planar mode, a DC mode, a horizontal intra prediction mode, a vertical intra prediction mode, a left lower diagonal intra prediction mode, and an upper right diagonal intra prediction mode. On the other hand, when non-adjacent reference sample lines are used, the candidate intra prediction modes may include at least one of a horizontal intra prediction mode, a vertical intra prediction mode, a left lower diagonal intra prediction mode, and an upper right diagonal intra prediction mode.

[0377] A first prediction block may be generated using a merge candidate selected in the merge mode, and a second prediction block may be generated using an intra prediction mode. In this case, when the first prediction block is generated, it may be set so that triangulation is not applied to the current block. That is, in a combined prediction mode that combines the merge mode and intra prediction, triangulation may not be allowed. Also, when the second prediction block is generated, it may be set so that PDPC is not applied to the current block. That is, in a combined prediction mode that combines the merge mode and intra prediction, PDPC may not be allowed.

[0378] The third predicted block may be generated by a weighted sum operation of the first predicted block and the second predicted block. For example, Equation 13 shows an example of generating the third predicted block by a weighted sum operation of the first predicted block and the second predicted block.

[0379] Equation 13 JPEG2024138541000024.jpg784

[0380] In Equation 13, P merge denotes the first predicted block obtained based on the merge mode, and P intra P denotes the second predicted block obtained by intra prediction. comb teeth 3 shows a third predicted block obtained by combining the first predicted block and the second predicted block. Here, w denotes a first weighting value applied to the first predicted block. A second weighting value applied to the second predicted block may be derived by subtracting the first weighting value w from a constant N, where N may have a value predefined by the encoder and decoder. For example, N may be 4 or 8. Alternatively, the constant N may be derived based on at least one of the size, shape, and intra prediction mode of the current block.

[0381] Unlike the example shown in Equation 13, w may be set as the second weighting value, and a value obtained by subtracting the second weighting value from a predetermined constant N may be set as the first weighting value.

[0382] The weighting values ​​to be applied to the first and second predicted blocks may be determined according to at least one of an intra prediction mode of the current block and whether the merge mode has bidirectional motion information. For example, the first weighting value w when the intra prediction mode of the current block is an angular prediction mode may be greater than the first weighting value w when the intra prediction mode of the current block is a non-angular prediction mode. Alternatively, the first weighting value w when bidirectional prediction is performed using the merge candidate may be greater than the first weighting value w when unidirectional prediction is performed using the merge candidate.

[0383] As another example, the weighting value may be determined in consideration of the predictive coding mode of the neighboring blocks adjacent to the current block. Here, the neighboring blocks may include at least one of an upper neighboring block adjacent to the upper side of the current block, a left neighboring block adjacent to the left side of the current block, and an upper left neighboring block adjacent to the upper left corner of the current block. The weighting value may be determined based on the number of neighboring blocks encoded by intra prediction among the neighboring blocks adjacent to the current block. For example, as the number of neighboring blocks encoded by intra prediction among the neighboring blocks adjacent to the current block increases, the first weighting value may have a relatively small value, and the second weighting value may have a relatively large value. Meanwhile, as the number of neighboring blocks encoded by non-intra prediction among the neighboring blocks adjacent to the current block decreases, the first weighting value may have a relatively large value, and the second weighting value may have a relatively small value. For example, if both the left neighboring block and the upper neighboring block are encoded by intra prediction, the second weighting value may be set as n. If only one of the left neighboring block and the upper neighboring block is encoded by intra prediction, the second weighting value may be set as n / 2. If neither the left neighboring block nor the upper neighboring block is coded by intra prediction, the second weighting value may be set as n / 4, where n is an integer equal to or less than N.

[0384] The weighting value may be determined on a sample or sub-block basis in the current block. For example, the current block may be divided into a plurality of sub-blocks, and the weighting value applied to each sub-block may be set to have a different value. In such a case, the number of sub-blocks may have a fixed value in the encoder and the decoder. Alternatively, the number of sub-blocks may be adaptively determined based on at least one of the size of the current block, the shape of the current block, the intra-prediction mode of the current block, and the predictive coding mode of the neighboring block.

[0385] FIG. 28 illustrates an embodiment for applying weighting values.

[0386] The current block may be divided into a plurality of sub-blocks, and the weighting value applied to each sub-block may be set to a different value. For example, in the example shown in FIG. 28(a) and FIG. 28(b), the second weighting value may be set to be greater than the first weighting value for the sub-block including the sample located at the upper left corner of the current block. Meanwhile, the first weighting value may be set to be greater than the second weighting value for the sub-block including the sample located at the lower right corner of the current block.

[0387] A third predicted sub-block of a sub-block may be obtained by performing weighted prediction on the first predicted sub-block and the second predicted sub-block corresponding to the sub-block. In such a case, in the example shown in FIG. 28, weighting values ​​for weighted prediction may be set according to the differences between each sub-block.

[0388] The weighting value assigned to a sub-block may be determined separately depending on whether the merge candidate has bidirectional motion information.

[0389] FIG. 29 illustrates an example of assigning different weighting values ​​to sub-blocks depending on whether the merge candidate has bidirectional motion information.

[0390] When a merging candidate has bidirectional motion information, the first weighting value w assigned to at least one sub-block may be set to a larger value than when the merging candidate has unidirectional motion information. For example, Figures 29(a) and 29(b) show that the weighting value w applied when bidirectional prediction is applied to all sub-blocks is larger than the weighting value w applied when unidirectional prediction is applied.

[0391] Alternatively, the weighting value applied to the sub-block may be determined based on the intra-prediction mode of the current block.

[0392] FIG. 30 is a diagram illustrating an example of assigning different weighting values ​​to sub-blocks based on the intra-prediction mode of the current block.

[0393] If the intra prediction mode of the current block is vertical, the second weighting value of the sub-block located above the current block may be set to be greater than the second weighting value of the sub-block located below the current block.

[0394] For example, for a sub-block located above the current block, the second weighting value may be set to be greater than the first weighting value, while for a sub-block located below the current block, the first weighting value and the second weighting value may be set to be equal, or the first weighting value may be set to be greater than the second weighting value.

[0395] If the intra prediction mode of the current block is horizontal, the second weighting value of the sub-block located to the left of the current block may be set to be greater than the second weighting value of the sub-block located to the right of the current block.

[0396] For example, for a sub-block located to the left of the current block, the second weighting value may be set to be greater than the first weighting value, while for a sub-block located to the right of the current block, the first weighting value and the second weighting value may be set to be equal, or the first weighting value may be set to be greater than the second weighting value.

[0397] As another example, if the intra prediction mode of the current block is vertical, the first weighting value of the sub-block located above the current block may be set to be greater than the first weighting value of the sub-block located below the current block, or if the intra prediction mode of the block is horizontal, the first weighting value of the sub-block located to the left of the current block may be set to be greater than the first weighting value of the sub-block located to the right of the current block.

[0398] As another example, the weighting value assigned to each sub-block may be determined based on the predictive coding mode of the adjacent block adjacent to the current block. For example, if the upper adjacent block of the current block is coded by intra prediction but the left adjacent block of the current block is not coded by intra prediction, the second weighting value of the sub-block located above the current block may be set to be greater than the second weighting value of the sub-block located below the current block. On the other hand, if the upper adjacent block of the current block is not coded by intra prediction but the left adjacent block of the current block is coded by intra prediction, the second weighting value of the sub-block located to the left of the current block may be set to be greater than the second weighting value of the sub-block located to the right of the current block.

[0399] For at least one of the plurality of subblocks, the first weighting value or the second weighting value may be set as 0. That is, for at least one of the plurality of subblocks, the first prediction subblock or the second prediction subblock may be set as the third prediction subblock. Therefore, it is not necessary to perform combined prediction for at least one of the plurality of subblocks. For example, for at least one of the plurality of subblocks, the first weighting value w may be set as 0, or the first weighting value w may be set as a constant N.

[0400] A block that applies a combined prediction mode that combines merge mode and intra prediction may be considered to be coded by inter prediction. Therefore, the intra prediction mode of a block coded by the combined prediction mode may be set as an intra prediction mode that cannot be used to derive a block to be coded / decoded later. When deriving the MPM, the intra prediction mode of the unavailable block may be considered as a planar mode.

[0401] Meanwhile, the motion information of a block coded in the combined prediction mode may be set as motion information that can be used to derive a block to be coded / decoded in the future.

[0402] In the merge mode and the combined merge mode, multiple merge candidates may be used to perform motion compensated prediction, specifically, a first merge candidate may be used to generate a first predicted block, and a second merge candidate may be used to generate a second predicted block. The third predicted block may be generated by performing a weighted sum operation on the first predicted block and the second predicted block.

[0403] Information for specifying the first merging candidate and the second merging candidate may be transmitted by signals via the streams. For example, index information merge_idx for specifying the first merging candidate and index information merge_2nd_idx for specifying the second merging candidate may be transmitted by signals via the streams. The second merging candidate may be determined based on the index information merge_2nd_idx and the index information merge_idx.

[0404] The index information merge_idx specifies one of the merge candidates included in the merge candidate list.

[0405] The index information merge_2nd_idx may specify any one of the remaining merge candidates other than the merge candidate specified by merge_idx. Thus, if the value of merge_2nd_idx is smaller than the value of merge_idx, the merge candidate with the value of index merge_2nd_idx may be set as the second merge candidate. If the value of merge_2nd_idx is equal to or greater than the value of merge_idx, the merge candidate with the value of index value merge_2nd_idx plus 1 may be set as the second merge candidate.

[0406] Alternatively, the second merging candidate may be specified taking into account the search order of the candidate blocks.

[0407] FIG. 31 is a diagram showing an example of specifying the second merging candidate in consideration of the search order of the candidate blocks.

[0408] In the example shown in Figure 31, the indices marked on the adjacent and non-adjacent samples indicate the search order of the candidate blocks, for example, the candidate blocks may be searched in order from position A0 to position A14.

[0409] If block A4 is selected as the first merging candidate, a merging candidate derived from a candidate block that is after A4 in the search order may be specified as the second merging candidate. For example, a merging candidate derived from A5 may be selected as the second merging candidate. If the candidate block at position A5 cannot be used as a merging candidate, a merging candidate derived from the next candidate block may be selected as the second merging candidate.

[0410] Additionally, the first and second merging candidates may be selected from merging candidates derived from non-adjacent blocks.

[0411] FIG. 32 is a diagram showing an example of selecting a first merging candidate and a second merging candidate from merging candidates derived from non-adjacent blocks.

[0412] In the example shown in Figure 32, merging candidates derived from a first candidate block and a second candidate block that are not adjacent to the current block may be selected as the first merging candidate and the second merging candidate, respectively. In this case, the block row to which the first candidate block belongs and the block row to which the second candidate block belongs may be different. For example, the first merging candidate may be derived from any one of candidate blocks A5 to A10, and the second merging candidate may be derived from any one of candidate blocks A11 to A15.

[0413] Alternatively, the first candidate block and the second candidate block may be set so as not to be included in the same line (eg, row or column).

[0414] As another example, the second merging candidate may be specified based on the first merging candidate. In such a case, the first merging candidate may be specified by index information merge_idx transmitted by a signal from the stream. For example, a merging candidate adjacent to the first merging candidate may be specified as the second merging candidate. Here, a merging candidate adjacent to the first merging candidate may refer to a merging candidate whose index difference with the first merging candidate is 1. For example, a merging candidate with index value merge_idx+1 may be set as the second merging candidate. In such a case, if the value of merge_idx+1 is greater than the maximum index value (or if the index value of the first merging candidate is the maximum index), a merging candidate with index value merge_idx-1 or a merging candidate whose index value is a predefined value (e.g., 0) may be set as the second merging candidate.

[0415] Alternatively, the term "neighboring merging candidates" for the first merging candidate may refer to merging candidates derived from candidate blocks that are spatially adjacent to the candidate block from which the first merging candidate is derived, where the term "neighboring candidate blocks" for a candidate block may refer to neighboring blocks to the left, right, above, below, or diagonally of the candidate block.

[0416] As another example, the second merging candidate may be specified based on motion information of the first merging candidate. For example, a merging candidate having the same reference picture as the first merging candidate may be selected as the second merging candidate. When there are multiple merging candidates that share the same reference picture as the first merging candidate, a merging candidate having the smallest index among the multiple merging candidates, a merging candidate having the smallest index difference with the first merging candidate, or a merging candidate having the smallest motion vector difference with the first merging candidate may be selected as the second merging candidate. Alternatively, the second merging candidate may be selected based on index information specifying any one of the multiple merging candidates.

[0417] Alternatively, when the first merging candidate is unidirectional prediction in the first direction, a merging candidate including motion information in the second direction may be set as the second merging candidate. For example, when the first merging candidate has motion information in the direction L0, a merging candidate having motion information in the direction L1 may be set as the second merging candidate. When there are multiple merging candidates having motion information in the direction L1, a merging candidate having the smallest index among the multiple merging candidates or a merging candidate having the smallest index difference from the first merging candidate is set as the second merging candidate. Alternatively, the second merging candidate may be selected based on index information specifying any one of the multiple merging candidates.

[0418] As another example, one of the merging candidates derived from adjacent blocks adjacent to the current block may be set as the first merging candidate, and one of the merging candidates derived from non-adjacent blocks not adjacent to the current block may be set as the second merging candidate.

[0419] As another example, one of the merging candidates derived from the candidate block located above the current block may be set as the first merging candidate, and one of the merging candidates derived from the candidate block located to the left of the current block may be set as the second merging candidate.

[0420] In the combined prediction mode of combining the above-mentioned merge modes, the merge mode may refer to a merge mode based on a translational motion model (hereinafter referred to as a translational merge mode) or a merge mode based on an affine motion model (hereinafter referred to as an affine merge mode). That is, motion compensated prediction may be performed by combining a translational merge mode with a translational merge mode, or by combining an affine merge mode with an affine merge mode.

[0421] As another example, when encoding adjacent blocks for deriving the first merging candidate based on affine motion information, a merging candidate derived from the adjacent block encoded based on affine motion information may be set as the second merging candidate. For example, when the first merging candidate is an affine merging candidate or the first merging candidate is derived based on motion information of a sub-block in an encoding block encoded based on affine motion information, the affine merging candidate or the merging candidate derived based on motion information of a sub-block in an encoding block encoded based on affine motion information may be selected as the second merging candidate. Here, the second merging candidate may include at least one of the merging candidate closest to the first merging candidate according to the search order, the merging candidate with the smallest index difference with the first merging candidate, the merging candidate with the smallest index, and the merging candidate with the smallest motion vector difference with the first merging candidate among the merging candidates satisfying the above conditions.

[0422] Contrary to the above example, when encoding adjacent blocks for deriving the first merging candidate based on translational motion information (i.e., non-affine motion information), a merging candidate derived from the adjacent blocks encoded based on the translational motion information may be set as the second merging candidate. For example, when the first merging candidate is a non-affine merging candidate, the non-affine merging candidate may be selected as the second merging candidate. Here, the second merging candidate may include at least one of the following among the non-affine merging candidates: a merging candidate closest to the first merging candidate according to the search order, a merging candidate having the smallest index difference with the first merging candidate, a merging candidate having the smallest index, and a merging candidate having the smallest motion vector difference with the first merging candidate.

[0423] Alternatively, the second merging candidate may be derived by adding an offset amount to or subtracting an offset amount from the index of the first merging candidate, and if the encoding methods of the adjacent blocks for deriving the first merging candidate and the second merging candidate are different, the second merging candidate may be re-set as another merging candidate.

[0424] FIG. 33 is a diagram illustrating an example of determining the second merging candidate.

[0425] If the first merging candidate is selected, a merging candidate having an index derived by adding or subtracting an offset amount to or from the index of the first merging candidate may be selected as the second merging candidate. For example, a merging candidate having an index indicated by merge_idx signaled from the stream may be selected as the first merging candidate, and a merging candidate having an index of merge_idx+1 may be selected as the second merging candidate.

[0426] In this case, if the first merge candidate is an affine merge candidate or is derived from a sub-block belonging to a coding block coded by an affine motion model, the affine merge candidate or the merge candidate derived from a sub-block belonging to a coding block coded by an affine motion model should be set as the second merge candidate. If the merge candidate having merge_idx+1 as an index is not an affine merge candidate or is derived from a sub-block belonging to a coding block coded by an affine motion model, another merge candidate may be set as the second merge candidate. Here, the other merge candidate may be the merge candidate with the smallest difference from merge_idx+1 among the first merge candidate, the affine merge candidate, or the merge candidate derived from a sub-block belonging to a coding block coded by an affine motion model.

[0427] On the other hand, if the first merging candidate is a non-affine merging candidate, the non-affine merging candidate should be set as the second merging candidate. If the merging candidate having the index merge_idx+1 is an affine merging candidate or is derived from a sub-block belonging to a coding block coded by an affine motion model, another merging candidate may be set as the second merging candidate. Here, the other merging candidate may be a merging candidate having the smallest difference from merge_idx+1 among the first merging candidate or the non-affine merging candidate. For example, the example shown in FIG. 33 shows that a merging candidate A3 with index merge_idx+2 is set as the second merging candidate.

[0428] As another example, the translational merge mode and the affine merge mode may be combined to perform motion compensated prediction, i.e., either the first merge candidate or the second merge candidate may be an affine merge candidate, and the other may be a non-affine merge candidate.

[0429] A combined prediction block may be obtained by performing a weighted sum operation on a first prediction block derived from the first merging candidate and a second prediction block derived from the second merging candidate. In this case, a weighting value applied to the first prediction block may be set to be greater than a weighting value applied to the second prediction block.

[0430] Alternatively, the weighting value may be determined based on the motion information of the first merging candidate and the motion information of the second merging candidate. For example, the weighting value to be applied to the first prediction block and the second prediction block may be determined according to the difference in the output order of the reference picture and the current picture. Specifically, the weighting value to be applied to the prediction block may be set to be smaller as the difference in the output order of the reference picture and the current picture becomes larger.

[0431] Alternatively, the weighting values ​​to be applied to the first prediction block and the second prediction block may be determined taking into consideration the size or shape of a candidate block for deriving a first merging candidate (hereinafter referred to as a first candidate block) and a candidate block for deriving a second merging candidate (hereinafter referred to as a second candidate block). For example, the weighting value to be applied to a prediction block derived from a candidate block of the first candidate block or the second candidate block, the shape of which is similar to that of the current block, may be set to a relatively large value. On the other hand, the weighting value to be applied to a prediction block derived from a candidate block of the first candidate block or the second candidate block, the shape of which is not similar to that of the current block, may be set to a relatively small value.

[0432] FIG. 34 is a diagram showing an example of determining the weighting value to be applied to a prediction block based on the shape of a candidate block.

[0433] We now assume that the block is non-square, with the width being greater than the height.

[0434] A first prediction block and a second prediction block may be derived based on the first merging candidate and the second merging candidate, and a combined prediction block may be generated by performing a weighted sum operation on the first prediction block and the second prediction block. In this case, weighting values ​​to be applied to the first prediction block and the second prediction block may be determined based on the shapes of the first candidate block and the second candidate block.

[0435] For example, in the example shown in FIG. 34, the first candidate block is a square and the second candidate block is a non-square with the width being greater than the height. Since the shape of the second candidate block is the same as the current block, the weighting value applied to the second predicted block may be set to be greater than the weighting value applied to the first predicted block. For example, a weighting value of 5 / 8 may be applied to the second predicted block, and a weighting value of 3 / 8 may be applied to the first predicted block. Equation 14 shows an example of deriving a combined predicted block by performing a weighted sum operation on the first predicted block and the second predicted block.

[0436] Equation 14 JPEG2024138541000025.jpg771

[0437] P(x,y) denotes a combined predicted block, P1(x,y) denotes the first predicted block, and P2(x,y) denotes the second predicted block.

[0438] As another example, the weighting values ​​applied to the first and second predicted blocks may be determined based on the shape of the current block. For example, if the current block is a non-square shape in which the width is greater than the height, a larger weighting value may be applied to a predicted block generated based on a merging candidate derived from a candidate block located above the current block, among the first and second merging candidates. If both the first and second merging candidates are derived from candidate blocks located above the current block, the weighting values ​​applied to the first and second predicted blocks may be set to be the same. Meanwhile, if the current block is non-square in which the height is greater than the width, a larger weighting value may be applied to a predicted block generated based on a merging candidate derived from a candidate block located to the left of the current block, of the first and second merging candidates. If the first and second merging candidates are both derived from candidate blocks located to the left of the current block, the weighting values ​​applied to the first and second predicted blocks may be set to be the same. If the current block is square, the weighting values ​​applied to the first and second predicted blocks may be set to be the same.

[0439] As another example, a weighting value to be applied to each prediction block may be determined based on a distance between the current block and the candidate block. Here, the distance may be derived based on an x-axis coordinate difference, a y-axis coordinate difference, or the minimum value thereof from the current block. A weighting value to be applied to a prediction block derived from a merging candidate having a relatively small distance from the current block may be set to be greater than a weighting value to be applied to a prediction block derived from a merging candidate having a relatively large distance from the current block. For example, in the example shown in FIG. 31, a first merging candidate is derived from an adjacent block adjacent to the current block, and a second merging candidate is derived from a non-adjacent block not adjacent to the current block. In this case, since the x-axis distance between the first candidate block and the current block is smaller than the x-axis distance between the second candidate block and the current block, a weighting value to be applied to the first prediction block may be set to be greater than a weighting value to be applied to the second prediction block.

[0440] Alternatively, when both the first and second merging candidates are derived from non-adjacent blocks, a larger weighting value may be assigned to a predicted block derived from a non-adjacent block that is closer to the current block than the first and second merging candidates. For example, in the example shown in FIG 32, since the y-axis distance between the first candidate block and the current block is smaller than the y-axis distance between the second candidate block and the current block, the weighting value applied to the first predicted block may be set to be larger than the weighting value applied to the second predicted block.

[0441] Based on the first and second merging candidates, matching motion information may be derived, and motion compensated prediction may be performed on the current block based on the matching motion information. For example, a motion vector of the current block may be derived by averaging or weighted summing the motion vectors of the first and second merging candidates. In this case, the weighting value of the motion vector applied to the first merging candidate and the weighting value of the motion vector applied to the second merging candidate may be determined according to the above embodiment.

[0442] If the first merge candidate is a non-affine merge candidate and the second affine merge candidate is an affine merge candidate, the motion vector of the current block may be derived by scaling the motion vector of the second merge candidate. Equation 15 shows an example of deriving the motion vector of the current block.

[0443] Equation 15 JPEG2024138541000026.jpg591

[0444] In Equation 15, (mvX, mvY) denotes the motion vector of the current block, (mv0x, mv0y) denotes the motion vector of the first merging candidate, and (mv1x, mv1y) denotes the motion vector of the second merging candidate. M denotes a scaling parameter. M may be predefined in the encoder and the decoder. Alternatively, the value of the scaling parameter M may be determined based on the size of the current block or the candidate block. For example, if the width or height of the second candidate block is greater than 32, M may be set as 3, otherwise M may be set as 2.

[0445] The motion information of the first merge candidate or the second merge candidate may be stored as motion information of a block to which a merge prediction mode that combines the merge mode is applied, and the stored motion information may be used to derive motion information of a block to be encoded / decoded in the future.

[0446] Alternatively, the block may be divided into a plurality of sub-blocks, and one of the motion information of the first merging candidate and the motion information of the second merging candidate may be stored as the motion information of each sub-block. In this case, the motion information of some of the sub-blocks may be set as the motion information of the first merging candidate, and the motion information of the other sub-blocks may be set as the motion information of the second merging candidate.

[0447] Alternatively, the matched motion information derived based on the motion information of the first merge candidate and the second merge candidate may be stored as motion information of a block to which a merge prediction mode that combines the merge modes is applied.

[0448] In a prediction mode that combines a merge mode and an advanced motion vector prediction mode, a first prediction block may be generated using motion information derived from a merge candidate, and a second prediction block may be generated using a motion vector derived from a motion vector prediction candidate.

[0449] In the advanced motion vector prediction mode, a motion vector prediction candidate may be derived from a neighboring block adjacent to the current block or a co-located block in a co-located picture. Then, any one of a plurality of motion vector prediction candidates may be designated, and the designated motion vector prediction candidate may be set as a motion vector prediction result of the current block. Then, the motion vector of the current block may be derived by adding the motion vector prediction result of the current block and the motion vector difference.

[0450] In a prediction mode that combines a merge mode and an advanced motion vector prediction mode, a merge candidate and a motion vector prediction candidate may be derived from the same candidate block. For example, when a merge candidate is specified by merge_idx, a motion vector of a candidate block for deriving a specified merge candidate may be set as a motion vector prediction result. Alternatively, when a motion vector prediction candidate is specified by mvp_flag, a merge candidate derived from a candidate block for deriving a specified merge candidate may be selected.

[0451] Alternatively, the candidate block for deriving the merging candidate and the candidate block for deriving the motion vector prediction candidate may be different. For example, when a merging candidate derived from a candidate block located above the current block is selected, a motion vector prediction candidate derived from a candidate block located to the left of the current block may be selected.

[0452] Alternatively, when a merge candidate selected by index information and a motion vector prediction candidate selected by index information are derived from the same candidate block, the motion vector prediction candidate may be replaced with a motion vector prediction candidate derived from an adjacent candidate block adjacent to the candidate block, or the merge candidate may be replaced with a merge candidate derived from a candidate block adjacent to the candidate block.

[0453] FIG. 35 illustrates an example of replacing merging candidates.

[0454] The example shown in Figure 35 (a) shows the selection of a merge candidate and a motion vector prediction candidate derived from a candidate block at position A2. As shown in the figure, when a merge candidate and a motion vector prediction candidate are derived from the same candidate block, a merge candidate or a motion vector prediction candidate derived from a candidate block adjacent to the candidate block may be used instead of the merge candidate or the motion vector prediction candidate. For example, as shown in Figure 35 (b), a merge candidate at position A1 may be used instead of a merge candidate at position A2.

[0455] A first predicted block may be derived based on a merge candidate of the current block, and a second predicted block may be derived based on the motion vector prediction candidate. Then, a combined predicted block may be derived by performing a weighted sum operation on the first predicted block and the second predicted block. In this case, a weighting value applied to the second predicted block generated in the advanced motion vector prediction mode may be set to be larger than a weighting value applied to the first predicted block generated in the merge mode.

[0456] The derived residual picture may be derived by subtracting the original picture from the predicted picture. In this case, when the residual picture is transformed into the frequency domain, the high frequency components in the frequency domain can be removed without significantly degrading the subjective quality of the video. Thus, by reducing the value of the high frequency components or setting the value of the high frequency components to 0, it is possible to improve the compression efficiency without causing noticeable visual distortion. To reflect the above characteristics, the current block may be transformed to decompose the residual picture into two-dimensional frequency components. The transformation may be performed by a transformation technique such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST).

[0457] While the DCT uses a cosine transform to decompose (or transform) the residual picture into two-dimensional frequency components, the DST uses a sine transform to decompose (or transform) the residual picture into two-dimensional frequency components. As a result of the transformation of the residual picture, the frequency components may be denoted as base pictures. For example, when performing a DCT transform on a block of size N×N, N 2 basic pattern components can be obtained. The size of each basic pattern component contained in a block of size N×N can be obtained by transformation. Depending on the transformation technique used, the size of the basic pattern component may be referred to as a DCT coefficient or a DST coefficient.

[0458] The DCT transform technique is mainly used for transforming pictures with a relatively large distribution of non-zero low-frequency components, while the DST transform technique is mainly used for pictures with a relatively large distribution of high-frequency components.

[0459] Furthermore, the residual picture may be transformed using transform techniques other than the DCT or DST.

[0460] Hereinafter, the process of transforming the residual picture into two-dimensional frequency components is referred to as two-dimensional picture transform. And the size of the basic pattern component obtained by the transform result is referred to as transform coefficient. For example, the transform coefficient may refer to a DCT coefficient or a DST coefficient. When the main transform and the secondary transform described below are applied simultaneously, the transform coefficient may indicate the size of the basic pattern component generated by the secondary transform result.

[0461] A transform technique may be determined on a block-by-block basis. The transform technique may be determined based on at least one of a predictive coding mode of the current block, a size of the current block, or a shape of the current block. For example, if the current block is coded in an intra prediction mode and the size of the current block is smaller than N×N, the transform technique DST may be used to perform the transform. On the other hand, if the above conditions cannot be satisfied, the transform technique DCT may be used to perform the transform.

[0462] In the residual picture, the 2D picture transform may not be performed on some blocks. Not performing the 2D picture transform may be referred to as a transform skip. When applying the transform skip, quantization may be applied to the residual values ​​that are not transformed.

[0463] After transforming the current block using a DCT or DST, the transformed current block may be transformed again. In this case, the transformation based on the DCT or DST may be defined as a primary transformation, and the process of transforming the block to which the primary transformation is applied again may be referred to as a secondary transformation.

[0464] The main transform may be performed using any one of a number of possible transform cores, for example, any one of DCT2, DCT8 or DCT7 may be used to perform the main transform.

[0465] Different transform cores may be used for the horizontal and vertical directions, and further, a signal may be sent over the stream indicating the combination of horizontal and vertical transform cores.

[0466] The execution units of the primary transform and the secondary transform are different. For example, the primary transform may be performed on an 8×8 block, and the secondary transform may be performed on a sub-block of size 4×4 in the transformed 8×8 block. In this case, the transform coefficients of the remaining area that is not further subjected to the secondary transform may be set to 0.

[0467] Alternatively, a primary transform may be performed on the 4x4 block, and a secondary transform may be performed on a region of size 8x8 that contains the transformed 4x4 block.

[0468] A signal may be sent over the stream indicating whether or not to perform a secondary transformation.

[0469] Alternatively, whether to perform a secondary transform may depend on whether the horizontal transform core and the vertical transform core are the same. For example, the secondary transform may be performed only if the horizontal transform core and the vertical transform core are the same. Alternatively, the secondary transform may be performed only if the horizontal transform core and the vertical transform core are different.

[0470] Alternatively, a secondary transform is allowed only if the horizontal transform and the vertical transform utilize a predefined transform core, for example, a secondary transform may be allowed if the horizontal transform and the vertical transform use a DCT2 transform core.

[0471] Alternatively, the determination of whether to perform the secondary transform may be based on the number of non-zero transform coefficients of the current block. For example, the secondary transform may be set not to be used if the non-zero transform coefficients of the current block are less than or equal to a threshold, and the secondary transform may be set to be used if the non-zero transform coefficients of the current block are greater than the threshold. Furthermore, the secondary transform may be set to be used only if the current block is coded by intra prediction.

[0472] Based on the shape of the current block, the size or shape of the sub-block on which the secondary transformation is to be performed may be determined.

[0473] 36 and 37 are diagrams showing sub-blocks for performing secondary transformation.

[0474] If the current block is a square, then after performing the primary transform, a secondary transform may be performed on the top-left sub-block of the current block with a size of N×N. For example, if the current block is an 8×8 sized coding block, then after performing the primary transform on the current block, a secondary transform may be performed on the top-left sub-block of the current block with a size of 4×4 (see FIG. 36).

[0475] If the current block is non-square, with the width being four or more times greater than the height, then after performing the primary transform, a secondary transform may be performed on a sub-block of size (kN)×(4kN) at the top left corner of the current block. For example, if the current block is non-square and has a size of 16×4, then a secondary transform may be performed on a sub-block of size 2×8 at the top left corner of the current block (see FIG. 37(a)).

[0476] If the current block is non-square, with the height being more than four times greater than the width, then after performing the primary transform, a secondary transform may be performed on the top left sub-block of the current block with size (4kN)×(kN). For example, if the current block is non-square with size 16×4, then a secondary transform may be performed on the top left sub-block of the current block with size 2×8 (see FIG. 37(b)).

[0477] In the decoder, an inverse transform (second inverse transform) of the secondary transform may be performed, and an inverse transform (first inverse transform) of the primary transform may be performed on the result. As a result of performing the second inverse transform and the first inverse transform, a residual signal of the current block may be obtained.

[0478] Information indicating a transform type of the current block may be transmitted by a signal via the stream, and the information may be index information tu_mts_idx indicating one of a combination of a horizontal transform type and a vertical transform type.

[0479] The vertical transform core and the horizontal transform core may be determined by the transform type candidates designated based on the index information tu_mts_idx. Tables 11 and 12 show the transform type combinations based on tu_mts_idx.

[0480] [Table 11]

[0481] [Table 12]

[0482] The transform type may be determined as any one of DCT2, DST7, DCT8 or skip transform. Alternatively, other than the transform skip, only the transform core may be utilized to configure the transform type combination candidates.

[0483] When using Table 11, if tu_mts_idx is 0, transform skip may be applied in the horizontal and vertical directions. If tu_mts_idx is 1, DCT2 may be applied in the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 may be applied in the horizontal direction and DCT7 may be applied in the vertical direction.

[0484] When using Table 12, if tu_mts_idx is 0, DCT2 may be applied in the horizontal and vertical directions. If tu_mts_idx is 1, transform skip may be applied in the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 may be applied in the horizontal direction and DCT7 may be applied in the vertical direction.

[0485] It may be determined whether to code the index information based on at least one of the size, shape, or number of non-zero coefficients of the current block. For example, if the number of non-zero coefficients is equal to or less than a threshold, the index information may not be transmitted by the signal, and a default transform type may be applied to the current block. The default transform type may be DST7. Alternatively, the default mode depends on the size, shape, or intra-prediction mode of the current block.

[0486] The threshold may be determined based on the size or shape of the current block. For example, if the size of the current block is equal to or smaller than 32×32, the threshold may be set as 2, and if the current block is larger than 32×32 (e.g., if the current block is a coding block of size 32×64 or 64×32), the threshold may be set as 4.

[0487] The plurality of lookup tables may be pre-stored in the encoder / decoder. At least one of index values ​​assigned to the transform type combination candidates, types of the transform type combination candidates, or the number of the transform type combination candidates may be different in the plurality of lookup tables.

[0488] The lookup table for the current block may be selected based on at least one of the size, shape, predictive coding mode, intra prediction mode, whether to apply a second transform, or whether to apply a transform skip to the neighboring block of the current block.

[0489] For example, if the size of the current block is 4×4 or less or the current block is encoded by inter prediction, the lookup table in Table 11 may be used, and if the size of the current block is greater than 4×4 or the current block is encoded by intra prediction, the lookup table in Table 12 may be used.

[0490] Alternatively, information indicating one of a number of look-up tables may be transmitted via a signal over the stream, and the decoder may select the look-up table for the current block based on the information.

[0491] As another example, an index assigned to a transform type combination candidate may be adaptively determined based on at least one of the size, shape, predictive coding mode, intra prediction mode, whether or not a second transform is performed, or whether or not a transform skip is applied to a neighboring block of the current block. For example, if the size of the current block is 4×4, an index assigned to the transform skip may have a value smaller than an index assigned to the transform skip if the size of the current block is larger than 4×4. Specifically, if the size of the current block is 4×4, an index 0 may be assigned to the transform skip, and if the current block is larger than 4×4 and is equal to or smaller than 16×16, an index larger than 0 (e.g., index 1) may be assigned to the transform skip. If the current block is larger than 16×16, a maximum value (e.g., 5) may be assigned as the index of the transform skip.

[0492] Alternatively, if the current block is coded using inter prediction, the transform skip may be assigned an index of 0. If the current block is coded using intra prediction, the transform skip may be assigned an index greater than 0 (e.g., index 1).

[0493] Alternatively, if the current block is a block of size 4×4 that is coded using inter prediction, the transform skip may be assigned an index of 0. On the other hand, if the current block is not coded using inter prediction or is larger than 4×4, the transform skip may be assigned an index of a value larger than 0 (e.g., index 1).

[0494] A transform type combination candidate different from the transform type combination candidates listed in Tables 11 and 12 may be defined and used. For example, a transform type combination candidate applied to a transform core such as DCT7, DCT8, or DST2 may be used to apply a transform skip to a horizontal or vertical transform and other transforms. In this case, it may be determined whether to use a transform skip as a horizontal or vertical transform type candidate based on at least one of a size (e.g., width and / or height), a shape, a predictive coding mode, or an intra-prediction mode of a current block.

[0495] Alternatively, information indicating whether the specified transform type candidate is available may be transmitted by a signal through the stream, for example, a flag indicating whether transform skip can be used as a transform type candidate for the horizontal direction and the vertical direction may be transmitted by a signal, and it may be determined whether the specified transform type combination candidate is included in the multiple transform type combination candidates based on the flag.

[0496] Alternatively, information indicating whether to apply a transform type candidate to the current block may be transmitted by a signal via the stream. For example, a flag cu_mts_flag indicating whether to apply DCT2 to the horizontal and vertical directions may be transmitted by a signal. If the value of cu_mts_flag is 1, DCT2 may be set as the transform core for the vertical and horizontal directions. If the value of cu_mts_flag is 0, DCT8 or DST7 may be set as the transform core for the vertical and horizontal directions. Alternatively, if the value of cu_mts_flag is 0, information tu_mts_idx specifying any one of a plurality of transform type combination candidates may be transmitted by a signal.

[0497] If the current block is a non-square block with width greater than height or height greater than width, further encoding of cu_mts_flag may be omitted and the value of cu_mts_flag may be considered as 0.

[0498] The number of available transform type combination candidates may be set differently based on the size, shape, or intra-prediction mode of the current block. For example, if the current block is square, three or more transform type combination candidates may be used, and if the current block is non-square, two transform type combination candidates may be used. Alternatively, if the current block is square, only transform type combination candidates with different horizontal and vertical transform types among the transform type combination candidates may be used.

[0499] If there are three or more transform type combination candidates available for the current block, index information tu_mts_idx indicating one of the transform type combination candidates may be transmitted by a signal. On the other hand, if there are two transform type combination candidates available for the current block, a flag mts_flag indicating one of the transform type combination candidates may be transmitted by a signal. Table 13 below shows a process of encoding information specifying a transform type combination candidate based on the shape of the current block.

[0500] [Table 13]

[0501] 5. Reorder (or reorder) the indices of the candidate transform type combinations based on the shape of the current block. For example, the indices assigned to the candidate transform type combinations when the current block is square may be different from the indices assigned to the candidate transform type combinations when the current block is non-square. For example, when the current block is square, the transform type combinations may be selected based on Table 14 below, and when the current block is non-square, the transform type combinations may be selected based on Table 15 below.

[0502] [Table 14]

[0503] [Table 15]

[0504] The transform type may be determined based on the number of horizontal non-zero coefficients or the number of vertical non-zero coefficients of the current block. The number of horizontal non-zero coefficients indicates the number of non-zero coefficients contained in 1×N (N is the width of the current block), and the number of vertical non-zero coefficients indicates the number of non-zero coefficients contained in N×1 (N is the height of the current block). If the maximum value of the horizontal non-zero coefficients is less than or equal to a threshold, a primary transform type may be applied in the horizontal direction, and if the maximum value of the horizontal non-zero coefficients is greater than the threshold, a secondary transform type may be applied in the horizontal direction. If the maximum value of the vertical non-zero coefficients is less than or equal to a threshold, a primary transform type may be applied in the vertical direction, and if the maximum value of the vertical non-zero coefficients is greater than the threshold, a secondary transform type may be applied in the vertical direction.

[0505] FIG. 38 is a diagram for explaining an example of determining the transformation type of the current block.

[0506] For example, if the current block is coded by intra prediction and the maximum value of the horizontal non-zero coefficients of the current block is less than or equal to 2 (see FIG. 38(a)), the horizontal transform type may be determined as DST7.

[0507] If the current block is coded by intra prediction and the maximum value of the vertical non-zero coefficients of the current block is greater than 2 (see FIG. 38(b)), the vertical transform type may be determined as DCT2 or DCT8.

[0508] A signal may be sent over the stream indicating whether to explicitly determine the transformation type for the current block based on information signaled from the stream. For example, at the sequence level, a signal may transmit information sps_explicit_intra_mts_flag indicating whether a block coded by intra prediction allows explicit transform type decision, and / or information sps_explicit_inter_mts_flag indicating whether a block coded by inter prediction allows explicit transform type decision.

[0509] If explicit transform type determination is allowed, the transform type of the current block may be determined based on index information tu_mts_idx signaled from the stream. On the other hand, if explicit transform type determination is not allowed, the transform type may be determined based on at least one of the size of the current block, the shape of the current block, whether sub-block unit transform is allowed, and the position of the sub-block containing a non-zero transform coefficient. For example, the horizontal transform type of the current block may be determined based on the width of the current block, and the vertical transform type of the current block may be determined based on the height of the current block. For example, if the width of the current block is smaller than 4 or larger than 16, the horizontal transform type may be determined as DCT2. Otherwise, the horizontal transform type may be determined as DST7. If the height of the current block is smaller than 4 or larger than 16, the vertical transform type may be determined as DCT2. Otherwise, the vertical transform type may be determined as DST7. Here, to determine the horizontal transform type and the vertical transform type, thresholds to be compared with the width and height may be determined based on at least one of the size, shape, and intra prediction mode of the current block.

[0510] Alternatively, if the current block is a square with the same height and width, the horizontal transform type and the vertical transform type may be set to be the same, but if the current block is a non-square with different heights and widths, the horizontal transform type and the vertical transform type may be set to be different. For example, if the width of the current block is greater than the height, the horizontal transform type may be determined as DST7 and the vertical transform type may be determined as DCT2. If the height of the current block is greater than the width, the vertical transform type may be determined as DST7 and the horizontal transform type may be determined as DCT2.

[0511] Depending on whether explicit transform type determination is allowed, the number and / or type of transform type candidates, or the number and / or type of transform type combination candidates, may be different. For example, if explicit transform type determination is allowed, DCT2, DST7, and DCT8 may be used as transform type candidates. Thus, each of the horizontal transform type and the vertical transform type may be set as DCT2, DST8, or DCT8. If explicit transform type determination is not allowed, DCT2 and DST7 are used only as transform type candidates. Thus, each of the horizontal transform type and the vertical transform type may be determined as DCT2 or DST7.

[0512] If the transform and quantization are performed in the encoder, the decoder can obtain the residual block by inverse quantization and inverse transform. The decoder can add the prediction block and the residual block to obtain a reconstructed block of the current block.

[0513] When a reconstructed block of a current block is obtained, information loss occurring during quantization and encoding processes can be reduced by in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter is referred to as a first reconstructed block, and the reconstructed block after applying the in-loop filter is referred to as a second reconstructed block.

[0514] At least one of a deblocking filter, SAO, or ALF may be applied to the first reconstructed block to obtain a second reconstructed block. In such a case, the SAO or ALF may be applied after applying the deblocking filter.

[0515] The deblocking filter is used to reduce image degradation (blocking artifacts) that occurs at block boundaries when quantization is performed on a block-by-block basis. To apply the deblocking filter, a blocking strength (BS) between a first reconstructed block and an adjacent reconstructed block may be determined.

[0516] FIG. 39 is a flow chart showing the process of determining the blocking strength.

[0517] In the example shown in Figure 39, P denotes the first reconstructed block and Q denotes the adjacent reconstructed block, which may be adjacent to the left or above the current block.

[0518] The example shown in Figure 39 illustrates determining the blocking strength taking into account the predictive coding modes of P and Q, whether they contain non-zero transform coefficients, whether inter prediction is performed using the same reference picture, and whether the difference in motion vectors is greater than or equal to a threshold.

[0519] Based on the blocking strength, it may be determined whether to apply a deblocking filter. For example, if the blocking strength is 0, no filtering may be performed.

[0520] SAO is used to reduce ringing artifacts that occur when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset amount determined in consideration of the pattern of the first reconstructed picture. Methods for determining the offset amount include edge offset (EO) or band offset. EO indicates a method of determining an offset of a current sample based on the pattern of surrounding pixels. BO indicates a method of applying the same offset to a set of pixels having similar luminance values ​​in a region. Specifically, pixel luminance is divided into 32 equal intervals, and pixels having similar luminance values ​​are set as one set. For example, four adjacent bands out of the 32 bands may be set as one set, and the same offset amount may be applied to samples belonging to the four bands.

[0521] ALF is a method of applying a filter of a predefined size / shape to a first reconstructed picture or a reconstructed picture to which a deblocking filter is applied to generate a second reconstructed picture. Equation 16 below shows an example of applying ALF.

[0522] Equation 16 JPEG2024138541000032.jpg18150

[0523] For each picture, coding tree unit, coding block, prediction block, or transform block, one of predefined filter candidates may be selected, and each filter candidate may have a different size or shape.

[0524] FIG. 40 is a diagram showing predefined filter candidates.

[0525] In the example shown in FIG. 40, at least one of a 5×5, 7×7 and 9×9 diamond may be selected.

[0526] Only diamonds of size 5x5 can be used for the chromaticity components.

[0527] It is within the scope of the present invention to use the embodiments described with reference to a decoding process or an encoding process for an encoding process or a decoding process. It is also within the scope of the present invention to modify the embodiments described according to a predetermined order in an order different from that in which they are described.

[0528] Although the embodiments are described based on a series of steps or flow charts, the time sequence order of the invention is not limited, and the invention may be performed simultaneously or according to another order as necessary. In addition, in the above embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagrams may further be realized as hardware devices or software, and multiple structural elements may be implemented in combination as a single hardware device or software. The embodiments may be implemented in the form of program instructions, which may be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., independently or in combination. Examples of computer-readable recording media may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, etc. The hardware devices may be configured to operate as one or more software modules to perform the processes according to the present invention, and vice versa. [Industrial Applicability]

[0529] The present invention may be applied to electronic devices that encode / decode video.

Claims

1. 1. A video decoding method comprising: determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained, the first predicted block is obtained based on motion information of a merge candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; obtaining a third predicted block of the current block by performing a weighted sum operation on the first predicted block and the second predicted block; determining whether to apply the combined prediction mode to the current block includes not applying the combined prediction mode to the current block when at least one of a width and a height of the current block is greater than a threshold; The second prediction block is obtained based on a reference sample line included in an adjacent reference sample line. Video decoding methods.

2. 2. The video decoding method of claim 1, wherein when performing the weighted sum operation, weighting values ​​to be applied to the first predicted block and the second predicted block are determined based on predictive coding modes of adjacent blocks adjacent to the current block.

3. 1. A video encoding method comprising: determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained, the first predicted block is obtained based on motion information of a merge candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; obtaining a third predicted block of the current block by performing a weighted sum operation on the first predicted block and the second predicted block; determining whether to apply the combined prediction mode to the current block includes not applying the combined prediction mode to the current block when at least one of a width and a height of the current block is greater than a threshold; The second prediction block is obtained based on a reference sample line included in an adjacent reference sample line. Video coding methods.

4. 4. The video encoding method of claim 3, wherein when performing the weighted sum operation, weighting values ​​to be applied to the first predicted block and the second predicted block are determined based on predictive coding modes of adjacent blocks adjacent to the current block.

5. 1. A video decoding apparatus comprising: a prediction unit; The prediction unit determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block; and obtaining a third predicted block of the current block by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; determining whether to apply the combined prediction mode to the current block includes not applying the combined prediction mode to the current block when at least one of a width and a height of the current block is greater than a threshold; The second prediction block is obtained based on a reference sample line included in an adjacent reference sample line.

6. 1. A video decoding apparatus comprising: a prediction unit; The prediction unit determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block; and obtaining a third predicted block of the current block by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; determining whether to apply the combined prediction mode to the current block includes not applying the combined prediction mode to the current block when at least one of a width and a height of the current block is greater than a threshold; The second prediction block is obtained based on a reference sample line included in an adjacent reference sample line.

7. 1. A video decoder comprising: a processor; and a memory configured to store a computer program executable by said processor; The processor: determining whether to apply a combined prediction mode to the current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained; a third predicted block of the current block is obtained by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; determining whether to apply the combined prediction mode to the current block includes not applying the combined prediction mode to the current block when at least one of a width and a height of the current block is greater than a threshold; The video decoder, wherein the second prediction block is obtained based on a reference sample line included in an adjacent reference sample line.

8. 1. A video encoder comprising: a processor; and a memory configured to store a computer program executable by the processor; The processor: determining whether to apply a combined prediction mode to the current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained; a third predicted block of the current block is obtained by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; determining whether to apply the combined prediction mode to the current block includes not applying the combined prediction mode to the current block when at least one of a width and a height of the current block is greater than a threshold; A video encoder, wherein the second prediction block is obtained based on a reference sample line included in an adjacent reference sample line.

9. A computer-readable storage medium on which a computer program and a bitstream are stored, comprising: A computer-readable storage medium, the computer program causing a processor to perform the video decoding method of claim 1 or 2 to decode the bitstream to generate video.

10. A computer-readable storage medium on which a computer program and a bitstream are stored, comprising: A computer-readable storage medium, the computer program causing a processor to perform the video encoding method of claim 3 or 4 to generate the bitstream.

Images