Video signal encoding / decoding method and device for the same

Abstract

To provide a combined prediction method that combines a plurality of prediction methods to improve inter-prediction efficiency when encoding and decoding a video signal.SOLUTION: A video decoding method includes determining whether to apply a combined prediction mode to a current block, when the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block, and obtaining a third prediction block of the current block by performing a weighted sum operation on the first prediction block and the second prediction block.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a method for encoding / decoding a video signal and to an apparatus for said method. [Background technology]

[0002] As display panels become larger, video services with higher image quality are becoming increasingly necessary. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, active research is being conducted to improve video compression rates. As a representative example, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC) in 2009. JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013. However, its compression performance is approximately twice that of H.264 / AVC. With the rapid development of high-definition video services, the performance limitations of HEVC are gradually becoming apparent. Summary of the Invention [Problem to be solved by the invention]

[0003] SUMMARY OF THE INVENTION An object of the present invention is to provide a combined prediction method for combining multiple prediction methods when encoding / decoding a video signal, and an apparatus for implementing said method.

[0004] SUMMARY OF THE INVENTION An object of the present invention is to provide a method for dividing a coding block into a plurality of prediction units when encoding / decoding a video signal, and an apparatus for carrying out the method.

[0005] The technical problems that the present invention aims to achieve are not limited to those mentioned above, and those skilled in the art will be able to clearly understand other technical problems not mentioned from the following explanation. [Means for solving the problem]

[0006] A video signal decoding / encoding method according to the present invention includes determining whether to apply a combined prediction mode to a current block, obtaining a first predicted block and a second predicted block of the current block if the combined prediction mode is applied to the current block, and obtaining a third predicted block of the current block by performing a weighted sum operation on the first predicted block and the second predicted block. In this case, the first predicted block may be obtained based on motion information of a merge candidate of the current block, and the second predicted block may be obtained based on an intra prediction mode of the current block.

[0007] In the video signal decoding / encoding method according to the present invention, when the combined prediction mode is applied to the current block, triangulation may be set not to be applied to the current block.

[0008] In the video signal decoding / encoding method according to the present invention, the intra prediction mode of the current block may be set as a planar mode.

[0009] In the video signal decoding / encoding method according to the present invention, the second prediction block may be obtained based on a reference sample line included in an adjacent reference sample line.

[0010] In the video signal decoding / encoding method according to the present invention, when performing the weighted sum operation, weighting values ​​to be applied to the first predicted block and the second predicted block may be determined based on the predictive coding modes of adjacent blocks adjacent to the current block.

[0011] In the video signal decoding / encoding method according to the present invention, if at least one of the width and height of the current block is greater than a threshold, the combined prediction mode may not be applied to the current block.

[0012] In the video signal decoding / encoding method according to the present invention, if a flag indicating that a merge mode is to be applied to the current block is true, the combined prediction mode may be set to be applicable to the current block.

[0013] The brief features of the invention described above are exemplary embodiments of the detailed description of the invention set forth below, and are not intended to limit the scope of the invention. [Effects of the Invention]

[0014] According to the present invention, by providing a combined prediction method that combines a plurality of prediction methods, it is possible to improve inter prediction efficiency.

[0015] According to the present invention, by providing a method for dividing a coding block into a plurality of prediction blocks and deriving motion information of each of the plurality of prediction blocks, it is possible to improve inter-prediction efficiency.

[0016] The effects that can be achieved by the present invention are not limited to the above effects, and those skilled in the art will be able to clearly understand other effects not mentioned from the following description. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram of a video encoder according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing a variety of division types of coding blocks. [Figure 5] FIG. 5 is a diagram showing an example of division of a coding tree unit. [Figure 6] FIG. 6 is a diagram showing the shape of a basic data unit. [Figure 7] FIG. 7 is a diagram showing an example of dividing a coding block into a plurality of sub-blocks. [Figure 8] FIG. 8 is a diagram showing an example of dividing a coding block into a plurality of sub-blocks. [Figure 9] FIG. 9 is a flowchart of an inter prediction method according to an embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart of a process for deriving motion information of a current block in merge mode. [Figure 11] FIG. 11 is a diagram showing an example of candidate blocks for deriving merge candidates. [Figure 12] FIG. 12 is a diagram showing an example of dividing a coding block into a plurality of prediction units using diagonals. [Figure 13] FIG. 13 is a diagram showing an example of dividing a coding block into two prediction units. [Figure 14] FIG. 14 is a diagram showing an example of dividing a coding block into a plurality of prediction blocks of various sizes. [Figure 15] FIG. 15 is a diagram showing an example in which diagonal division is applied to a transform unit. [Figure 16] FIG. 16 is a diagram showing adjacent blocks for deriving triangle merge candidates. [Figure 17] FIG. 17 is a diagram illustrating an example of determining the availability of neighboring blocks for each triangular prediction unit. [Figure 18] 18 and 19 are diagrams illustrating an example of deriving a predicted sample by performing a weighted sum operation on a first predicted sample and a second predicted sample. [Figure 19] FIG. 19 is a diagram illustrating an example of deriving a predicted sample by performing a weighted sum operation on a first predicted sample and a second predicted sample. [Figure 20]FIG. 20 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention. [Figure 21] FIG. 21 is a diagram showing intra prediction modes. [Figure 22] FIG. 22 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Figure 23] FIG. 23 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in one row. [Figure 24] FIG. 24 is a diagram showing angular intra prediction modes and angles formed by lines parallel to the x-axis. [Figure 25] FIG. 25 illustrates an example of obtaining predicted samples when the current block is non-square. [Figure 26] FIG. 26 is a diagram showing wide-angle intra prediction modes. [Figure 27] FIG. 27 is a diagram showing an embodiment in which PDPC is applied. [Figure 28] FIG. 28 shows an example of applying weighting values. [Figure 29] FIG. 29 illustrates an example of assigning different weighting values ​​to sub-blocks depending on whether the merge candidate has bidirectional motion information. [Figure 30] FIG. 30 is a diagram illustrating an example of assigning different weighting values ​​to sub-blocks based on the intra-prediction mode of the current block. [Figure 31] FIG. 31 is a diagram showing an example of specifying a second merging candidate in consideration of the search order of candidate blocks. [Figure 32] FIG. 32 is a diagram showing an example of selecting a first merging candidate and a second merging candidate from merging candidates derived from non-adjacent blocks. [Figure 33] FIG. 33 is a diagram illustrating an example of determining the second merging candidate. [Figure 34] FIG. 34 is a diagram showing an example of determining a weighting value to be applied to a prediction block based on the shape of a candidate block. [Figure 35] FIG. 35 shows an example of replaced merging candidates. [Figure 36] FIG. 36 shows the sub-blocks that perform the secondary transformation. [Figure 37] FIG. 37 shows the sub-blocks that perform the secondary transformation. [Figure 38] FIG. 38 is a diagram for explaining an example of determining the transformation type of the current block. [Figure 39] FIG. 39 is a flowchart showing the process of determining the block strength. [Figure 40] FIG. 40 is a diagram showing predefined filter candidates. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0019] Video encoding and decoding is performed on a block-by-block basis, e.g., encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, or reconstruction can be performed on coding blocks, transform blocks, or prediction blocks.

[0020] Hereinafter, the block to be coded / decoded is referred to as a “current block.” For example, depending on the current coding / decoding process step, the current block may refer to a coding block, a transformation block, or a prediction block.

[0021] Additionally, as used herein, the term "unit" may be understood to refer to a basic unit for performing a specific encoding / decoding process, and "block" may be understood to refer to a sample array of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described below, a coding block and a coding unit may be understood to have the same meaning.

[0022] FIG. 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0023] As shown in FIG. 1, the video encoding device 100 may include a picture partitioning unit 110, prediction units 120 and 125, a transform unit 130, a quantization unit 135, a reordering unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0024] 1 are shown independently, which does not imply that each component is a separate hardware or software component, but rather indicates various characteristic functions of the video encoding device. That is, for convenience of explanation, each component may be arranged in such a manner that at least two of the components are combined into one component, or one component is divided into multiple components, thereby performing its function. Such embodiments in which each component is combined and those in which each component is separated are also within the scope of the present invention, provided that they do not deviate from the essence of the present invention.

[0025] Some structural elements are not necessary to perform the essential functions of the present invention, but are optional structural elements used only to improve performance. The present invention may be implemented by including only the components necessary to realize the essence of the present invention, other than the structural elements used only to improve performance, and a structure including only the necessary structural elements other than the optional structural elements used only to improve performance also falls within the scope of the present invention.

[0026] The picture partitioning unit 110 may divide an input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture partitioning unit 110 may divide one picture into a plurality of combinations of coding units, prediction units, and transform units, and may select one combination of coding units, prediction units, and transform units based on a predetermined standard (e.g., a cost function) to code the picture.

[0027] For example, a picture may be divided into multiple coding units. To divide a picture into coding units, a recursive tree structure, such as a quad tree structure, may be used, with one video or largest coding unit as the root, and the coding unit may be divided into other coding units, which may have child nodes corresponding to the number of coding units into which it is divided. Coding units that are not further divided due to some restriction become leaf nodes. That is, assuming that a coding unit can only realize square divisions, one coding unit can be divided into at most four other coding units.

[0028] Hereinafter, in the embodiments of the present invention, the encoding unit may refer to a unit that performs encoding, and may also refer to a unit that performs decoding.

[0029] The prediction units in one coding unit may be divided into at least one of square or rectangular shapes of the same size, and one prediction unit in one coding unit may be divided into a different shape and / or size than the other prediction units.

[0030] When a prediction unit for performing intra prediction using a coding unit is not the smallest coding unit, intra prediction may be performed without dividing the prediction unit into a plurality of N×N prediction units.

[0031] The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode may be determined by the prediction unit, or the prediction may be performed by a transform unit. The generated residual values ​​(residual block) between the prediction block and the original block may be input to the transform unit 130. Furthermore, prediction mode information, motion vector information, etc. for prediction may be coded together with the residual values ​​by the entropy coding unit 165 and transmitted to the decoder. When a specific coding mode is used, the original block may be directly coded and transmitted to the decoder without generating a prediction block by the prediction units 120 and 125.

[0032] The inter prediction unit 120 may predict a prediction unit based on information about at least one picture before or after the current picture. In some cases, the prediction unit may also be predicted based on information about some coded regions in the current picture. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0033] The reference picture interpolator receives reference picture information from the memory 155 and can generate integer or fractional pixel information based on the reference picture. For luma pixels, a DCT-based 8-tap interpolation filter with different filter coefficients may be used to generate fractional pixel information in 1 / 4 pixel units. For chroma signals, a DCT-based 4-tap interpolation filter with different filter coefficients may be used to generate fractional pixel information in 1 / 8 pixel units.

[0034] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. A motion vector may be calculated using a variety of methods, such as a full search-based block matching algorithm (FBMA), a three-step search (TSS), or a new three-step search algorithm (NTS). The motion vector may have a motion vector value in half-pixel or quarter-pixel units based on the interpolated pixels. The motion prediction unit may predict the current prediction unit using various motion prediction methods. The motion prediction method may include a skip method, a merge method, an advanced motion vector prediction (AMVP), an intra block copy method, and the like.

[0035] The intra prediction unit 125 may generate a prediction unit based on reference pixel information surrounding a current block as pixel information in the current picture. If a neighboring block of the current prediction unit is a block on which inter prediction has been performed and the reference pixels are pixels on which inter prediction has been performed, the reference pixels included in the block on which inter prediction has been performed may be used as reference pixel information for the neighboring block on which intra prediction has been performed. That is, if reference pixels are unavailable, at least one of the available reference pixels may replace the unavailable reference pixel information.

[0036] In intra prediction, prediction modes may include an angular prediction mode that uses reference pixel information based on the prediction direction, and a non-angular mode that does not use direction information when performing prediction. The prediction mode for luma information and the prediction mode for chroma information may be different, and intra prediction mode information used to predict luma information or predicted luma signal information may be applied to predict chroma information.

[0037] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction may be performed on the prediction unit based on the pixels located to the left, the upper left, and the upper part of the prediction unit. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction may be performed using reference pixels based on the transform unit. In addition, intra prediction using NxN division may be applied only to the smallest coding unit.

[0038] The intra prediction method may generate a predicted block after applying an adaptive intra smoothing (AIS) filter to reference pixels based on the prediction mode. The type of the adaptive intra smoothing filter for the reference pixels may vary. To perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted according to the intra prediction mode of prediction units existing around the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted by the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same may be transmitted using predetermined flag information. If the prediction modes of the current prediction unit and the surrounding prediction units are different, the prediction mode information of the current block may be encoded by performing entropy coding.

[0039] In addition, a residual block including residual information can be generated, where the residual information is the difference between the prediction unit and the original block of the prediction unit, which is predicted by the prediction unit generated in the prediction unit 120, 125. The generated residual block can be input to the conversion unit 130.

[0040] In the transform unit 130, a discrete cosine transform (DCT) is performed. The residual block may be transformed using a transform method such as a DCT (Digital Transform) or a Discrete Sine Transform (DST). The residual block includes residual information between the original block and the prediction unit generated by the prediction units 120 and 125. The DCT transform core includes at least one of DCT2 or DCT8, and the DST transform core includes DST7. Whether to transform the residual block using DCT or DST may be determined based on intra-prediction mode information of the prediction unit used to generate the residual block. Furthermore, the transform of the residual block may be skipped. A flag indicating whether to skip the transform of the residual block may be coded. For residual blocks whose size is equal to or smaller than a threshold, luma component, or chroma component (4:4:4 format or smaller), the transform may be skipped.

[0041] The quantization unit 135 can quantize the values ​​transformed into the frequency domain by the transform unit 130. The quantization coefficients can vary depending on the importance of the block or video. The values ​​calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the reordering unit 160.

[0042] The reordering unit 160 can perform reordering of the coefficient values ​​on the quantized residual values.

[0043] The reordering unit 160 may convert two-dimensional block shape coefficients into one-dimensional vector format using a coefficient scanning method. For example, the reordering unit 160 may scan DC coefficients through high-frequency region coefficients using a zig-zag scan method and convert them into one-dimensional vector format. Depending on the size of the transform unit and the intra prediction mode, instead of zig-zag scanning, vertical scanning, which scans two-dimensional block shape coefficients along the column direction, and horizontal scanning, which scans two-dimensional block shape coefficients along the row direction, may be used. That is, depending on the size of the transform unit and the intra prediction mode, it may be determined which scanning method to use among zig-zag scanning, vertical scanning, and horizontal scanning.

[0044] The entropy coding unit 165 can perform entropy coding based on the values ​​calculated by the reordering unit 160. For example, the entropy coding may use a number of coding methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0045] The entropy coding unit 165 can encode multiple information such as residual value coefficient information and block type information of the coding unit from the rearrangement unit 160 and the prediction units 120 and 125, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0046] The entropy coding unit 165 may perform entropy coding on the coefficient values ​​of the coding unit input from the rearrangement unit 160 .

[0047] The inverse quantization unit 140 and the inverse transform unit 145 inversely quantize the values ​​quantized by the quantization unit 135 and inversely transform the values ​​transformed by the transform unit 130. The residual values ​​generated by the inverse quantization unit 140 and the inverse transform unit 145 can be merged with prediction units predicted by the motion prediction unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0048] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0049] A deblocking filter can remove block artifacts generated in a reconstructed picture due to boundaries between blocks. To determine whether to perform deblocking, it can determine whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied based on the required deblocking filtering strength. In addition, when performing vertical filtering and horizontal filtering in the process of using a deblocking filter, horizontal filtering and vertical filtering can be processed synchronously.

[0050] The offset correction unit can correct the offset between the original video and the video on which deblocking is performed on a pixel-by-pixel basis. To perform offset correction on a specified picture, a method may be used in which pixels included in the video are divided into a predetermined number of regions, regions on which offset is to be performed are determined, and an offset is applied to the corresponding region, or an offset is applied taking into account edge information of each pixel.

[0051] Adaptive loop filtering (ALF) may be performed based on a comparison between the filtered reconstructed picture and the original video. After dividing the pixels included in the video into predetermined sets, filtering can be performed differentially for each set by determining a filter to be used for the corresponding set. Information related to whether adaptive loop filtering is to be applied and a luminance signal may be transmitted by a coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may vary for each block. Furthermore, the same type (fixed type) of adaptive loop filter may be applied regardless of the characteristics of the block to which it is applied.

[0052] The memory 155 can store the reconstructed blocks or pictures calculated by the filter unit 150 and can provide the stored reconstructed blocks or pictures to the prediction units 120, 125 when performing inter prediction.

[0053] FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0054] As shown in FIG. 2, the video decoder 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0055] When a video stream is input from a video encoder, the input stream may be decoded according to the reverse steps of the video encoder.

[0056] The entropy decoding unit 210 may perform entropy decoding in a manner that is the reverse of the entropy encoding performed by the entropy encoding unit of the video encoder. For example, the entropy decoding unit 210 may apply a number of methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC), in accordance with the methods performed in the video encoder.

[0057] The entropy decoding unit 210 can decode information related to intra-prediction and inter-prediction performed by the encoder.

[0058] The reordering unit 215 may perform reordering based on the method used in the encoding unit to reorder the stream entropy decoded by the entropy decoding unit 210. Multiple coefficients represented in one-dimensional vector format may be reconstructed into two-dimensional block-shaped coefficients and reordered. The reordering unit 215 may receive information related to coefficient scanning performed in the encoding unit and perform reordering by a method of performing a backward scan in accordance with the scanning order performed in the corresponding encoding unit.

[0059] The inverse quantization unit 220 may perform inverse quantization based on a quantization parameter provided by the encoder and the coefficient values ​​of the block to be reordered.

[0060] The inverse transform unit 225 may perform an inverse discrete cosine transform and an inverse discrete sine transform on the quantization result performed by the video encoder. The inverse discrete cosine transform and the inverse discrete sine transform are inverse transforms of the transforms performed in the transform unit, i.e., belong to the inverse transforms of the discrete cosine transform and the discrete sine transform. The DCT transform core may include at least one of a DCT2 or a DCT8, and the DST transform core may include a DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform may not be performed in the inverse transform unit 225. The inverse transform may be performed by a transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder may selectively perform a transform method (e.g., DCT or DST) based on multiple information such as a prediction method, a size of the current block, and a prediction direction.

[0061] The prediction units 230 and 235 can generate prediction blocks based on information related to the generation of the prediction blocks provided by the entropy decoding unit 210 and information on previously decoded blocks or pictures provided by the memory 245 .

[0062] As described above, when performing intra prediction in the same manner as the operation in a video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixels located to the left, the upper left, and the upper pixels of the prediction unit; if the size of the prediction unit when performing intra prediction is different from the size of the transform unit, intra prediction may be performed using reference pixels based on the transform unit. In addition, intra prediction using NxN division may be applied only to the smallest coding unit.

[0063] The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives multiple types of information, such as prediction unit information, prediction mode information for the intra prediction method, and motion prediction-related information for the inter prediction method, input from the entropy decoding unit 210, classifies the prediction unit based on the current coding unit, and determines whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may use information necessary for inter prediction of the current prediction unit provided by the video encoder and perform inter prediction on the current prediction unit based on information included in at least one picture preceding or following the current picture to which the current prediction unit belongs. Alternatively, the inter prediction may also be performed based on information about a reconstructed region in the current picture to which the current prediction unit belongs.

[0064] To perform inter prediction, the coding unit can determine whether the motion prediction method of the prediction unit included in the corresponding coding unit is skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), or intra block copy mode.

[0065] The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit for which intra prediction has been performed, the intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The adaptive intra smoothing filter performs filtering on reference pixels of the current block and may determine whether to apply the filter based on the prediction mode of the current prediction unit. Adaptive intra smoothing filtering may be performed on reference pixels of the current block using the prediction mode of the prediction unit and adaptive intra smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode in which adaptive intra smoothing filtering is not performed, the adaptive intra smoothing filter may not be applied.

[0066] The reference pixel interpolator may generate reference pixels in units of integer or fractional pixels by interpolating reference pixels when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values ​​for interpolating reference pixels. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating reference pixels, the reference pixels may not need to be interpolated. When the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0067] The reconstructed block or picture may be provided to a filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0068] The video decoder may receive, from the video encoder, information related to whether to apply a deblocking filter to a corresponding block or picture, and information related to whether to apply a strong filter or a weak filter when applying the deblocking filter. The video decoder may receive, from a deblocking filter of the video decoder, information related to a deblocking filter provided by the video encoder, and perform deblocking filtering on the corresponding block in the video decoder.

[0069] The offset correction unit can perform offset correction on the reconstructed video based on the type and offset amount information for offset correcting the video during encoding.

[0070] The ALF can be applied to the coding unit based on information provided by the encoder regarding whether to apply the ALF, ALF coefficient information, etc. Such ALF information can be provided by being included in a specific parameter set.

[0071] The memory 245 stores the reconstructed pictures or blocks so that they can be used as reference pictures or blocks and provide the reconstructed pictures to an output.

[0072] FIG. 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0073] A coding block of the largest size may be defined as a coding tree block. A picture may be divided into multiple coding tree units (CTUs). A coding tree unit is a coding unit of the largest size and may also be referred to as a largest coding unit (LCU). Figure 3 shows an example of dividing a picture into multiple coding tree units.

[0074] The size of the coding tree unit may be defined at the picture level or the sequence level, and thus the picture parameter set or the sequence parameter set may be signaled to transmit information indicating the size of the coding tree unit.

[0075] For example, the coding tree unit size for the entire picture in the sequence may be set as 128 x 128. Alternatively, the coding tree unit size may be determined to be either 128 x 128 or 256 x 256 at the picture level. For example, the coding tree unit size for the first picture may be set as 128 x 128, and the coding tree unit size for the second picture may be set as 256 x 256.

[0076] Coding blocks can be generated by dividing a coding tree unit. A coding block indicates a basic unit for encoding / decoding processing. For example, prediction or transformation may be performed by different coding blocks, or a predictive coding mode may be determined by different coding blocks. The predictive coding mode indicates a method for generating a predicted picture. For example, the predictive coding mode may include intra prediction (intra prediction), inter prediction (inter prediction), current picture referencing (CPR), intra block copy (IBC), or combined prediction. For a coding block, at least one predictive coding mode among intra prediction, inter prediction, current picture referencing, and combined prediction may be used to generate a predictive block related to the coding block.

[0077] Information indicating the predictive coding mode of the current block may be transmitted by a signal via the stream. For example, the information may be a one-bit flag indicating whether the predictive coding mode is intra mode or inter mode. Current picture reference or combined prediction may be used only when it is determined that the predictive coding mode of the current block is inter mode.

[0078] The current picture reference is used to set the current picture as a reference picture and obtain a prediction block for the current block from a coded / decoded area in the current picture. The current picture refers to a picture including the current block. Information indicating whether the current picture reference is applied to the current block may be transmitted via a signal through a stream. For example, the information may be a 1-bit flag. If the flag is true, the predictive coding mode of the current block may be determined as the current picture reference; if the flag is false, the prediction mode of the current block may be determined as inter prediction.

[0079] Alternatively, the predictive coding mode of the current block may be determined based on a reference picture index. For example, if the reference picture index points to the current picture, the predictive coding mode of the current block may be determined as current picture reference. If the reference picture index points to another picture other than the current picture, the predictive coding mode of the current block may be determined as inter prediction. That is, current picture reference is a prediction method that uses information on an area in the current picture after encoding / decoding, and inter prediction is a prediction method that uses information on another picture after encoding / decoding.

[0080] Combined prediction refers to a coding mode that combines two or more of intra prediction, inter prediction, and current picture reference. For example, when combined prediction is applied, a first predicted block may be generated by one of intra prediction, inter prediction, or current picture reference, and a second predicted block may be generated by another of intra prediction, inter prediction, or current picture reference. When generating the first predicted block and the second predicted block, a final predicted block may be generated by an average or weighted sum operation of the first predicted block and the second predicted block. Information indicating whether combined prediction is applied may be transmitted via a signal through the stream. The information may be a one-bit flag.

[0081] FIG. 4 is a diagram showing a variety of division types of coding blocks.

[0082] A coding block may be divided into multiple coding blocks by quadtree division, binary tree division, or ternary tree division, and the divided coding block may be further divided into multiple coding blocks by quadtree division, binary tree division, or ternary tree division.

[0083] Quadtree partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of the quadtree partitioning, the current block may be divided into four square segments (see "SPLIT_QT" in part (a) of Figure 4).

[0084] Binary tree splitting refers to a splitting technique that splits a current block into two blocks. The process of splitting the current block into two blocks along the vertical direction (i.e., using a vertical line crossing the current block) may be referred to as vertical binary tree splitting, and the process of splitting the current block into two blocks along the horizontal direction (i.e., using a horizontal line crossing the current block) may be referred to as horizontal binary tree splitting. After binary tree splitting, the current block may be split into two non-square segments. 'SPLIT_BT_VER' in part (b) of Figure 4 indicates the result of vertical binary tree splitting, and 'SPLIT_BT_HOR' in part (c) of Figure 4 indicates the result of horizontal binary tree splitting.

[0085] Ternary tree partitioning refers to a partitioning technique that divides a current block into three blocks. The process of dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) may be referred to as vertical ternary tree partitioning, and the process of dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines crossing the current block) may be referred to as horizontal ternary tree partitioning. After ternary tree partitioning, the current block may be divided into three non-square segments. In such a case, the width / height of the segment located at the center of the current block may be twice the width / height of the other segments. 'SPLIT_TT_VER' in part (d) of Figure 4 indicates the result of vertical ternary tree partitioning, and 'SPLIT_TT_HOR' in part (e) of Figure 4 indicates the result of horizontal ternary tree partitioning.

[0086] The number of times a coding tree unit is divided may be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit may be determined at the sequence or picture level. This allows the maximum partitioning depth of a coding tree unit to be different for different sequences or pictures.

[0087] Alternatively, the maximum decomposition depth for each of multiple decomposition techniques may be determined independently, e.g., the maximum decomposition depth allowed for quadtree decomposition may be different from the maximum decomposition depth allowed for binary tree and / or ternary tree decomposition.

[0088] The encoder may transmit information indicating at least one of the partition shape or partition depth of the current block by a signal via the stream, and the decoder may determine the partition shape and partition depth of the coding tree unit based on the information parsed from the stream.

[0089] FIG. 5 is a diagram showing an example of division of a coding tree unit.

[0090] The process of dividing a coding block using a division technique such as quad-tree division, binary tree division, and / or ternary tree division may be referred to as multi-tree partitioning.

[0091] The coding blocks generated by applying multi-tree partitioning to a coding block may be referred to as multiple downstream coding blocks. If the partitioning depth of the coding block is k, the partitioning depth of the multiple downstream coding blocks is set to k+1.

[0092] On the other hand, among multiple coding blocks with a division depth of k+1, the coding block with a division depth of k may be referred to as an upstream coding block.

[0093] The partition type of the currently coded block may be determined based on at least one of the partition shape of the upstream coded block or the partition type of the adjacent coded block. The adjacent coded block may be adjacent to the currently coded block and may include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the currently coded block. The partition type may include at least one of whether to partition into a quadtree, whether to partition into a binary tree, the partition direction of the binary tree, whether to partition into a ternary tree, or the partition direction of the ternary tree.

[0094] To determine the partitioning shape of the coding block, information indicating whether the coding block has been split may be transmitted via a signal through the stream, which is a 1-bit flag "split_cu_flag" that, if true, indicates that the coding block is split using the multi-tree partitioning technique.

[0095] If "split_cu_flag" is true, a signal may be sent over the stream indicating whether the coding block has been quadtree split. The information is a 1-bit flag "split_qt_flag", and if the flag is true, the coding block may be split into four blocks.

[0096] 5 shows an example in which a coding tree unit is quadtree divided to generate four coding blocks with a division depth of 1. It also shows that quadtree division is applied again to the first and fourth coding blocks among the four coding blocks generated as a result of the quadtree division. Finally, four coding blocks with a division depth of 2 can be generated.

[0097] In addition, quadtree division may be applied again to the coding block with division depth 2 to generate a coding block with division depth 3.

[0098] When quadtree partitioning is not applied to a coding block, it may be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block, taking into consideration at least one of the size of the coding block, whether the coding block is located on a picture boundary, the maximum partition depth, or the partition shape of adjacent blocks. When it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating the partitioning direction may be transmitted by a signal via a stream. The information may be a one-bit flag "mtt_split_cu_vertical_flag". Whether the partitioning direction is vertical or horizontal may be determined based on the flag. Furthermore, information indicating whether binary tree partitioning or ternary tree partitioning is to be applied to the coding block may be transmitted by a signal via a stream. The information may be a one-bit flag "mtt_split_cu_binary_flag". Whether binary tree partitioning or ternary tree partitioning is to be applied to the coding block may be determined based on the flag.

[0099] For example, the example shown in Figure 5 shows that vertical binary tree partitioning is applied to a coding block with a partitioning depth of 1, vertical ternary tree partitioning is applied to the left coding block of the coding blocks generated as a result of the partitioning, and vertical binary tree partitioning is applied to the right coding block.

[0100] When implementing a video encoding or decoding device, there is a problem that it is difficult to process an area exceeding a threshold due to hardware performance. For example, if the hardware performance allows simultaneous processing of at most 4096 samples, a data unit of size 64x64 must be redundantly accessed and processed, and there is a possibility that data in an area of ​​more than 4096 samples cannot be processed simultaneously. As described above, a basic unit of data processing may be defined as a pipeline-based basic data unit (virtual processing data unit, VPDU, hereinafter referred to as a basic data unit).

[0101] The elementary data units may be divided into square, non-square or non-rectangular types.

[0102] FIG. 6 is a diagram showing the shape of a basic data unit.

[0103] A basic data unit may include samples up to the maximum number of samples that can be processed simultaneously. For example, in the example shown in Figure 6(a), a square block of 64x64 size may be set as the basic data unit. Alternatively, a non-square block may be set as the basic data unit. For example, in the example shown in Figure 6(b) or 6(c), a block of 32x128 size or a block of 64x32 size may be set as the basic data unit.

[0104] Although not shown, basic data units of triangles, L-shapes or polygons may be defined.

[0105] Information for determining the elementary data units may be transmitted by a signal via the stream, and the information may be used to determine at least one of the size or shape of the elementary data units. Based on the information, it may be determined whether non-square elementary data units or non-rectangular elementary data units are allowed.

[0106] Alternatively, at least one of the size or shape of the elementary data units may be predefined in the encoder and decoder.

[0107] Whether a division type of a coding block is permissible may be determined taking into account the size of the basic data unit. For example, if a coding block generated as a result of dividing the coding block is larger than the basic data unit, such division may not be permissible. Alternatively, if a non-square coding block generated as a result of dividing the coding block is larger than the basic data unit, such division may not be permissible. For example, if the width or height of the coding block is larger than a threshold or the number of samples included in the coding block is larger than a threshold, binary tree or ternary tree division may not be permissible. Therefore, encoding of information related to binary tree or ternary tree division may be omitted.

[0108] Alternatively, it may be set so that coding blocks larger than the basic data unit must be split. Alternatively, it may be set so that binary tree splitting or ternary tree splitting must be performed on coding blocks larger than the basic data unit. Therefore, for coding blocks larger than the basic data unit, the flag split_flag indicating whether the coding block has been split may not be coded, but the value of the flag may be derived as 1.

[0109] As another example, a coding block larger than a basic data unit may be divided into multiple sub-blocks, where a sub-block may be set as a prediction unit as a basic unit for prediction or a transform unit as a basic unit for transformation and / or quantization. In such a case, dividing a coding block into multiple prediction units may be defined as VPDU prediction unit division, and dividing a coding block into multiple transform units may be defined as VPDU transform unit division.

[0110] At least one of VPDU prediction unit segmentation and VPDU transform unit segmentation may be applied to the coding block. The segmentation type of the coding block to which VPDU prediction unit segmentation is applied may be set to be the same as the segmentation type of the coding block to which VPDU transform unit segmentation is applied.

[0111] When only VPDU prediction unit splitting is applied to a coding block, prediction is performed for each sub-block, but transformation and / or quantization may be performed for the coding block. In such a case, a prediction mode, such as a predictive coding mode, an intra-prediction mode, or an inter-prediction mode, may be determined for the coding block.

[0112] When only VPDU transform unit splitting is applied to a coding block, prediction is performed on sub-blocks, but transform and / or quantization may be performed on each sub-block.

[0113] 7 and 8 are diagrams showing examples of dividing a coding block into a plurality of sub-blocks.

[0114] FIG. 7 is a diagram showing the division modes when only square basic data units are allowed, and FIG. 8 is a diagram showing the division modes when square basic data units and non-square basic data units are allowed.

[0115] Assuming that only square basic data units are allowed, in Figures 7(a) and 7(b), CU0 and CU2 are defined as two different VPDUs, and CU1 is defined as four different VPDUs. Therefore, CU0 and CU2 may be divided into two sub-blocks, and CU1 may be divided into four sub-blocks.

[0116] Assuming that square and non-square basic data units are allowed, in Figures 8(a) and 8(b), CU0 and CU2 may be defined as one VPDU, and CU1 may be defined as two different VPDUs. Therefore, CU0 and CU2 are not divided into sub-blocks, but CU1 may be divided into two sub-blocks.

[0117] In such a case, CU1 may be divided into square or non-square sub-blocks. For example, CU1 may be divided into two square sub-blocks based on a horizontal line dividing CU1 into upper and lower parts. Alternatively, CU1 may be divided into two non-square sub-blocks based on a vertical line dividing CU1 into left and right parts.

[0118] If there are multiple candidate partition types that can be applied to a coding block, information about any one of the multiple candidate partition types may be transmitted by a signal via the stream. For example, the information may indicate whether the coding block is partitioned into square sub-blocks or whether the coding block is partitioned into non-square sub-blocks.

[0119] Alternatively, dividing a coding block into square sub-blocks may be set to have a higher priority than dividing a coding block into non-square sub-blocks, e.g., if a coding block cannot be divided into square sub-blocks, dividing the coding block into non-square sub-blocks may be allowed.

[0120] Alternatively, the division type of the coding block may be determined based on the division type of the coding block as the father node. For example, when the father node coding block is divided by a ternary tree, the coding block may be set to be divided into square sub-blocks. On the other hand, when the father node coding block is divided by a binary tree or a ternary tree, the coding block may be set to be divided into non-square sub-blocks.

[0121] Inter-prediction refers to predicting the predictive coding mode of a current block using information from a previous picture. For example, a block located at the same position as the current block in the previous picture (hereinafter referred to as a collocated block) may be set as the predictive block of the current block. Hereinafter, a predictive block generated based on a block located at the same position as the current block is referred to as a collocated prediction block.

[0122] On the other hand, if an object in a previous picture has already moved to another position in the current picture, the current block may be effectively predicted based on the object's motion. For example, if the object's movement direction and size can be determined by comparing the previous picture with the current picture, a predicted block (or predicted picture) of the current block may be generated taking into account the object's motion information. Hereinafter, the predicted block generated based on the motion information may be referred to as a motion predicted block.

[0123] A residual block is obtained by subtracting the predicted block from the current block. In this case, if the object is moving, the motion prediction block may be substituted for the collocated prediction block, thereby reducing the energy of the residual block and improving the compression performance of the residual block.

[0124] The process of generating a prediction block using motion information as described above may be referred to as motion compensated prediction. In most inter predictions, a prediction block may be generated using motion compensated prediction.

[0125] The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weighting value index. The motion vector indicates the movement direction and size of an object. The reference picture index specifies a reference picture of the current block among multiple reference pictures included in a reference picture list. The prediction direction refers to one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of L0 direction motion information or L1 direction motion information may be used based on the prediction direction of the current block. The bidirectional weighting value index specifies a weighting value for the L0 prediction block and a weighting value applied to the L1 prediction block.

[0126] FIG. 9 is a flowchart of an inter prediction method according to an embodiment of the present invention.

[0127] As shown in FIG. 9, the inter prediction method includes determining an inter prediction mode of a current block (S901), obtaining motion information of the current block based on the determined inter prediction mode (S902), and performing motion compensation prediction of the current block based on the obtained motion information (S903).

[0128] The inter prediction mode indicates a number of techniques for determining motion information of the current block, and may include an inter prediction mode using translational motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translational motion information may include a merge mode and an advanced motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. Depending on the inter prediction mode, the motion information of the current block may be determined based on information analyzed from neighboring blocks or streams adjacent to the current block.

[0129] The motion information of the current block may be derived from the motion information of another block of the current block. The other block may be a block that is preferentially encoded / decoded using inter prediction rather than the current block. When the motion information of the current block is set to be the same as the motion information of another block, this may be defined as a merge mode. Furthermore, when the motion vector of the other block is set as a predicted value of the motion vector of the current block, this may be defined as a motion vector prediction mode.

[0130] FIG. 10 is a flowchart of a process for deriving motion information of a current block in merge mode.

[0131] Merge candidates for a current block may be derived (S1001). Merge candidates for the current block may be derived from blocks that are encoded / decoded using inter prediction before the current block.

[0132] FIG. 11 is a diagram showing an example of candidate blocks for deriving merge candidates.

[0133] A candidate block may include at least one of a neighboring block including samples neighboring the current block, or a non-neighboring block including samples not neighboring the current block. Hereinafter, samples for determining a candidate block are designated as reference samples. Also, reference samples neighboring the current block are referred to as neighboring reference samples, and reference samples not neighboring the current block are referred to as non-neighboring reference samples.

[0134] The neighboring reference sample may be included in the neighboring column of the leftmost column of the current block or the neighboring row of the topmost row of the current block. For example, if the coordinates of the top left sample of the current block are (0,0), at least one of the block including the reference sample at the (-1,H-1) position, the block including the reference sample at the (W-1,-1) position, the block including the reference sample at the (W,-1) position, the block including the reference sample at the (-1,H) position, or the block including the reference sample at the (-1,-1) position may be used as a candidate block. Referring to the drawing, the neighboring blocks with indexes 0 to 4 may be used as candidate blocks.

[0135] A non-adjacent reference sample indicates a sample having at least one of a predetermined x-axis distance or a predetermined y-axis distance from a reference sample adjacent to the current block. For example, at least one of a block including a reference sample having a predetermined x-axis distance from a left reference sample, a block including a non-adjacent sample having a predetermined y-axis distance from an upper reference sample, or a block including a non-adjacent sample having predetermined x-axis and y-axis distances from an upper-left reference sample may be used as a candidate block. The predetermined value may be an integer such as 4, 8, 12, or 16. Referring to the drawing, at least one of blocks with indexes 5 to 26 may be used as a candidate block.

[0136] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block may be set to be ineligible for use as merge candidates. For example, if a reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, the candidate block containing the reference sample may be set to be ineligible for use as a merge candidate.

[0137] Furthermore, merging candidates may be derived from temporally adjacent blocks included in a picture different from the current block. For example, merging candidates may be derived from co-located blocks included in a co-located picture. Any one of multiple reference pictures included in the reference picture list may be set as the co-located picture. Index information indicating the co-located picture among the multiple reference pictures may be transmitted via a signal via the stream. Alternatively, a reference picture with a predefined index among the multiple reference pictures may be determined as the co-located picture.

[0138] The motion information of the merge candidate may be set to be the same as the motion information of the candidate block, for example, at least one of the motion vector, reference picture index, prediction direction, or bidirectional weighting value index of the candidate block may be set as the motion information of the merge candidate.

[0139] A merge candidate list including the merge candidates may be generated (S1002).

[0140] The indices of the multiple merge candidates in the merge candidate list may be assigned according to a predetermined order, for example, the order of merge candidates derived from the left neighboring block, merge candidates derived from the above neighboring block, merge candidates derived from the above-right neighboring block, merge candidates derived from the below-left neighboring block, merge candidates derived from the above-left neighboring block, and merge candidates derived from the temporal neighboring blocks.

[0141] If the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates may be selected (S1003). Specifically, information for specifying one of the multiple merge candidates may be transmitted via a signal over the stream. For example, index information merge_idx indicating one of the multiple merge candidates included in the merge candidate list may be transmitted via a signal over the stream.

[0142] If the number of merge candidates included in the merge candidate list is less than a threshold, merge candidates included in the inter-motion information list may be added to the merge candidate list. The threshold may be the maximum number of merge candidates that can be included in the merge candidate list or the maximum number of merge candidates minus an offset amount. The offset amount may be an integer such as 1 or 2. The inter-motion information list may include merge candidates derived based on blocks to be coded / decoded before the current block.

[0143] The inter motion information list includes merge candidates derived from blocks coded / decoded by inter prediction in the current picture. For example, the motion information of the merge candidates included in the inter motion information list may be set to be the same as the motion information of the blocks coded / decoded by inter prediction. The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weighting value index.

[0144] A coding block may be divided into a plurality of prediction units, and prediction may be performed for each of the divided prediction units. A prediction unit refers to a basic unit for prediction.

[0145] The coding block may be divided using at least one of vertical lines, horizontal lines, diagonal lines, or diagonal lines. Information for determining at least one of the number, angle, or position of lines dividing the coding block may be transmitted via a signal via the stream. For example, information indicating any one of candidate division types for the coding block may be transmitted via a signal via the stream, or information specifying any one of multiple line candidates for dividing the coding block may be transmitted via a signal via the stream. Alternatively, information for determining the number or type of line candidates for dividing the coding block may be transmitted via a signal via the stream. For example, a one-bit flag may be used to determine whether diagonal lines with angles greater than a diagonal and / or diagonal lines with angles smaller than a diagonal can be used as line candidates.

[0146] Alternatively, at least one of the number, angle or position of lines dividing the coding block may be adaptively determined based on at least one of the intra prediction mode, inter prediction mode of the coding block, the position of available merging candidates or the division type of neighboring blocks.

[0147] When a coding block is divided into multiple prediction units, intra prediction or inter prediction may be performed on each of the divided prediction units.

[0148] FIG. 12 is a diagram showing an example of dividing a coding block into a plurality of prediction units using diagonals.

[0149] In the examples shown in Figures 12(a) and 12(b), a diagonal may be used to divide the coding block into two triangular prediction units.

[0150] 12(a) and 12(b) show that a coding block is divided into two prediction units using a diagonal line connecting two vertices of the coding block. However, a coding block may also be divided into two prediction units using a diagonal line where at least one end of the line does not pass through a vertex of the coding block.

[0151] FIG. 13 is a diagram showing an example of dividing a coding block into two prediction units.

[0152] In the examples shown in Figures 13(a) and 13(b), the coding block may be divided into two prediction units using a diagonal line whose ends touch the upper and lower boundaries of the coding block, respectively.

[0153] Alternatively, in the examples shown in Figures 13(c) and 13(d), the coding block may be divided into two prediction units using a diagonal line whose ends touch the left and right boundaries of the coding block, respectively.

[0154] Alternatively, the coding block may be divided into two prediction blocks of different sizes, for example, the diagonal line dividing the coding block is set to touch two boundary surfaces that form one vertex, thereby dividing the coding block into two prediction units of different sizes.

[0155] FIG. 14 is a diagram showing an example of dividing a coding block into a plurality of prediction blocks of different sizes.

[0156] In the examples shown in Figures 14(a) and 14(b), the coding block can be divided into two prediction units of different sizes by setting the diagonal line connecting the upper left or lower right corner of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block without passing through the upper left or lower right corner of the coding block.

[0157] Alternatively, in the examples shown in Figures 14(c) and 14(d), the coding block can be divided into two prediction units of different sizes by setting the diagonal line connecting the upper right or lower left corner of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block without passing through the upper left or lower right corner of the coding block.

[0158] Each prediction unit generated by dividing a coding block is referred to as an "Nth prediction unit." For example, in the examples shown in Figures 12 to 14, PU1 may be defined as the first prediction unit, and PU2 may be defined as the second prediction unit. The first prediction unit may refer to a prediction unit including a lower left sample or an upper left sample in a coding block, and the second prediction unit may refer to a prediction unit including an upper right sample or a lower right sample in a coding block.

[0159] Conversely, a prediction unit including the top right sample or bottom right sample in a coding block may be defined as the first prediction unit, and a prediction unit including the bottom left sample or top left sample in a coding block may be defined as the second prediction unit.

[0160] The following embodiments mainly describe a division example using a diagonal line. In particular, the process of dividing a coding block into two prediction units using a diagonal line is called diagonal division or triangular division, and the prediction unit generated by diagonal division is called a triangular prediction unit. However, of course, the following embodiments may use a division example using a vertical line, a horizontal line, or a diagonal line at an angle different from the diagonal line.

[0161] Whether to apply diagonal splitting to a coding block may be determined based on at least one of the slice type, the maximum number of merge candidates that can be included in the merge candidate list, the size of the coding block, the shape of the coding block, the predictive coding mode of the coding block, or the splitting type of the parent node.

[0162] For example, whether to apply diagonal division to a coding block may be determined depending on whether the current slice is a B-slice, and diagonal division is only allowed when the current slice is a B-slice.

[0163] Alternatively, whether to apply diagonal splitting to a coding block may be determined depending on whether the maximum number of merge candidates included in the merge candidate list is greater than or equal to 2. Diagonal splitting may be allowed only if the maximum number of merge candidates included in the merge candidate list is greater than or equal to 2.

[0164] Alternatively, if at least one of the width and height of the hardware is greater than 64, a 64x64 size data processing unit will be accessed redundantly. As a result, if at least one of the width and height of the coding block is greater than a threshold, it may not be permitted to divide the coding block into multiple prediction blocks. For example, if at least one of the height and width of the coding block is greater than 64 (e.g., if at least one of the width and height is 128), diagonal division may not be used.

[0165] Alternatively, given the maximum number of samples that can be processed simultaneously in a hardware implementation, diagonal splitting may not be allowed for coding blocks with a number of samples greater than a threshold, e.g., for coding tree blocks with a number of samples greater than 4096, diagonal splitting may not be allowed.

[0166] Alternatively, diagonal division may not be permitted for coding blocks that contain fewer samples than a threshold value. For example, if the number of samples in a coding block is fewer than 64, diagonal division may not be applied to the coding block.

[0167] Alternatively, whether to apply diagonal division to a coding block may be determined depending on whether the ratio of the width to the height of the coding block is smaller than a first threshold or whether the ratio of the width to the height of the coding block is larger than a second threshold, where the ratio of the width to the height of the coding block, whRatio, may be determined as the ratio of the width CbW to the height CbH of the coding block, and is shown in Equation 1 below.

[0168] equation 1 JPEG2025183407000002.jpg544

[0169] The second threshold may be the reciprocal of the first threshold, for example, if the first threshold is k, the second threshold may be 1 / k.

[0170] Diagonal division can be applied to a coding block only if the ratio of width to height of the coding block is between the first and second thresholds.

[0171] Alternatively, triangular division can be used only if the ratio of width to height of the coding block is less than a first threshold or greater than a second threshold, e.g., if the first threshold is 16, then diagonal division is not allowed for coding blocks of size 64x4 or 4x64.

[0172] Alternatively, whether diagonal splitting is allowed may be determined based on the split type of the parent node. For example, if the coding block as the parent node is split using a quadtree, diagonal splitting can be applied to the coding block as the leaf node. On the other hand, if the coding block as the parent node is split using a binary or ternary tree, diagonal splitting is not allowed for the coding block as the leaf node.

[0173] Alternatively, whether diagonal division is allowed may be determined based on the predictive coding mode of the coding block. For example, diagonal division may be allowed only when the coding block is coded by intra prediction, when the coding block is coded by inter prediction, or when the coding block is coded in a predefined inter prediction mode. The predefined inter prediction mode may indicate at least one of a merge mode, an advanced motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0174] Alternatively, whether diagonal division is allowed may be determined based on the size of the parallel processing region, for example, if the size of the coding block is larger than the size of the parallel processing region, diagonal division may not be used.

[0175] Furthermore, it may be possible to determine whether to apply diagonal division to a coding block by taking into consideration two or more of the above listed conditions.

[0176] As another example, information indicating whether diagonal partitioning is applied to a coding block may be transmitted by a signal via the stream. The information may be transmitted by a signal at the sequence level, picture level, slice level, or block level. For example, at the coding block level, a flag triangle_partition_flag may be transmitted by a signal indicating whether triangular partitioning is applied to the coding block.

[0177] When it is decided to apply a diagonal division to a coding block, information indicating the number of lines or the position of the lines dividing the coding block may be transmitted by a signal over the stream.

[0178] For example, if a coding block is divided by a diagonal, information indicating the direction of the diagonal dividing the coding block may be transmitted by a signal via the stream. For example, a flag triangle_partition_type_flag indicating the direction of the diagonal may be transmitted by a signal via the stream. The flag indicates whether the coding block is divided by a diagonal connecting the upper left and lower right, or by a diagonal connecting the upper right and lower left. Dividing a coding block by a diagonal connecting the upper left and lower right is called a left triangular partition type, while dividing a coding block by a diagonal connecting the upper right and lower left is called a right triangular partition type. For example, a value of 0 for the flag indicates that the division type of the coding block is a left triangular partition type, and a value of 1 for the flag indicates that the division type of the coding block is a right triangular partition type.

[0179] Furthermore, information indicating whether the prediction units have the same size or information indicating the position of a diagonal for dividing the coding block may be transmitted by a signal via the stream. For example, if the information indicating the size of the prediction units indicates that the sizes of the prediction units are the same, encoding of the information indicating the diagonal position may be omitted, and the coding block may be divided into two prediction units using a diagonal passing through two vertices of the coding block. On the other hand, if the information indicating the size of the prediction units indicates that the sizes of the prediction units are different, the position of the diagonal for dividing the coding block may be determined based on the information indicating the position of the diagonal. For example, if a left triangulation type is applied to a coding block, the position information may indicate whether the diagonal touches the left and bottom boundaries or the top and right boundaries of the coding block. Alternatively, if a right triangulation type is applied to a coding block, the position information may indicate whether the diagonal touches the right and bottom boundaries or the top and left boundaries of the coding block.

[0180] Information indicating the partition type of a coding block may be signaled at the coding block level, which may determine the partition type for different coding blocks to which diagonal partitioning is applied.

[0181] As another example, information indicating the partition type may be transmitted by a signal for a sequence, a picture, a slice, a segment, or a coding tree unit. In such a case, the partition type of coding blocks to which diagonal partitioning is applied may be set to be the same for the sequence, the picture, the slice, the segment, or the coding tree unit.

[0182] Alternatively, for the first coding unit that applies diagonal partitioning in the coding tree unit, information for determining the partitioning type is coded and signaled, and the second and subsequent coding units that apply diagonal partitioning are configured to use the same partitioning type as the first coding unit.

[0183] As another example, the division type of the coding block may be determined based on the division type of neighboring blocks. The neighboring blocks may include at least one of the neighboring block adjacent to the upper left corner of the coding block, the neighboring block adjacent to the upper right corner, the neighboring block adjacent to the lower left corner, the neighboring block located above, or the neighboring block located to the left. For example, the division type of the current block may be set to be the same as the division type of the neighboring block. Alternatively, the division type of the current block may be determined based on whether the upper left neighboring block applies a left triangulation type and whether the upper right neighboring block or the lower left neighboring block applies a right triangulation type.

[0184] The size of the prediction unit may be set to be equal to or smaller than the size of the transform unit, and may be set such that one transform unit must include one prediction unit. Thus, when applying VPDU transform unit partitioning to a coding block, diagonal partitioning may be applied to the transform unit instead of applying diagonal partitioning to the coding block.

[0185] FIG. 15 is a diagram showing an example in which diagonal division is applied to a transform unit.

[0186] In the example shown in Figure 15, when a coding block is divided into multiple transform units, whether to apply diagonal division may be determined for each transform unit. Alternatively, whether to apply diagonal division may be determined for each block whose size is smaller than the size of the transform unit.

[0187] Alternatively, when applying VPDU transform unit splitting to a coding block, it may determine whether to apply diagonal splitting to the coding block, and all transform units included in the coding block may be set to comply with the determination. For example, when it is determined to apply diagonal splitting to the coding block, all transform units included in the coding block may be split into two prediction units.

[0188] To perform motion prediction compensation on the first triangular prediction unit and the second triangular prediction unit, motion information for each of the first triangular prediction unit and the second triangular prediction unit may be derived. In this case, the motion information for the first triangular prediction unit and the second triangular prediction unit may be derived from merge candidates included in a merge candidate list. To distinguish between a general merge candidate list and a merge candidate list used to derive motion information for triangular prediction units, a merge candidate list for deriving motion information for triangular prediction units is referred to as a triangle merge candidate list, and merge candidates included in the triangle merge candidate list are referred to as triangle merge candidates. However, the present invention also encompasses the use of the above merge candidate derivation method and merge candidate list construction method for triangle merge candidates and triangle merge candidate list construction methods.

[0189] The signal may transmit information over the stream to determine the maximum number of triangle merge candidates that can be included in the triangle merge candidate list, and the information may indicate a difference between the maximum number of merge candidates that can be included in the merge candidate list and the maximum number of triangle merge candidates that can be included in the triangle merge candidate list.

[0190] Triangle merging candidates may be derived from the spatial and temporal neighboring blocks of a coding block.

[0191] FIG. 16 is a diagram showing adjacent blocks for deriving triangle merge candidates.

[0192] Triangle merging candidates may be derived using at least one of a neighboring block located above the coding block, a neighboring block located to the left of the coding block, or a co-located block included in a different picture from the coding block. The upper neighboring block may include at least one of a block including samples (xCb+CbW-1, yCb-1) located above the coding block, a block including samples (xCb+CbW, yCb-1) located above the coding block, or a block including samples (xCb-1, yCb-1) located above the coding block. The left neighboring block may include at least one of a block including samples (xCb-1, yCb+CbH-1) located to the left of the coding block, or a block including samples (xCb-1, yCb+CbH) located to the left of the coding block. The collocated block may be determined as either the block containing the sample (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coding block in the collocated picture, or the block containing the sample (xCb / 2, yCb / 2) located at the center of the coding block.

[0193] The neighboring blocks may be searched in a predefined order, and the triangle merge candidates may be constructed in a triangle merge candidate list in a predefined order, for example, the triangle merge candidates may be searched in the order of B1, A1, B0, A0, C0, B2, and C1 to construct the triangle merge candidate list.

[0194] The motion information of the triangle prediction units may be derived based on the triangle merge candidate list, i.e., the triangle prediction units may share one triangle merge candidate list.

[0195] To derive the motion information of the triangle merging unit, information for specifying at least one of the triangle merge candidates included in the triangle merge candidate list may be transmitted by a signal via the stream, for example, index information merge_triangle_idx for specifying at least one of the triangle merge candidates may be transmitted by a signal via the stream.

[0196] The index information may specify a combination of merge candidates for the first triangular prediction unit and the second triangular prediction unit. For example, Table 1 below shows an example of a combination of merge candidates based on the index information merge_triangle_idx.

[0197] [Table 1]

[0198] When the value of the index information merge_triangle_idx is 1, it indicates that the motion information of the first triangular prediction unit is derived from the merge candidate with index 1, and the motion information of the second triangular prediction unit is derived from the merge candidate with index 0. The index information merge_triangle_idx may determine the triangle merge candidate for deriving the motion information of the first triangular prediction unit and the triangle merge candidate for deriving the motion information of the second triangular prediction unit.

[0199] Furthermore, the partition type of the coding block to which diagonal partitioning is applied may be determined based on index information. That is, the index information may specify a combination of merge candidates for the first triangular prediction unit, merge candidates for the second triangular prediction unit, and the partition direction of the coding block. When the partition type of the coding block is determined based on index information, it is not necessary to encode information triangle_partition_type_flag indicating the direction of the diagonal line that divides the coding block. Table 2 shows the partition types of the coding block with respect to index information merge_triangle_idx.

[0200] [Table 2]

[0201] When the variable TriangleDir is 0, it indicates that the coding block applies a left triangulation type, and when the variable TriangleDir is 1, it indicates that the coding block applies a right triangulation type. By combining Table 1 and Table 2, it is possible to specify a combination of merge candidates for the first triangular prediction unit, merge candidates for the second triangular prediction unit, and the division direction of the coding block based on the index information merge_triangle_idx.

[0202] As another example, index information used for only one of the first triangular prediction unit and the second triangular prediction unit may be transmitted via a signal, and an index of a triangle merge candidate used for the other of the first triangular prediction unit and the second triangular prediction unit may be determined based on the index information. For example, a triangle merge candidate for the first triangular prediction unit may be determined based on index information merge_triangle_idx indicating an index of one of the triangle merge candidates. Furthermore, a triangle merge candidate for the second triangular prediction unit may be specified based on the merge_triangle_idx. For example, a triangle merge candidate for the second triangular prediction unit may be derived by adding or subtracting an offset amount to or from the index information merge_triangle_idx. The offset amount may be an integer such as 1 or 2. For example, a triangle merge candidate for the second triangular prediction unit may be determined as a triangle merge candidate indexed by adding 1 to merge_triangle_idx. If merge_triangle_idx indicates the triangle merge candidate with the largest index value among the triangle merge candidates, motion information of the second triangular prediction unit may be derived from the triangle merge candidate with index 0 or the triangle merge candidate with an index obtained by subtracting 1 from merge_triangle_idx.

[0203] Alternatively, the motion information of the second triangular prediction unit may be derived from a triangle merge candidate having the same reference picture as the triangle merge candidate of the first triangular prediction unit specified based on the index information. The triangle merge candidate having the same reference picture as the triangle merge candidate of the first triangular prediction unit may indicate at least one of an L0 reference picture or an L1 reference picture having the same reference picture as the triangle merge candidate of the first triangular prediction unit. When there are multiple triangle merge candidates having the same reference picture as the triangle merge candidate of the first triangular prediction unit, one of them may be selected based on at least one of whether the merge candidate includes bidirectional motion information or a difference between the index of the merge candidate and the index information.

[0204] As another example, index information may be transmitted by a signal to each of the first triangular prediction unit and the second triangular prediction unit. For example, first index information 1st_merge_idx for determining triangle merge candidates for the first triangular prediction unit and second index information 2nd_merge_idx for determining triangle merge candidates for the second triangular prediction unit may be transmitted by a signal via a stream. Motion information for the first triangular prediction unit may be derived from the triangle merge candidates determined based on the first index information 1st_merge_idx, and motion information for the second triangular prediction unit may be derived from the triangle merge candidates determined based on the second index information 2nd_merge_idx.

[0205] The first index information 1st_merge_idx may indicate the index of any one of the triangle merge candidates included in the triangle merge candidate list, and the triangle merge candidate of the first triangular prediction unit may be determined as the triangle merge candidate indicated by the first index information 1st_merge_idx.

[0206] The triangle merge candidate indicated by the first index information 1st_merge_idx is set to be unavailable as a triangle merge candidate for the second triangular prediction unit. Thus, the second index information 2nd_merge_idx of the second triangular prediction unit may indicate the index of any one of the remaining triangle merge candidates other than the triangle merge candidate indicated by the first index information. If the value of the second index information 2nd_merge_idx is smaller than the value of the first index information 1st_merge_idx, the triangle merge candidate for the second triangular prediction unit may be determined as the triangle merge candidate having index information indicated by the second index information 2nd_merge_idx. On the other hand, if the value of the second index information 2nd_merge_idx is the same as or greater than the value of the first index information 1st_merge_idx, the triangle merge candidate for the second triangular prediction unit may be determined as the triangle merge candidate indexed by adding 1 to the value of the second index information 2nd_merge_idx.

[0207] Alternatively, whether to transmit the second index information via a signal may be determined based on the number of triangle merge candidates included in the triangle merge candidate list. For example, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is two or less, transmitting the second index information via a signal may be omitted. If transmitting the second index information via a signal is omitted, the second triangle merge candidate may be derived by adding or subtracting an offset amount to or from the first index information. For example, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is two and the first index information is an index of zero, the second triangle merge candidate may be derived by adding one to the first index information. Alternatively, if the maximum number of triangle merge candidates that can be included in the triangle merge candidate list is two and the first index information is one, the second triangle merge candidate may be derived by subtracting one from the first index information.

[0208] Alternatively, if the second index information is not transmitted by the signal, the second index information may be set as a default value, which may be 0. The second triangle merge candidate can be derived by comparing the first index information with the second index information. For example, if the second index information is smaller than the first index information, the merge candidate with index 0 is set as the second triangle merge candidate; if the second index information is equal to or larger than the first index information, the merge candidate with index 1 is set as the second triangle merge candidate.

[0209] If a triangle merge candidate has unidirectional motion information, the unidirectional motion information of the triangle merge candidate is set as the motion information of the triangle prediction unit. On the other hand, if a triangle merge candidate has bidirectional motion information, only one of the L0 motion information or the L1 motion information is set as the motion information of the triangle prediction unit. Whether the L0 motion information or the L1 motion information is to be acquired may be determined based on the index of the triangle merge candidate or the motion information of other triangle prediction units.

[0210] For example, if the index of a triangle merge candidate is even, the L0 motion information of the triangle prediction unit is set to 0, and the L1 motion information of the triangle merge candidate is set to the L1 motion information of the triangle prediction unit. On the other hand, if the index of a triangle merge candidate is odd, the L1 motion information of the triangle prediction unit is set to 0, and the L0 motion information of the triangle merge candidate is set to 0. Conversely, if the index of a triangle merge candidate is even, the L0 motion information of the triangle merge candidate may be set to the L0 motion information of the triangle prediction unit, and if the index of a triangle merge candidate is odd, the L1 motion information of the triangle merge candidate may also be set to the L1 motion information of the triangle prediction unit. Alternatively, for a first triangle prediction unit, if the triangle merge candidate is even, the L0 motion information of the triangle merge candidate is set to the L0 motion information of the first triangle prediction unit, while for a second triangle prediction unit, if the triangle merge candidate is odd, the L1 motion information of the triangle merge candidate is set to the L1 motion information of the second triangle prediction unit.

[0211] Alternatively, if the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit is set to 0, and the L1 motion information of the triangle merge candidate is set to the L1 information of the second triangle prediction unit. On the other hand, if the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit is set to 0, and the L0 motion information of the triangle merge candidate is set to the L0 motion signal of the second triangle prediction unit.

[0212] Furthermore, the triangle merge candidate list for deriving motion information of the first triangular prediction unit and the triangle merge candidate list for deriving motion information of the second triangular prediction unit may be set to be different.

[0213] For example, when a triangle merge candidate for deriving motion information of the first triangular prediction unit is specified in a triangle merge candidate list based on index information related to the first triangular prediction unit, the motion information of the second triangular prediction unit may be derived using a triangle merge list including remaining triangle merge candidates other than the triangle merge candidate indicated by the index information. Specifically, the motion information of the second triangular prediction unit may be derived from any one of the remaining triangle merge candidates.

[0214] As a result, the maximum number of triangle merge candidates included in the triangle merge candidate list of the first triangular prediction unit is different from the maximum number of triangle merge candidates included in the triangle merge candidate list of the second triangular prediction unit. For example, if the triangle merge candidate list of the first triangular prediction unit includes M merge candidates, the triangle merge candidate list of the second triangular prediction unit may include M-1 merge candidates other than the triangle merge candidate indicated by the index information of the first triangular prediction unit.

[0215] As another example, merging candidates for each triangular prediction unit may be derived based on neighboring blocks adjacent to the coding block, and the availability of neighboring blocks may be determined taking into account the shape or position of the triangular prediction unit.

[0216] FIG. 17 is a diagram illustrating an example of determining the availability of neighboring blocks for each triangular prediction unit.

[0217] Neighboring blocks that are not adjacent to the first triangular prediction unit may be set as unavailable neighboring blocks for the first triangular prediction unit, and neighboring blocks that are not adjacent to the second triangular prediction unit may be set as unavailable neighboring blocks for the second triangular prediction unit.

[0218] 17(a), when the left triangulation type is applied to the coding block, it can be determined that blocks A1, A0, and A2 adjacent to the first triangular prediction unit among the adjacent blocks adjacent to the coding block can be used for the first triangular prediction unit, but blocks B0 and B1 cannot be used for the first triangular prediction unit. As a result, the triangle merge candidate list associated with the first triangular prediction unit includes triangle merge candidates derived from blocks A1, A0, and A2, but does not include triangle merge candidates derived from blocks B0 and B1.

[0219] 17(b), when applying the left triangulation type to the coding block, it can be determined that blocks B0 and B1 adjacent to the second triangular prediction unit can be used for the second triangular prediction unit, but blocks A1, A0, and A2 cannot be used for the second triangular prediction unit. As a result, the triangle merge candidate list associated with the second triangular prediction unit includes triangle merge candidates derived from blocks B0 and B1, but does not include triangle merge candidates derived from blocks A1, A0, and A2.

[0220] Thereby, the number or range of triangle merging candidates that the triangular prediction unit can use may be determined based on at least one of the position of the triangular prediction unit or the division type of the coding block.

[0221] As another example, a merge mode may be applied to only one of the first triangular prediction unit and the second triangular prediction unit, and the motion information of the other of the first triangular prediction unit and the second triangular prediction unit may be set to be the same as the motion information of the triangular prediction unit to which the merge mode is applied, or may be derived by subdividing the motion information of the triangular prediction unit to which the merge mode is applied.

[0222] For example, the motion vector and reference picture index of the first triangular prediction unit may be derived based on the triangle merge candidate, or the motion vector of the second triangular prediction unit may be derived by subdividing the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding the refined motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit or by subtracting the refined motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0223] The information for determining a fine motion vector indicating a difference between the motion vector of the first triangular prediction unit and the motion vector of the second triangular prediction unit may be transmitted by a signal via the stream, and the information may include at least one of information indicating a size of the fine motion vector or information indicating a symbol of the fine motion vector.

[0224] Alternatively, the symbols of the fine motion vector may be derived based on at least one of the position, index, or partition type applied to the coding block of the triangular prediction unit.

[0225] As another example, a motion vector and a reference picture index for one of the first triangular prediction unit and the second triangular prediction unit may be transmitted by a signal, and the motion vector for the other of the first triangular prediction unit and the second triangular prediction unit may be derived by subdividing the transmitted motion vector.

[0226] For example, the motion vector and reference picture index of the first triangular prediction unit may be determined based on information signaled from the stream. The motion vector of the second triangular prediction unit may be derived by subdividing the motion vector of the first triangular prediction unit. For example, the motion vector of the second triangular prediction unit may be derived by adding the refined motion vector {Rx, Ry} to the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit or by subtracting the refined motion vector {Rx, Ry} from the motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. The reference picture index of the second triangular prediction unit may be set to be the same as the reference picture index of the first triangular prediction unit.

[0227] Motion prediction and compensation prediction may be performed on the coding blocks based on the motion information of the first triangular prediction unit and the motion information of the second triangular prediction unit. In this case, image quality may be degraded at the boundary between the first triangular prediction unit and the second triangular prediction unit. For example, image quality may be continuously degraded near the edge on the boundary between the first triangular prediction unit and the second triangular prediction unit. To reduce image quality degradation at the boundary, prediction samples may be derived using smoothing filtering or weighted prediction.

[0228] Prediction samples for diagonal division of a coding block may be derived by performing a weighted sum operation on first prediction samples obtained based on motion information of the first triangular prediction unit and second prediction samples obtained based on motion information of the second triangular prediction unit. Alternatively, prediction samples for the first triangular prediction unit may be derived from a first prediction block determined based on the motion information of the first triangular prediction unit, prediction samples for the second triangular prediction unit may be derived from a second prediction block determined based on the motion information of the second triangular prediction unit, and prediction samples located in a boundary area between the first triangular prediction unit and the second triangular prediction unit may be derived by performing a weighted sum operation on the first prediction samples included in the first prediction block and the second prediction samples included in the second prediction block. For example, Equation 2 below shows an example of deriving prediction samples for the first triangular prediction unit and the second triangular prediction unit.

[0229] equation 2 JPEG2025183407000005.jpg572

[0230] In Equation 2, P1 denotes the first predicted sample, P2 denotes the second predicted sample, w1 denotes the weighting value applied to the first predicted sample, and (1-w1) denotes the weighting value applied to the second predicted sample. In the example shown in Equation 2, the weighting value applied to the second predicted sample can be derived by subtracting the weighting value applied to the first predicted sample from a constant.

[0231] When a left triangulation type is applied to a coding block, the boundary region may include predicted samples having the same x-axis and y-axis coordinates, whereas when a right triangulation type is applied to a coding block, the boundary region may include predicted samples whose sum of the x-axis and y-axis coordinates is greater than or equal to a first threshold and less than or equal to a second threshold.

[0232] The size of the boundary area may be determined based on at least one of the size of the coding block, the shape of the coding block, the motion information of the triangular prediction unit, the motion vector difference of the triangular prediction unit, the output order of the reference picture, or the difference between the first predicted sample and the second predicted sample at the diagonal boundary.

[0233] Figures 18 and 19 show examples of deriving a predicted sample by performing a weighted sum operation on a first predicted sample and a second predicted sample. Figure 18 shows an example in which a left triangular decomposition type is applied to a coding block, and Figure 19 shows an example in which a right triangular decomposition type is applied to a coding block. Figures 18(a) and 19(a) show prediction patterns for the luma component, and Figures 18(b) and 19(b) show prediction patterns for the chroma component.

[0234] In the drawing, a number written in a prediction sample near the boundary between the first prediction unit and the second prediction unit indicates a weighting value applied to the first prediction sample. For example, if the number written in a prediction sample is N, a weighting value of N / 8 is applied to the first prediction sample, and a weighting value of (1-(N / 8)) is applied to the second prediction sample, thereby deriving the prediction sample.

[0235] In a non-boundary region, the first prediction sample or the second prediction sample may be determined as the prediction sample. In the example of Fig. 18, in a region belonging to a first triangular prediction unit among regions where the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, the first prediction sample derived based on the motion information of the first triangular prediction unit may be determined as the prediction sample. On the other hand, in a region belonging to a second triangular prediction unit among regions where the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, the second prediction sample derived based on the motion information of the second triangular prediction unit may be determined as the prediction sample.

[0236] 19, in a region where the sum of the x-axis coordinate and the y-axis coordinate is smaller than a first threshold, a first prediction sample derived based on the motion information of the first triangular prediction unit may be determined as the prediction sample, while in a region where the sum of the x-axis coordinate and the y-axis coordinate is larger than a second threshold, a second prediction sample derived based on the motion information of the second triangular prediction unit may be determined as the prediction sample.

[0237] The threshold for determining the non-border region may be determined based on at least one of the size of the coding block, the shape of the coding block, or the color component. For example, if the threshold associated with the luma component is set as N, the threshold associated with the chroma component may be set as N / 2.

[0238] The predicted sample included in the boundary region may be derived by performing a weighted sum operation on the first predicted sample and the second predicted sample. In such a case, weighting values ​​to be applied to the first predicted sample and the second predicted sample may be determined based on at least one of the position of the predicted sample, the size of the coding block, the shape of the coding block, or the color component.

[0239] 18(a), by applying the same weighting value to the first predicted sample and the second predicted sample, predicted samples at positions having the same x-axis coordinate and y-axis coordinate can be derived. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample to (3:1) or (1:3), predicted samples whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived. Furthermore, by setting the weighting value ratio applied to the first predicted sample and the second predicted sample to (7:1) or (1:7), predicted samples whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 2 can be derived.

[0240] Alternatively, in the example shown in Figure 18(b), by applying the same weighting value to the first predicted sample and the second predicted sample, predicted samples at positions having the same x-axis coordinate and y-axis coordinate can be derived, and by setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), a predicted sample whose absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived.

[0241] 19(a), by applying the same weighting value to the first predicted sample and the second predicted sample, it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample to (3:1) or (1:3), it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as the width or height of the coding block or 2 less than the width or height of the coding block. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample to (7:1) or (1:7), it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 more than the width or height of the coding block or 3 less than the width or height of the coding block.

[0242] 19(b), by applying the same weighting value to the first predicted sample and the second predicted sample, it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is smaller than the width or height of the coding block by 1. By setting the weighting value ratio applied to the first predicted sample and the second predicted sample as (7:1) or (1:7), it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is equal to the width or height of the coding block or smaller by 2.

[0243] As another example, the weighting values ​​may be determined taking into account the position of the predicted sample or the shape of the coding block. Equations 3 to 5 show examples of deriving weighting values ​​when a left triangulation type is applied to the coding block. Equation 3 shows an example of deriving a weighting value to be applied to the first predicted sample when the coding block is square.

[0244] equation 3 JPEG2025183407000006.jpg530

[0245] In Equation 3, x and y indicate the position of the predicted sample. If the coding block is non-square, the weighting value to be applied to the first predicted sample may be derived as shown in Equation 4 or 5. Equation 4 illustrates the case where the width of the coding block is greater than the height, and Equation 5 illustrates the case where the width of the coding block is less than the height.

[0246] Equation 4 JPEG2025183407000007.jpg550

[0247] equation 5 JPEG2025183407000008.jpg552

[0248] When a right triangulation type is applied to a coding block, the weighting values ​​to be applied to the first predicted samples may be determined as shown in Equations 6 to 8. Equation 6 shows an example of deriving the weighting values ​​to be applied to the first predicted samples when the coding block is square.

[0249] equation 6 JPEG2025183407000009.jpg547

[0250] In Equation 6, CbW denotes the width of the coding block. If the coding block is non-square, the weighting value to be applied to the first predicted sample may be derived as shown in Equation 7 or 8. Equation 7 illustrates the case where the width of the coding block is greater than the height, and Equation 8 illustrates the case where the width of the coding block is less than the height.

[0251] Equation 7 JPEG2025183407000010.jpg566

[0252] equation 8 JPEG2025183407000011.jpg567

[0253] In equation 7, CbH denotes the height of the coding block.

[0254] In the illustrated example, for prediction samples in the boundary region, by assigning a higher weighting value to the first prediction sample than to the second prediction sample, the samples included in the first triangular prediction unit can be derived, and by assigning a higher weighting value to the second prediction sample than to the first prediction sample, the samples included in the second triangular prediction unit can be derived.

[0255] When diagonal division is applied to a coding block, the coding block may be set not to apply a combined prediction mode that combines an intra prediction mode and a merge mode.

[0256] Intra prediction is to predict the current block using reconstructed samples after encoding / decoding of the surrounding blocks of the current block. In this case, the intra prediction of the current block may use reconstructed samples before applying the in-loop filter.

[0257] Intra prediction techniques include matrix-based intra prediction and general intra prediction that takes into account the directionality of surrounding reconstructed samples. Information indicating the intra prediction technique for the current block may be transmitted via a signal through the stream. The information may be a one-bit flag. Alternatively, the intra prediction technique for the current block may be determined based on at least one of the position, size, and shape of the current block or the intra prediction techniques of neighboring blocks. For example, if the current block straddles a picture boundary, the current block is set to not apply matrix-based intra prediction.

[0258] Matrix-based intra prediction is a method of obtaining a prediction block for a current block by matrix multiplication of a matrix stored in an encoder and a decoder with reconstructed samples surrounding the current block. Information for specifying one of a plurality of stored matrices may be transmitted via a signal through the stream. The decoder may determine a matrix for intra prediction of the current block based on the information and the size of the current block.

[0259] General intra prediction is a method of obtaining a predicted block related to a current block based on a non-angular intra prediction mode or an angular intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in more detail with reference to the accompanying drawings.

[0260] FIG. 20 is a flowchart illustrating an intra prediction method according to an embodiment of the present invention.

[0261] A reference sample line of the current block may be determined (S2001). The reference sample line refers to a set of reference samples included in the K-th row from the top and / or left of the current block. The reference sample may be derived from reconstructed samples after encoding / decoding around the current block.

[0262] Index information indicating a reference sample line of a current block among a plurality of reference sample lines may be transmitted by a signal via a stream. For example, index information intra_luma_ref_idx for specifying a reference sample line of the current block may be transmitted by a signal via a stream. The index information may be transmitted by a signal in units of coding blocks.

[0263] The plurality of reference sample lines may include at least one of a first line, a second line, a third line, and a fourth line located above and / or to the left of the current block. Of the plurality of reference sample lines, a reference sample line consisting of a row adjacent to the above of the current block and a column adjacent to the left of the current block may be referred to as an adjacent reference sample line, and the other reference sample lines may be further referred to as non-adjacent reference sample lines.

[0264] Some of the plurality of reference sample lines may be selected as the reference sample line of the current block. For example, the remaining reference sample lines other than the third non-adjacent reference sample line among the plurality of reference sample lines may be set as candidate reference sample lines. Table 3 shows the indexes assigned to each candidate reference sample line.

[0265] [Table 3]

[0266] More or fewer candidate reference sample lines may be set than the described sample lines. Furthermore, the number or positions of non-adjacent reference sample lines set as candidate reference sample lines is not limited to the described examples. For example, the first and third non-adjacent reference sample lines may be set as candidate reference sample lines, or the second and third non-adjacent reference sample lines may be set as candidate reference sample lines. Alternatively, all of the first, second, and third non-adjacent reference sample lines may be set as candidate reference sample lines.

[0267] The number or type of candidate reference sample lines may be determined based on at least one of the size, shape, and position of the current block, whether sub-blocks are split, or the intra-prediction mode.

[0268] Furthermore, the reference sample line of the current block may be determined based on at least one of the position, size, shape, or predictive coding mode of the neighboring block of the current block. For example, if the current block contacts the boundary of a picture, a segment, a slice, or a coding tree unit, the first reference sample line may be determined as the reference sample line of the current block.

[0269] Alternatively, if the current block is non-square, the adjacent reference sample line may be set as the reference sample line of the current block. Alternatively, if the width-to-height ratio of the current block is greater than or equal to a threshold value or less than or equal to a threshold value, the adjacent reference sample line may be determined as the reference sample line of the current block.

[0270] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper reference sample and the left reference sample may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before an in-loop filter is applied.

[0271] Next, an intra prediction mode of the current block may be determined (S2002). At least one of a non-angular intra prediction mode or an angular intra prediction mode may be determined as the intra prediction mode of the current block. The non-angular intra prediction modes include planar and DC, and the angular intra prediction modes include 33 or 65 modes from the lower-left diagonal to the upper-right diagonal.

[0272] FIG. 21 is a diagram showing intra prediction modes.

[0273] FIG. 21(a) shows 35 intra prediction modes, and FIG. 21(b) shows 67 intra prediction modes.

[0274] More or fewer intra-prediction modes than those shown in FIG. 21 may be defined.

[0275] A Most Probable Mode (MPM) may be set based on the intra-prediction modes of neighboring blocks adjacent to the current block. The neighboring blocks may include a left neighboring block adjacent to the left of the current block and an upper neighboring block adjacent above the current block.

[0276] The number of MPMs included in the MPM list may be predefined in the encoder and decoder. For example, the number of MPMs may be three, four, five, or six. Alternatively, information indicating the number of MPMs may be transmitted by a signal via the stream. Alternatively, the number of MPMs may be determined based on at least one of the predictive coding mode of neighboring blocks and the size or shape of the current block. For example, when neighboring reference sample lines are determined as reference sample lines for the current block, N MPMs may be used, whereas when non-neighboring reference sample lines are determined as reference sample lines for the current block, M MPMs may be used. M is an integer smaller than N, e.g., N may be 6, or M may be 5, 4, or 3. Thus, when the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra prediction mode of the current block may be determined as one of six candidate intra prediction modes. However, when the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra prediction mode of the current block may be determined as one of five candidate intra prediction modes.

[0277] Alternatively, a fixed number of MPM candidates (eg, six or five) may be used regardless of the index of the reference sample line of the current block.

[0278] An MPM list including multiple MPMs may be generated, and information indicating whether the MPM list includes the same MPM as the intra prediction mode of the current block may be transmitted via a signal through a stream. The information may be a 1-bit flag and may be referred to as an MPM flag. If the MPM flag indicates that the same MPM as the current block is included in the MPM list, index information indicating one of the MPMs may be transmitted via a signal through a stream. For example, index information mpm_idx specifying one of the multiple MPMs may be transmitted via a signal through a stream. The MPM specified by the index information may be set as the intra prediction mode of the current block. If the MPM flag indicates that the same MPM as the current block is not included in the MPM list, remaining mode information indicating one of the remaining intra prediction modes other than the MPM may be transmitted via a signal through a stream. The remaining mode information indicates an index value corresponding to the intra prediction mode of the current block when reassigning indexes to the remaining intra prediction modes other than the MPM. The decoder may determine the intra prediction mode of the current block by arranging the MPMs in ascending order and comparing the remaining mode information with the MPM. For example, if the residual mode information is equal to or smaller than the MPM, the intra prediction mode of the current block may be derived by adding 1 to the residual mode information.

[0279] When deriving the intra prediction mode of the current block, comparison of some of the MPMs with the remaining mode information may be omitted. For example, the MPM of the non-angular intra prediction mode may be excluded from the comparison object. When a non-angular intra prediction mode is set as the MPM, the remaining mode information clearly indicates the angular intra prediction mode, so the intra prediction mode of the current block may be derived by comparing the remaining MPMs other than the non-angular intra prediction mode with the remaining mode information. Instead of excluding the non-angular intra prediction mode from the comparison object, the MPM may be compared with a value obtained by adding the number of non-angular intra prediction modes to the remaining mode information.

[0280] Instead of setting the default mode as an MPM, information indicating whether the intra prediction mode of the current block is the default mode may be transmitted by a signal via the stream. The information may be a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be transmitted by a signal only if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As described above, the default mode may include at least one of planar, DC, vertical mode, or horizontal mode. For example, if planar is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is planar. If the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

[0281] When the default mode flag is used, the intra prediction mode that is the same as the default mode does not need to be set as the MPM. For example, if the default mode flag indicates whether the intra prediction mode of the current block is planar, the intra prediction mode of the current block may be derived using five MPMs other than the MPM corresponding to planar.

[0282] When multiple intra prediction modes are set as default modes, the signal may further transmit index information indicating one of the default modes, and the intra prediction mode of the current block may be set as the default mode indicated by the index information.

[0283] If the index of the reference sample line of the current block is not 0, the default mode is set not to be used. For example, if a non-adjacent reference sample line is determined as the reference sample line of the current block, a non-angular intra prediction mode such as DC mode or planar mode may be set not to be used. Thus, if the index of the reference sample line is not 0, the default mode flag may not be transmitted by a signal, and the value of the default mode flag may be set to a predefined value (i.e., false).

[0284] When the intra prediction mode of the current block is determined, a prediction sample of the current block may be obtained based on the determined intra prediction mode (S2003).

[0285] When the DC mode is selected, a predicted sample related to the current block may be generated based on an average value of reference samples. Specifically, the values ​​of all samples in the predicted block may be generated based on the average value of the reference samples. The average value may be derived using at least one of an upper reference sample located above the current block and a left reference sample located to the left of the current block.

[0286] The number or range of reference samples used to derive the average value varies depending on the shape of the current block. For example, if the current block is a non-square block whose width is greater than its height, the average value may be calculated using only the upper reference samples. On the other hand, if the current block is a non-square block whose width is less than its height, the average value may be calculated using only the left reference samples. That is, if the width and height of the current block are different, the average value may be calculated using only the reference samples adjacent to the longer side. Alternatively, whether to calculate the average value using only the upper reference samples or only the left reference samples may be determined based on the ratio of the width and height of the current block.

[0287] When the planar mode is selected, a prediction sample may be obtained using a horizontal prediction sample and a vertical prediction sample. The horizontal prediction sample is obtained based on a left reference sample and a right reference sample located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on an upper reference sample and a lower reference sample located on the same vertical line as the prediction sample. A right reference sample may be generated by copying a reference sample adjacent to the upper right corner of the current block, and a lower reference sample may be generated by copying a reference sample adjacent to the lower left corner of the current block. The horizontal prediction sample may be obtained by a weighted sum operation of the left reference sample and the right reference sample, and the vertical prediction sample may be obtained by a weighted sum operation of the upper reference sample and the lower reference sample. In this case, a weighting value to be assigned to each reference sample may be determined based on the position of the prediction sample. The prediction sample may be obtained by averaging or weighting the horizontal prediction sample and the vertical prediction sample. When performing the weighting sum operation, a weighting value to be assigned to the horizontal prediction sample and the vertical prediction sample may be determined based on the position of the prediction sample.

[0288] When an angular prediction mode is selected, a parameter indicating the prediction direction (or prediction angle) of the selected angular prediction mode may be determined. Table 4 below shows the intra prediction parameter intraPredAng for each intra prediction mode.

[0289] [Table 4]

[0290] Table 4 shows the intra direction parameters of each intra prediction mode having an index of 2 to 34 when 35 intra prediction modes are defined. When 33 or more angular intra prediction modes are defined, Table 4 is further subdivided to set the intra direction parameters of each angular intra prediction mode.

[0291] After arranging the upper reference sample and the left reference sample of the current block in a row, a predicted sample may be obtained based on the value of the intra direction parameter. In this case, if the value of the intra direction parameter is negative, the left reference sample and the upper reference sample may be arranged in a row.

[0292] 22 and 23 are diagrams showing examples of a one-dimensional array in which reference samples are arranged in one row.

[0293] Figure 22 shows an example of a one-dimensional vertical array in which reference samples are arranged in the vertical direction, and Figure 23 shows an example of a one-dimensional horizontal array in which reference samples are arranged in the horizontal direction. The examples of Figures 22 and 23 will be described assuming that 35 intra prediction modes are defined.

[0294] If the intra prediction mode index is one of 11 to 18, a horizontal one-dimensional array in which the upper reference sample is rotated counterclockwise may be applied. If the intra prediction mode index is one of 19 to 25, a vertical one-dimensional array in which the left reference sample is rotated clockwise may be applied. When arranging the reference samples in a row, the intra prediction mode angle may be taken into consideration.

[0295] Reference sample decision parameters may be determined based on the intra direction parameters, and the reference sample decision parameters may include a reference sample index for specifying the reference sample and a weighting value parameter for determining a weighting value to be applied to the reference sample.

[0296] The reference sample index iIdx and the weighting parameter ifact may be obtained by the following Equations 9 and 10, respectively.

[0297] equation 9 JPEG2025183407000014.jpg639

[0298] Equation 10 JPEG2025183407000015.jpg643

[0299] In equations 9 and 10, P ang indicates an intra direction parameter. The reference sample specified based on the reference sample index iIdx corresponds to an integer pel.

[0300] One or more reference samples may be specified to derive a prediction sample. Specifically, the positions of the reference samples used to derive the prediction sample may be specified in consideration of the gradient of the prediction mode. For example, a reference sample index iIdx may be used to specify the reference sample used to derive the prediction sample.

[0301] In such a case, if the gradient of the intra prediction mode cannot be represented by a single reference sample, a prediction sample may be generated by interpolating multiple reference samples. For example, if the gradient of the intra prediction mode is a value between the gradient between the prediction sample and a first reference sample and the gradient between the prediction sample and a second reference sample, the prediction sample may be obtained by interpolating the first reference sample and the second reference sample. That is, if an angular line according to the intra prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample may be obtained by interpolating reference samples adjacent to the left, right, top, or bottom of the position where the angular line passes.

[0302] Equation 11 below shows an example of obtaining a predicted sample based on a reference sample.

[0303] Equation 11 JPEG2025183407000016.jpg6138

[0304] In Equation 11, P denotes a predicted sample, and Ref_1D denotes any one of the reference samples in the one-dimensional array. In such a case, the position of the reference sample may be determined based on the position (x, y) of the predicted sample and the reference sample index iIdx.

[0305] If the gradient of the intra prediction mode may be represented as one reference sample, the weighting value parameter ifact may be set as 0. Therefore, Equation 11 may be simplified to the following Equation 12.

[0306] Equation 12 JPEG2025183407000017.jpg553

[0307] Furthermore, intra prediction may be performed on the current block based on multiple intra prediction modes. For example, intra prediction modes may be derived for different prediction samples, or prediction samples may be derived based on the intra prediction mode assigned to each prediction sample.

[0308] Alternatively, intra-prediction modes may be derived for different regions, and intra-prediction may be performed for each region based on the intra-prediction mode assigned to each region. The regions may include at least one sample. At least one of the sizes or shapes of the regions may be adaptively determined based on at least one of the size, shape, or intra-prediction mode of the current block. Alternatively, the encoder and decoder may predefine at least one of the sizes or shapes of the regions, regardless of the size or shape of the current block.

[0309] Alternatively, intra prediction may be performed based on multiple intra predictions, and a final prediction sample may be derived by averaging or weighted summing multiple prediction samples obtained by the multiple intra predictions. For example, a first prediction sample may be obtained by performing intra prediction based on a first intra prediction mode, and a second prediction sample may be obtained by performing intra prediction based on a second intra prediction mode. The final prediction sample may then be obtained by averaging or weighted summing the first and second prediction samples. In this case, the weights assigned to the first and second prediction samples may be determined based on at least one of whether the first intra prediction mode is a non-angular / angular prediction mode, whether the second intra prediction mode is a non-angular / angular prediction mode, or the intra prediction modes of neighboring blocks.

[0310] The multiple intra prediction modes may be a combination of a non-angular intra prediction mode and an angular prediction mode, a combination of angular prediction modes, or a combination of non-angular prediction modes.

[0311] FIG. 24 is a diagram showing angular intra prediction modes and angles formed by lines parallel to the x-axis.

[0312] In the example shown in Figure 24, the angular prediction modes may exist between the bottom left diagonal direction and the top right diagonal direction. When described as the angle between the x-axis and the angular prediction modes, the angular prediction modes may exist between 45 degrees (bottom left diagonal direction) and -135 degrees (top right diagonal direction).

[0313] If the current block is non-square, a situation may occur in which a prediction sample is derived using a reference sample that is far from the prediction sample among the reference samples located on a corner line according to the intra prediction angle, rather than a reference sample that is close to the prediction sample, based on the intra prediction mode of the current block.

[0314] FIG. 25 illustrates an example of obtaining predicted samples when the current block is non-square.

[0315] For example, in the example shown in Figure 25(a), it is assumed that the current block is a non-square block whose width is greater than its height, and that the intra prediction mode of the current block is an angular intra prediction mode whose angle is between 0 and 45 degrees. In this case, when deriving a predicted sample A near the right column of the current block, a situation occurs in which a left reference sample L far from the predicted sample among reference samples in the angular mode located at the angle is substituted for an upper reference sample T close to the predicted sample.

[0316] As another example, in the example shown in Figure 25(b), it is assumed that the current block is a non-square block with its height greater than its width, and the intra prediction mode of the current block is an angular intra prediction mode with an angle between -90 degrees and -135 degrees. In this case, when deriving a predicted sample A near the bottom row of the current block, a situation occurs in which an upper reference sample T far from the predicted sample among reference samples in the angular mode located at the angle is substituted for a left reference sample L close to the predicted sample.

[0317] To solve this problem, if the current block is non-square, the intra prediction mode of the current block may be replaced with an intra prediction mode in the opposite direction. Therefore, for a non-square block, an angular prediction mode having an angle larger or smaller than the angle of the angular prediction mode shown in FIG. 21 may be used. Such an angular intra prediction mode may be defined as a wide-angle intra prediction mode. A wide-angle intra prediction mode refers to an angular intra prediction mode that is not located in the range of 45 degrees to -135 degrees.

[0318] FIG. 26 is a diagram showing wide-angle intra prediction modes.

[0319] In the example shown in FIG. 26, the intra prediction modes with indexes −1 to −14 and the intra prediction modes with indexes 67 to 80 indicate wide-angle intra prediction modes.

[0320] Figure 26 shows 14 wide-angle intra prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra prediction modes (67 to 80) with angles less than -135 degrees, but a greater or lesser number of wide-angle intra prediction modes may be defined.

[0321] When using a wide-angle intra prediction mode, the length of the upper reference sample is set as 2W+1, and the length of the left reference sample is set as 2H+1.

[0322] When using wide-angle intra prediction mode, reference sample T may be used to predict sample A shown in Figure 25(a), and reference sample L may be used to predict sample A shown in Figure 25(b).

[0323] By adding the existing intra prediction modes and the N wide-angle intra prediction modes, a total of 67+N intra prediction modes can be used. For example, Table 5 shows the intra direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0324] [Table 5]

[0325] If the current block is non-square and the intra prediction mode of the current block obtained in step S2002 is within the conversion range, the intra prediction mode of the current block may be converted to a wide-angle intra prediction mode. The conversion range may be determined based on at least one of the size, shape, or ratio of the current block. The ratio may indicate the ratio of the width to the height of the current block.

[0326] If the current block is non-square, with its width greater than its height, the transform range may be set to the upper right diagonal intra prediction mode index (e.g., 66) to (upper right diagonal intra prediction mode index - N). N may be determined based on the ratio of the current block. If the intra prediction mode of the current block is within the transform range, the intra prediction mode may be converted to a wide-angle intra prediction mode. The transform may be performed by subtracting a predefined value from the intra prediction mode, and the predefined value may be the total number of intra prediction modes other than the wide-angle intra prediction mode (e.g., 67).

[0327] Based on the above embodiment, the 66th to 53rd intra prediction modes may be converted into the −1st to −14th wide-angle intra prediction modes, respectively.

[0328] If the current block is non-square, with its height greater than its width, the transform range may be set to the lower-left diagonal intra prediction mode index (e.g., 2) to (lower-left diagonal intra prediction mode index + M). M may be determined based on the ratio of the current block. If the intra prediction mode of the current block is within the transform range, the intra prediction mode may be converted to a wide-angle intra prediction mode. The transform may be performed by adding a predefined value to the intra prediction mode, and the predefined value may be the total number of angle intra prediction modes other than the wide-angle intra prediction mode (e.g., 65).

[0329] Based on the above embodiment, the 2nd to 15th intra prediction modes are converted into the 67th to 80th wide-angle intra prediction modes, respectively.

[0330] Hereinafter, the intra prediction modes within the transform range are referred to as wide-angle intra substitution prediction modes.

[0331] The transform range may be determined based on the ratio of the current block. For example, Tables 6 and 7 show transform ranges when 35 and 67 intra prediction modes other than the wide-angle intra prediction mode are defined, respectively.

[0332] [Table 6]

[0333] [Table 7]

[0334] In the examples shown in Tables 6 and 7, the number of wide-angle intra-substitution prediction modes within the transform range may vary depending on the proportion of the current block.

[0335] By subdividing the ratio of the current block, the conversion range can be set as shown in Table 8.

[0336] [Table 8]

[0337] When a multi-line intra prediction coding method for determining a non-adjacent reference sample line as a reference sample line of a current block or selecting one of a plurality of reference sample lines is used, it may be set not to use a wide-angle intra prediction mode. That is, even if the current block is non-square and the intra prediction mode of the current block is within the conversion range, the intra prediction mode of the current block may not be converted to the wide-angle intra prediction mode.

[0338] Alternatively, when the intra prediction mode of the current block is determined as a wide-angle intra prediction mode, non-adjacent reference sample lines may be set to be unavailable as reference sample lines of the current block, or a multi-line intra prediction coding method for selecting any one of a plurality of reference sample lines may be set not to be used. When the multi-line intra prediction coding method is not used, adjacent reference sample lines may be set as reference sample lines of the current block.

[0339] When the wide-angle intra prediction mode is not used, refW and refH are set as the sum of nTbW and nTbH. Thus, in addition to the top-left reference sample, non-adjacent reference samples that are a distance i from the current block may include (nTbW + nTbH + offsetX[i]) upper reference samples and (nTbW + nTbH + offsetY[i]) left reference samples. That is, non-adjacent reference samples that are a distance i from the current block may include (2nTbW + 2nTbH + offsetX[i] + offsetY[i] + 1) reference samples. For example, if the value of whRatio is greater than 1, the value of offsetX is set to a value greater than offsetY. For example, the value of offsetX is set to 1, and the value of offsetY is set to 0. On the other hand, if the value of whRatio is less than 1, the value of offsetY is set to a value greater than offsetX. For example, set the value of offsetX as 0 and the value of offsetY as 1.

[0340] As wide-angle intra-prediction modes are used in addition to existing intra-prediction modes, the resources required to encode the wide-angle intra-prediction modes increase, which may reduce coding efficiency. Therefore, coding efficiency can be improved by encoding a replacement intra-prediction mode related to the wide-angle intra-prediction mode instead of directly encoding the wide-angle intra-prediction mode.

[0341] For example, when the current block is encoded using the 67th wide-angle intra prediction mode, the intra prediction mode of the current block may be coded as the 67th wide-angle replacement intra prediction mode, i.e., number 2. When the current block is encoded using the −1th wide-angle intra prediction mode, the intra prediction mode of the current block may be coded as the −1st wide-angle replacement intra prediction mode, i.e., number 66.

[0342] The decoder may decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode is included in the transform range. If the decoded intra-prediction mode is a wide-angle replacement intra-prediction mode, the decoder may transform the intra-prediction mode to a wide-angle intra-prediction mode.

[0343] Alternatively, if the current block is coded in a wide-angle intra-prediction mode, it may be coded directly in a wider-angle intra-prediction mode.

[0344] The encoding of the intra prediction mode may be realized based on the MPM list. Specifically, when encoding a neighboring block in a wide-angle intra prediction mode, the MPM may be set based on a wide-angle replacement intra prediction mode corresponding to the wide-angle intra prediction mode.

[0345] When generating a prediction block based on the result of intra prediction, prediction samples may be updated based on the position of each prediction sample included in the prediction block. The updating method described above may be referred to as a sample-position-based intra weighted prediction method (or Position Dependent Prediction Combination (PDPC)).

[0346] Whether to use PDPC may be determined taking into account the intra prediction mode of the current block, the reference sample line of the current block, the size or color component of the current block. For example, PDPC may be used when the intra prediction mode of the current block is at least one of a planar mode, a DC mode, a vertical mode, a horizontal mode, a mode in which the index value is smaller than the vertical direction, or a mode in which the index value is larger than the horizontal direction. Alternatively, PDPC may be used only when at least one of the width and height of the current block is greater than 4. Alternatively, PDPC may be used only when the index of the reference picture line of the current block is 0. Alternatively, PDPC may be used only when the index of the reference picture line of the current block is equal to or greater than a predefined value. Alternatively, PDPC may be used only for the luma component. Alternatively, whether to use PDPC may be determined depending on whether two or more of the above listed conditions are satisfied.

[0347] As another example, information indicating whether or not PDPC applies may be transmitted by a signal via the stream.

[0348] When a prediction sample is obtained by intra-prediction sample, a reference sample for correcting the prediction sample may be determined according to the position of the obtained prediction sample.For convenience of description, in the following embodiments, the reference sample for correcting the prediction sample is referred to as a PDPC reference sample.Then, the prediction sample obtained by intra-prediction is referred to as a first prediction sample, and the prediction sample obtained by correcting the first prediction sample is referred to as a second prediction sample.

[0349] FIG. 27 is a diagram showing an embodiment in which PDPC is applied.

[0350] The first predicted sample may be corrected using at least one PDPC reference sample, which may include at least one of a reference sample adjacent to the upper left corner of the current block, an upper reference sample located above the current block, or a left reference sample located to the left of the current block.

[0351] At least one of the reference samples belonging to the reference sample line of the current block may be set as the PDPC reference sample. Alternatively, regardless of the reference sample line of the current block, at least one of the reference samples belonging to the reference sample line with index 0 may be set as the PDPC reference sample. For example, even if the first predicted sample is obtained using the reference sample included in the reference sample line with index 1 or 2, the second predicted sample may be obtained using the reference sample included in the reference sample line with index 0.

[0352] The number or positions of PDPC reference samples for correcting the first predicted sample may be determined taking into consideration at least one of the intra prediction mode of the current block, the size of the current block, the shape of the current block, or the position of the first predicted sample.

[0353] For example, if the intra prediction mode of the current block is planar mode or DC mode, the second predicted sample may be obtained using an upper reference sample and a left reference sample, where the upper reference sample may be a reference sample perpendicular to the first predicted sample (e.g., a reference sample having the same x-coordinate), and the left reference sample may be a reference sample parallel to the first predicted sample (e.g., a reference sample having the same y-coordinate).

[0354] If the intra prediction mode of the current block is a horizontal intra prediction mode, the second predicted sample may be obtained using an upper reference sample, where the upper reference sample may be a reference sample perpendicular to the first predicted sample.

[0355] If the intra prediction mode of the current block is a vertical intra prediction mode, a second predicted sample may be obtained using a left reference sample, where the left reference sample may be a reference sample parallel to the first predicted sample.

[0356] If the intra prediction mode of the current block is a left-bottom diagonal intra prediction mode or an right-top diagonal intra prediction mode, a second predicted sample may be obtained based on an upper-left reference sample, an upper reference sample, and a left reference sample. The upper-left reference sample may be a reference sample adjacent to the upper left corner of the current block (e.g., a reference sample at the (-1,-1) position). The upper reference sample may be a reference sample located diagonally above and to the right of the first predicted sample, and the left reference sample may be a reference sample located diagonally below and to the left of the first predicted sample.

[0357] In other words, if the position of the first predicted sample is (x, y), R(-1, -1) may be set as the upper-left reference sample, and R(x+y+1, -1) or R(x, -1) may be set as the upper reference sample, and R(-1, x+y+1) or R(-1, y) may be set as the left reference sample.

[0358] A single prediction mode may be applied to the current block multiple times, or multiple prediction modes may be repeatedly applied to the current block. In this way, a prediction method using the same or different prediction modes may be called a combined prediction mode (or a multi-hypothesis prediction mode).

[0359] The combined prediction modes may include at least one of a mode that combines a merge mode with a merge mode, a mode that combines inter prediction with intra prediction, a mode that combines a merge mode with an advanced motion vector prediction mode, and a mode that combines a merge mode with intra prediction.

[0360] In the combined prediction mode, a first predicted block may be generated based on a first prediction mode, a second predicted block may be generated based on a second prediction mode, and then a third predicted block may be generated by performing a weighted sum operation on the first predicted block and the second predicted block. The third predicted block may be set as the final predicted block of the current block.

[0361] Whether to use the combined prediction mode may be determined based on the size or shape of the current block. For example, whether to use the combined prediction mode may be determined based on at least one of the size of the coding block, the number of samples included in the coding block, the width of the coding block, and the height of the coding block. For example, if at least one of the width and height of the coding block is 128 or more, or the number of samples included in the coding block is 64 or less, the combined prediction mode may not be applied to the coding block.

[0362] Furthermore, information indicating whether a combined prediction mode has been applied to the current block may be transmitted via a signal through a stream. For example, the information may be a 1-bit flag. For example, a flag mh_intra_flag indicating whether a combined prediction mode combining merge mode and intra prediction is to be used may be transmitted via a signal through a stream. When mh_intra_flag is 1, this indicates that a combined prediction mode combining merge mode and intra prediction is to be used, and when mh_intra_flag is 0, this indicates that a combined prediction mode combining merge mode and intra prediction is not to be used.

[0363] A combined prediction mode that combines the merge mode and intra prediction may be applied to the current block only when the prediction coding mode of the current block is an inter prediction mode and a merge mode is applied to the current block. That is, when the value of the merge_flag flag indicating whether the merge mode is applied to the current block is 1, the mh_intra_flag signal may be transmitted.

[0364] In a combined prediction mode that combines merge mode and intra prediction, the intra prediction mode of the current block may be set to a predefined intra prediction mode. For example, when using the combined prediction mode, the intra prediction mode of the current block may be set to planar mode.

[0365] As another example, when a combined prediction mode that combines merge mode and intra prediction is used, one of the candidate intra prediction modes may be determined as the intra prediction mode of the current block. Here, the candidate intra prediction modes may include at least one of a non-angular intra prediction mode and a specific-direction angular intra prediction mode. Here, the non-angular intra prediction mode includes at least one of a DC mode and a planar mode, and the angular intra prediction mode includes at least one of a horizontal intra prediction mode, a vertical intra prediction mode, and a diagonal intra prediction mode. For example, when a combined prediction mode is used, only DC mode, planar mode, horizontal intra prediction mode, or vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when a combined prediction mode is used, only planar mode, horizontal intra prediction mode, or vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when a combined prediction mode is used, only DC mode, planar mode, or vertical intra prediction mode may be set as the intra prediction mode of the current block. Alternatively, when using a combined prediction mode, only the DC mode, the planar mode, or the horizontal intra prediction mode may be set as the intra prediction mode of the current block.

[0366] Index information for specifying one of the candidate intra-prediction modes may be transmitted via a signal through the stream. For example, an index mh_intra_idx specifying one of the candidate intra-prediction modes may be transmitted via a signal through the stream. Tables 9 and 10 show intra-prediction modes based on the value of mh_intra_idx. The intra-prediction mode indicated by mh_intra_idx may be determined as the intra-prediction mode of the current block.

[0367] [Table 9]

[0368] [Table 10]

[0369] In the encoder and decoder, the number of candidate intra-prediction modes may have a fixed value. Alternatively, the number or types of candidate intra-prediction modes vary depending on at least one of the size of the current block, the shape of the current block, or whether neighboring blocks are coded by intra-prediction.

[0370] As another example, when a combined prediction mode that combines merge mode and intra prediction is applied, the intra prediction mode of the current block may be determined based on the MPM. In this case, when general intra prediction is applied, the number of MPMs included in the MPM list may be different from the number of MPMs included in the MPM list when the combined prediction mode is applied. For example, when the combined prediction mode is not applied, the MPM list may include six or five MPMs, but when the combined prediction mode is applied, the MPM list may include four or three MPMs. In other words, when general intra prediction is performed, N MPMs are used, but when the combined prediction mode is applied, fewer than N MPMs may be used.

[0371] For example, when applying a common intra prediction mode, assume that the MPMs derived based on the intra prediction modes of neighboring blocks adjacent to the current block are planar mode, DC mode, INTRA_MODE32, INTRA_MODE31, INTRA_MODE33, and INTRA_MODE30. When applying a combined prediction mode to the current block, the intra prediction mode of the current block may be determined using only some of the six MPMs. For example, the intra prediction mode of the current block may be determined using an MPM list including three MPMs with the smallest indexes (i.e., planar mode, DC mode, and INTRA_MODE32). Alternatively, any one of the MPMs with a predetermined index value may be determined as the intra prediction mode of the current block.

[0372] As another example, the number of MPMs for determining the intra prediction mode of the current block when applying general intra prediction may be set to be the same as the number of MPMs for determining the intra prediction mode of the current block when applying a combined prediction mode.

[0373] When a combined prediction mode is applied, encoding of the MPM flag may be omitted and the MPM flag may be regarded as true. That is, when a combined prediction mode is applied, any one of the MPMs may be determined as the intra prediction mode of the current block.

[0374] When a combined prediction mode that combines merge mode and intra prediction is applied to the current block, the multi-line intra prediction coding method may be set not to be used. When the multi-line intra prediction coding method is not used, neighboring reference sample lines may be used for intra prediction of the current block.

[0375] Alternatively, when a combined prediction mode that combines merge mode and intra prediction is applied to the current block, a multi-line intra prediction coding method may be set to be used. Information for determining a reference sample line of the current block from multiple reference sample lines may be transmitted by a signal via the stream. Alternatively, the reference sample line of the current block may be determined by taking into consideration at least one of the size of the current block, the shape of the current block, the intra prediction mode of the current block, or the reference sample line of a neighboring block.

[0376] The number of available candidate intra prediction modes may be set according to differences in reference sample lines of the current block. For example, when neighboring reference sample lines are used, the candidate intra prediction modes may include at least one of a planar mode, a DC mode, a horizontal intra prediction mode, a vertical intra prediction mode, a left-bottom diagonal intra prediction mode, and an right-top diagonal intra prediction mode. On the other hand, when non-neighboring reference sample lines are used, the candidate intra prediction modes may include at least one of a horizontal intra prediction mode, a vertical intra prediction mode, a left-bottom diagonal intra prediction mode, and an right-top diagonal intra prediction mode.

[0377] A first predicted block may be generated using a merge candidate selected in the merge mode, and a second predicted block may be generated using an intra prediction mode. In this case, when generating the first predicted block, triangulation may be set not to be applied to the current block. That is, triangulation may not be permitted in a combined prediction mode that combines the merge mode and intra prediction. Furthermore, when generating the second predicted block, PDPC may be set not to be applied to the current block. That is, PDPC may not be permitted in a combined prediction mode that combines the merge mode and intra prediction.

[0378] The third predicted block may be generated by a weighted sum operation of the first predicted block and the second predicted block. For example, Equation 13 shows an example of generating the third predicted block by a weighted sum operation of the first predicted block and the second predicted block.

[0379] Equation 13 JPEG2025183407000024.jpg684

[0380] In equation 13, P merge denotes the first predicted block obtained based on the merge mode, and P intra indicates the second predicted block obtained by intra prediction. comb where N denotes a third predicted block obtained by combining the first predicted block and the second predicted block. w denotes a first weighting value applied to the first predicted block. A second weighting value applied to the second predicted block can be derived by subtracting the first weighting value w from a constant N. Here, N may have a value predefined by the encoder and decoder. For example, N may be 4 or 8. Alternatively, the constant N may be derived based on at least one of the size, shape, and intra prediction mode of the current block.

[0381] Unlike the example shown in Equation 13, w may be set as the second weighting value, and the value obtained by subtracting the second weighting value from a predetermined constant N may be set as the first weighting value.

[0382] The weighting values ​​to be applied to the first and second predicted blocks may be determined according to at least one of the intra prediction mode of the current block and whether the merge mode has bidirectional motion information. For example, when the intra prediction mode of the current block is an angular prediction mode, the first weighting value w may be greater than the first weighting value w when the intra prediction mode of the current block is a non-angular prediction mode. Alternatively, when bidirectional prediction is performed using a merge candidate, the first weighting value w may be greater than the first weighting value w when unidirectional prediction is performed using a merge candidate.

[0383] As another example, the weighting value may be determined taking into account the predictive coding modes of neighboring blocks adjacent to the current block. Here, the neighboring blocks may include at least one of an upper neighboring block adjacent to the top of the current block, a left neighboring block adjacent to the left of the current block, and an upper left neighboring block adjacent to the upper left corner of the current block. The weighting value may be determined based on the number of neighboring blocks adjacent to the current block that are coded using intra prediction. For example, as the number of neighboring blocks adjacent to the current block that are coded using intra prediction increases, the first weighting value may have a relatively small value and the second weighting value may have a relatively large value. On the other hand, as the number of neighboring blocks adjacent to the current block that are coded using non-intra prediction decreases, the first weighting value may have a relatively large value and the second weighting value may have a relatively small value. For example, if both the left neighboring block and the upper neighboring block are coded using intra prediction, the second weighting value may be set to n. If only one of the left neighboring block and the upper neighboring block is coded using intra prediction, the second weighting value may be set to n / 2. If neither the left neighboring block nor the upper neighboring block is coded by intra prediction, the second weighting value may be set as n / 4, where n is an integer equal to or less than N.

[0384] The weighting value may be determined in units of samples or sub-blocks in the current block. For example, the current block may be divided into multiple sub-blocks, and different weighting values ​​may be applied to each sub-block. In such a case, the number of sub-blocks may be fixed in the encoder and decoder. Alternatively, the number of sub-blocks may be adaptively determined based on at least one of the size of the current block, the shape of the current block, the intra-prediction mode of the current block, and the predictive coding mode of a neighboring block.

[0385] FIG. 28 shows an example of applying weighting values.

[0386] The current block may be divided into multiple sub-blocks, and different weighting values ​​may be applied to each sub-block. For example, in the examples shown in Figures 28(a) and 28(b), the second weighting value may be set to be greater than the first weighting value for a sub-block including a sample located in the upper left corner of the current block. On the other hand, the first weighting value may be set to be greater than the second weighting value for a sub-block including a sample located in the lower right corner of the current block.

[0387] The third predicted sub-block of a sub-block may be obtained by performing weighted prediction on the first predicted sub-block and the second predicted sub-block corresponding to the sub-block. In this case, in the example shown in Figure 28, weights for weighted prediction may be set according to the differences between the sub-blocks.

[0388] The weighting values ​​assigned to the sub-blocks may be determined separately depending on whether the merge candidate has bidirectional motion information.

[0389] FIG. 29 illustrates an example of assigning different weighting values ​​to sub-blocks depending on whether the merge candidate has bidirectional motion information.

[0390] When a merging candidate has bidirectional motion information, the first weighting value w assigned to at least one sub-block may be set to a larger value than when the merging candidate has unidirectional motion information. For example, Figures 29(a) and 29(b) show that the weighting value w applied when bidirectional prediction is applied to all sub-blocks is larger than the weighting value w applied when unidirectional prediction is applied.

[0391] Alternatively, the weighting value to be applied to the sub-block may be determined based on the intra-prediction mode of the current block.

[0392] FIG. 30 illustrates an example of assigning different weighting values ​​to sub-blocks based on the intra-prediction mode of the current block.

[0393] If the intra prediction mode of the current block is vertical, the second weighting value of the sub-block located above the current block may be set to be greater than the second weighting value of the sub-block located below the current block.

[0394] For example, for a sub-block located above the current block, the second weighting value may be set to be greater than the first weighting value, while for a sub-block located below the current block, the first weighting value and the second weighting value may be set to be equal, or the first weighting value may be set to be greater than the second weighting value.

[0395] If the intra prediction mode of the current block is horizontal, the second weighting value of the sub-block located to the left of the current block may be set to be greater than the second weighting value of the sub-block located to the right of the current block.

[0396] For example, for a sub-block located to the left of the current block, the second weighting value may be set to be greater than the first weighting value, while for a sub-block located to the right of the current block, the first weighting value and the second weighting value may be set to be equal, or the first weighting value may be set to be greater than the second weighting value.

[0397] As another example, if the intra prediction mode of the current block is vertical, the first weighting value of the sub-block located above the current block may be set to be greater than the first weighting value of the sub-block located below the current block. Alternatively, if the intra prediction mode of the block is horizontal, the first weighting value of the sub-block located to the left of the current block may be set to be greater than the first weighting value of the sub-block located to the right of the current block.

[0398] As another example, the weighting value assigned to each sub-block may be determined based on the predictive coding mode of the adjacent blocks adjacent to the current block. For example, if the block adjacent to the upper side of the current block is coded using intra prediction but the block adjacent to the left side of the current block is not coded using intra prediction, the second weighting value of the sub-block positioned above the current block may be set to be greater than the second weighting value of the sub-block positioned below the current block. On the other hand, if the block adjacent to the upper side of the current block is not coded using intra prediction but the block adjacent to the left side of the current block is coded using intra prediction, the second weighting value of the sub-block positioned to the left of the current block may be set to be greater than the second weighting value of the sub-block positioned to the right of the current block.

[0399] For at least one of the plurality of sub-blocks, the first weighting value or the second weighting value may be set to 0. That is, for at least one of the plurality of sub-blocks, the first predictive sub-block or the second predictive sub-block may be set as the third predictive sub-block. Therefore, combined prediction may not be performed for at least one of the plurality of sub-blocks. For example, for at least one of the plurality of sub-blocks, the first weighting value w may be set to 0, or the first weighting value w may be set to a constant N.

[0400] A block that applies a combined prediction mode that combines merge mode and intra prediction may be considered to be coded by inter prediction. Therefore, the intra prediction mode of a block coded in a combined prediction mode may be set as an intra prediction mode that cannot be used to derive a block to be coded / decoded subsequently. When deriving an MPM, the intra prediction mode of the unavailable block may be considered as a planar mode.

[0401] Meanwhile, the motion information of a block coded in the combined prediction mode may be set as motion information that can be used to derive a block to be coded / decoded later.

[0402] In the merge mode and the combined merge mode, motion compensated prediction may be performed using multiple merge candidates. Specifically, a first predicted block may be generated using a first merge candidate, a second predicted block may be generated using a second merge candidate, and a third predicted block may be generated by performing a weighted sum operation on the first predicted block and the second predicted block.

[0403] Information for specifying the first merging candidate and the second merging candidate may be transmitted by signals via the stream. For example, index information merge_idx for specifying the first merging candidate and index information merge_2nd_idx for specifying the second merging candidate may be transmitted by signals via the stream. The second merging candidate may be determined based on the index information merge_2nd_idx and the index information merge_idx.

[0404] The index information merge_idx specifies one of the merge candidates included in the merge candidate list.

[0405] The index information merge_2nd_idx may specify any one of the remaining merge candidates other than the merge candidate specified by merge_idx. Therefore, if the value of merge_2nd_idx is smaller than the value of merge_idx, the merge candidate with the value of index merge_2nd_idx may be set as the second merge candidate. If the value of merge_2nd_idx is greater than or equal to the value of merge_idx, the merge candidate with the value of index merge_2nd_idx plus 1 may be set as the second merge candidate.

[0406] Alternatively, the second merging candidate may be specified taking into account the search order of the candidate blocks.

[0407] FIG. 31 is a diagram showing an example of specifying a second merging candidate in consideration of the search order of candidate blocks.

[0408] In the example shown in Figure 31, the indices marked on the adjacent and non-adjacent samples indicate the search order for candidate blocks, for example, candidate blocks may be searched in order from position A0 to position A14.

[0409] If block A4 is selected as the first merging candidate, a merging candidate derived from a candidate block that follows A4 in the search order may be designated as the second merging candidate. For example, a merging candidate derived from A5 may be selected as the second merging candidate. If the candidate block at position A5 cannot be used as a merging candidate, a merging candidate derived from the next candidate block may be selected as the second merging candidate.

[0410] Additionally, the first and second merging candidates may be selected from merging candidates derived from non-adjacent blocks.

[0411] FIG. 32 is a diagram showing an example of selecting a first merging candidate and a second merging candidate from merging candidates derived from non-adjacent blocks.

[0412] In the example shown in Figure 32, merging candidates derived from a first candidate block and a second candidate block that are not adjacent to the current block may be selected as the first merging candidate and the second merging candidate, respectively. In this case, the block row to which the first candidate block belongs may be different from the block row to which the second candidate block belongs. For example, the first merging candidate may be derived from any one of candidate blocks A5 to A10, and the second merging candidate may be derived from any one of candidate blocks A11 to A15.

[0413] Alternatively, the first candidate block and the second candidate block may be set so as not to be included in the same line (eg, row or column).

[0414] As another example, a second merging candidate may be specified based on a first merging candidate. In such a case, the first merging candidate may be specified by index information merge_idx signaled from the stream. For example, a merging candidate adjacent to the first merging candidate may be specified as the second merging candidate. Here, a merging candidate adjacent to the first merging candidate may refer to a merging candidate whose index difference from the first merging candidate is 1. For example, a merging candidate with index value merge_idx+1 may be set as the second merging candidate. In such a case, if the value of merge_idx+1 is greater than the maximum index value (or if the index value of the first merging candidate is the maximum index), a merging candidate with index value merge_idx-1 or a merging candidate with an index value of a predefined value (e.g., 0) may be set as the second merging candidate.

[0415] Alternatively, the term "neighboring merge candidates" may refer to merge candidates derived from candidate blocks that are spatially adjacent to the candidate block used to derive the first merge candidate, where the term "neighboring candidate blocks" may refer to neighboring blocks to the left, right, above, below, or diagonally adjacent to the candidate block.

[0416] As another example, the second merging candidate may be designated based on motion information of the first merging candidate. For example, a merging candidate that shares the same reference picture as the first merging candidate may be selected as the second merging candidate. If there are multiple merging candidates that share the same reference picture as the first merging candidate, the merging candidate with the smallest index, the merging candidate with the smallest index difference from the first merging candidate, or the merging candidate with the smallest motion vector difference from the first merging candidate may be selected as the second merging candidate. Alternatively, the second merging candidate may be selected based on index information that designates one of the multiple merging candidates.

[0417] Alternatively, if the first merging candidate is unidirectional prediction in a first direction, a merging candidate including motion information in a second direction may be set as the second merging candidate. For example, if the first merging candidate has motion information in a direction L0, a merging candidate having motion information in a direction L1 may be set as the second merging candidate. If there are multiple merging candidates having motion information in a direction L1, the merging candidate with the smallest index or the smallest index difference from the first merging candidate may be set as the second merging candidate. Alternatively, the second merging candidate may be selected based on index information specifying one of the multiple merging candidates.

[0418] As another example, one of the merging candidates derived from adjacent blocks adjacent to the current block may be set as the first merging candidate, and one of the merging candidates derived from non-adjacent blocks not adjacent to the current block may be set as the second merging candidate.

[0419] As another example, one of the merge candidates derived from the candidate block located above the current block may be set as the first merge candidate, and one of the merge candidates derived from the candidate block located to the left may be set as the second merge candidate.

[0420] In the combined prediction mode of combining two merge modes, the merge mode may refer to a merge mode based on a translational motion model (hereinafter referred to as a translational merge mode) or a merge mode based on an affine motion model (hereinafter referred to as an affine merge mode). That is, motion compensated prediction may be performed by combining a translational merge mode with a translational merge mode or a affine merge mode with an affine merge mode.

[0421] As another example, when encoding adjacent blocks for deriving a first merging candidate based on affine motion information, a merging candidate derived from the adjacent block encoded based on affine motion information may be set as the second merging candidate. For example, if the first merging candidate is an affine merging candidate or is derived based on motion information of a sub-block in a coding block encoded based on affine motion information, the affine merging candidate or a merging candidate derived based on motion information of a sub-block in a coding block encoded based on affine motion information may be selected as the second merging candidate. Here, the second merging candidate may include at least one of the merging candidate that satisfies the above conditions: the merging candidate closest to the first merging candidate in the search order, the merging candidate with the smallest index difference from the first merging candidate, the merging candidate with the smallest index, and the merging candidate with the smallest motion vector difference from the first merging candidate, among the merging candidates that satisfy the above conditions.

[0422] Conversely to the above example, when encoding neighboring blocks for deriving a first merging candidate based on translational motion information (i.e., non-affine motion information), a merging candidate derived from the neighboring block encoded based on the translational motion information may be set as a second merging candidate. For example, if the first merging candidate is a non-affine merging candidate, a non-affine merging candidate may be selected as the second merging candidate. Here, the second merging candidate may include at least one of the non-affine merging candidates that is closest to the first merging candidate according to the search order, the merging candidate with the smallest index difference from the first merging candidate, the merging candidate with the smallest index, and the merging candidate with the smallest motion vector difference from the first merging candidate.

[0423] Alternatively, the second merging candidate may be derived by adding or subtracting an offset amount to or from the index of the first merging candidate, and if the encoding methods of the adjacent blocks used to derive the first merging candidate and the second merging candidate are different, the second merging candidate may be re-set as another merging candidate.

[0424] FIG. 33 is a diagram illustrating an example of determining the second merging candidate.

[0425] If a first merging candidate is selected, a second merging candidate may be selected that has an index derived by adding or subtracting an offset from the index of the first merging candidate. For example, a first merging candidate may be selected that has an index indicated by merge_idx signaled from the stream, and a second merging candidate may be selected that has an index of merge_idx+1.

[0426] In this case, if the first merge candidate is an affine merge candidate or is derived from a sub-block belonging to a block coded using an affine motion model, an affine merge candidate or a merge candidate derived from a sub-block belonging to a block coded using an affine motion model should be set as the second merge candidate. If the merge candidate having the index merge_idx+1 is not an affine merge candidate or is not derived from a sub-block belonging to a block coded using an affine motion model, another merge candidate may be set as the second merge candidate. Here, the other merge candidate may be the first merge candidate, an affine merge candidate, or a merge candidate derived from a sub-block belonging to a block coded using an affine motion model that has the smallest difference from merge_idx+1.

[0427] On the other hand, if the first merge candidate is a non-affine merge candidate, the non-affine merge candidate should be set as the second merge candidate. If the merge candidate with index merge_idx+1 is an affine merge candidate or is derived from a sub-block belonging to a coding block coded using an affine motion model, another merge candidate may be set as the second merge candidate. Here, the other merge candidate may be the first merge candidate or non-affine merge candidate that has the smallest difference from merge_idx+1. For example, the example shown in Figure 33 indicates that merge candidate A3 with index merge_idx+2 is set as the second merge candidate.

[0428] As another example, motion compensated prediction may be performed by combining translational and affine merge modes, i.e., either the first or second merge candidate may be an affine merge candidate, and the other may be a non-affine merge candidate.

[0429] A combined predicted block may be obtained by performing a weighted sum operation on a first predicted block derived from a first merging candidate and a second predicted block derived from a second merging candidate. In this case, a weighting value applied to the first predicted block may be set to be greater than a weighting value applied to the second predicted block.

[0430] Alternatively, the weighting values ​​may be determined based on motion information of the first merging candidate and motion information of the second merging candidate. For example, the weighting values ​​to be applied to the first and second predictive blocks may be determined based on the difference in output order between the reference picture and the current picture. Specifically, the greater the difference in output order between the reference picture and the current picture, the smaller the weighting values ​​to be applied to the predictive blocks may be.

[0431] Alternatively, the weighting values ​​to be applied to the first and second predicted blocks may be determined taking into consideration the sizes or shapes of a candidate block for deriving a first merging candidate (hereinafter referred to as a first candidate block) and a candidate block for deriving a second merging candidate (hereinafter referred to as a second candidate block). For example, the weighting value to be applied to a predicted block derived from either the first or second candidate block, whose shape is similar to that of the current block, may be set to a relatively large value. On the other hand, the weighting value to be applied to a predicted block derived from either the first or second candidate block, whose shape is not similar to that of the current block, may be set to a relatively small value.

[0432] FIG. 34 is a diagram showing an example of determining a weighting value to be applied to a prediction block based on the shape of a candidate block.

[0433] We now assume that the blocks are non-square, with width greater than height.

[0434] The first and second prediction blocks may be derived based on the first and second merging candidates, and a combined prediction block may be generated by performing a weighted sum operation on the first and second prediction blocks. In this case, weighting values ​​to be applied to the first and second prediction blocks may be determined based on the shapes of the first and second candidate blocks.

[0435] For example, in the example shown in Figure 34, the first candidate block is square, and the second candidate block is non-square, with its width greater than its height. Because the shape of the second candidate block is the same as the current block, the weighting value applied to the second predicted block may be set to be greater than the weighting value applied to the first predicted block. For example, a weighting value of 5 / 8 may be applied to the second predicted block, and a weighting value of 3 / 8 may be applied to the first predicted block. Equation 14 shows an example of deriving a combined predicted block by performing a weighted sum operation on the first predicted block and the second predicted block.

[0436] Equation 14 JPEG2025183407000025.jpg671

[0437] P(x,y) denotes the combined predicted block, P1(x,y) denotes the first predicted block, and P2(x,y) denotes the second predicted block.

[0438] As another example, the weighting values ​​to be applied to the first and second predicted blocks may be determined based on the shape of the current block. For example, if the current block is non-square in shape, where the width is greater than the height, a larger weighting value may be applied to a predicted block generated based on a merge candidate derived from one of the first and second merging candidates located above the current block. If both the first and second merging candidates are derived from candidate blocks located above the current block, the weighting values ​​to be applied to the first and second predicted blocks may be set to be the same. On the other hand, if the current block is non-square in shape, where the height is greater than the width, a larger weighting value may be applied to a predicted block generated based on a merge candidate derived from one of the first and second merging candidates located to the left of the current block. If both the first and second merging candidates are derived from candidate blocks located to the left of the current block, the weighting values ​​to be applied to the first and second predicted blocks may be set to be the same. If the current block is square in shape, the weighting values ​​to be applied to the first and second predicted blocks may be set to be the same.

[0439] As another example, a weighting value to be applied to each prediction block may be determined based on the distance between the current block and the candidate block. Here, the distance may be determined based on the difference in x-axis coordinate, the difference in y-axis coordinate, or the minimum value thereof. A weighting value to be applied to a prediction block derived from a merging candidate having a relatively small distance from the current block may be set to be greater than a weighting value to be applied to a prediction block derived from a merging candidate having a relatively large distance from the current block. For example, in the example shown in FIG. 31, a first merging candidate is derived from a neighboring block adjacent to the current block, and a second merging candidate is derived from a non-neighboring block not adjacent to the current block. In this case, because the x-axis distance between the first candidate block and the current block is smaller than the x-axis distance between the second candidate block and the current block, a weighting value to be applied to the first prediction block may be set to be greater than a weighting value to be applied to the second prediction block.

[0440] Alternatively, when both the first and second merging candidates are derived from non-adjacent blocks, a larger weighting value may be assigned to a predicted block derived from a non-adjacent block that is relatively close to the current block. For example, in the example shown in FIG. 32, since the y-axis distance between the first candidate block and the current block is smaller than the y-axis distance between the second candidate block and the current block, the weighting value applied to the first predicted block may be set to be larger than the weighting value applied to the second predicted block.

[0441] Matched motion information may be derived based on the first and second merge candidates, and motion compensated prediction may be performed on the current block based on the matched motion information. For example, the motion vector of the current block may be derived by averaging or weighted summing the motion vectors of the first and second merge candidates. In this case, the weighting values ​​of the motion vectors applied to the first and second merge candidates may be determined according to the above embodiment.

[0442] If the first merge candidate is a non-affine merge candidate and the second affine merge candidate is an affine merge candidate, the motion vector of the current block can be derived by scaling the motion vector of the second merge candidate. Equation 15 shows an example of deriving the motion vector of the current block.

[0443] Equation 15 JPEG2025183407000026.jpg591

[0444] In Equation 15, (mvX, mvY) denotes the motion vector of the current block, (mv0x, mv0y) denotes the motion vector of the first merging candidate, and (mv1x, mv1y) denotes the motion vector of the second merging candidate. M denotes a scaling parameter. M may be predefined in the encoder and decoder. Alternatively, the value of the scaling parameter M may be determined based on the size of the current block or the candidate block. For example, if the width or height of the second candidate block is greater than 32, M may be set as 3; otherwise, M may be set as 2.

[0445] The motion information of the first merge candidate or the second merge candidate may be stored as motion information of a block to which a merge mode and a merge prediction mode that combines the merge modes are applied, and the stored motion information may be used to derive motion information of a block to be encoded / decoded in the future.

[0446] Alternatively, the block may be divided into multiple sub-blocks, and one of the motion information of the first merging candidate and the motion information of the second merging candidate may be stored as the motion information of each sub-block. In this case, the motion information of some of the multiple sub-blocks may be set as the motion information of the first merging candidate, and the motion information of other sub-blocks may be set as the motion information of the second merging candidate.

[0447] Alternatively, the matched motion information derived based on the motion information of the first merge candidate and the second merge candidate may be stored as motion information of a block to which a merge prediction mode that combines the merge modes is applied.

[0448] In a prediction mode that combines a merge mode and an advanced motion vector prediction mode, a first prediction block may be generated using motion information derived from a merge candidate, and a second prediction block may be generated using a motion vector derived from a motion vector prediction candidate.

[0449] In the advanced motion vector prediction mode, a motion vector prediction candidate may be derived from a neighboring block adjacent to the current block or a co-located block in a co-located picture. Then, one of a plurality of motion vector prediction candidates may be designated, and the designated motion vector prediction candidate may be set as the motion vector prediction result for the current block. Then, the motion vector of the current block may be derived by combining the motion vector prediction result for the current block and the motion vector difference.

[0450] In a prediction mode that combines a merge mode and an advanced motion vector prediction mode, a merge candidate and a motion vector prediction candidate may be derived from the same candidate block. For example, when a merge candidate is specified by merge_idx, the motion vector of the candidate block used to derive the specified merge candidate may be set as the motion vector prediction result. Alternatively, when a motion vector prediction candidate is specified by mvp_flag, a merge candidate derived from the candidate block used to derive the specified merge candidate may be selected.

[0451] Alternatively, the candidate blocks for deriving the merge candidate and the candidate blocks for deriving the motion vector prediction candidate may be different. For example, when a merge candidate derived from a candidate block located above the current block is selected, a motion vector prediction candidate derived from a candidate block located to the left of the current block may be selected.

[0452] Alternatively, if the merge candidate selected by the index information and the motion vector prediction candidate selected by the index information are derived from the same candidate block, the motion vector prediction candidate may be replaced with a motion vector prediction candidate derived from an adjacent candidate block adjacent to the candidate block, or the merge candidate may be replaced with a merge candidate derived from a candidate block adjacent to the candidate block.

[0453] FIG. 35 shows an example of replacing merging candidates.

[0454] The example shown in Figure 35(a) shows the selection of a merge candidate and a motion vector prediction candidate derived from a candidate block at position A2. As shown in the figure, when a merge candidate and a motion vector prediction candidate are derived from the same candidate block, a merge candidate or a motion vector prediction candidate derived from a candidate block adjacent to the candidate block may be used instead of the merge candidate or the motion vector prediction candidate. For example, as shown in Figure 35(b), a merge candidate at position A1 may be used instead of the merge candidate at position A2.

[0455] A first predicted block may be derived based on a merge candidate of the current block, and a second predicted block may be derived based on the motion vector prediction candidate. Then, a combined predicted block may be derived by performing a weighted sum operation on the first predicted block and the second predicted block. In this case, a weighting value applied to the second predicted block generated in the advanced motion vector prediction mode may be set to be greater than a weighting value applied to the first predicted block generated in the merge mode.

[0456] The residual picture may be derived by subtracting the original picture from the predicted picture. In this case, when the residual picture is transformed into the frequency domain, removing high-frequency components in the frequency domain does not significantly degrade the subjective quality of the video. Therefore, reducing or setting the high-frequency components to zero improves compression efficiency without causing noticeable visual distortion. To reflect these characteristics, the current block may be transformed to decompose the residual picture into two-dimensional frequency components. The transformation may be performed using a transform technique such as a discrete cosine transform (DCT) or a discrete sine transform (DST).

[0457] While the DCT uses a cosine transform to decompose (or transform) the residual picture into two-dimensional frequency components, the DST uses a sine transform to decompose (or transform) the residual picture into two-dimensional frequency components. As a result of the transformation of the residual picture, the frequency components may be referred to as base pictures. For example, when performing a DCT transform on a block of size NxN, N 2 basic pattern components can be obtained. The size of each basic pattern component contained in a block of size N×N can be obtained by transformation. Depending on the transformation technique used, the size of the basic pattern component may be referred to as a DCT coefficient or a DST coefficient.

[0458] The DCT transform technique is mainly used to transform pictures with a relatively large distribution of non-zero low-frequency components, while the DST transform technique is mainly used to transform pictures with a relatively large distribution of high-frequency components.

[0459] Additionally, transform techniques other than DCT or DST may be used to transform the residual picture.

[0460] Hereinafter, the process of transforming the residual picture into two-dimensional frequency components is referred to as two-dimensional picture transform. The size of the basic pattern components obtained by the transform result is referred to as transform coefficients. For example, the transform coefficients may refer to DCT coefficients or DST coefficients. When a primary transform and a secondary transform (described below) are simultaneously applied, the transform coefficients may indicate the size of the basic pattern components generated by the secondary transform result.

[0461] A transform technique may be determined on a block-by-block basis. The transform technique may be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, if the current block is coded in intra prediction mode and the size of the current block is smaller than N×N, the transform technique DST may be used to perform the transform. On the other hand, if the above conditions cannot be met, the transform technique DCT may be used to perform the transform.

[0462] In the residual picture, two-dimensional picture transform may not be performed on some blocks. Not performing two-dimensional picture transform may be referred to as transform skip. When transform skip is applied, quantization may be applied to residual values ​​that are not transformed.

[0463] After transforming a current block using a DCT or DST, the transformed current block may be transformed again. In this case, the DCT or DST-based transformation may be defined as a primary transformation, and the process of re-transforming the block to which the primary transformation is applied may be referred to as a secondary transformation.

[0464] The main transform may be performed using any one of a number of possible transform cores, for example, any one of the DCT2, DCT8 or DCT7 may be used to perform the main transform.

[0465] Different transform cores may be used for the horizontal and vertical directions, and further, information indicating the combination of horizontal and vertical transform cores may be transmitted via a signal through the stream.

[0466] The execution units of the primary transform and the secondary transform are different. For example, the primary transform may be performed on an 8x8 block, and the secondary transform may be performed on a 4x4 sub-block of the transformed 8x8 block. In such a case, the transform coefficients of the remaining area where the secondary transform is not performed may be set to 0.

[0467] Alternatively, the primary transform may be performed on the 4x4 block, and the secondary transform may be performed on a region of size 8x8 that contains the transformed 4x4 block.

[0468] A signal may be sent over the stream indicating whether or not to perform a secondary transformation.

[0469] Alternatively, whether to perform a secondary transform may depend on whether the horizontal transform core and the vertical transform core are the same. For example, the secondary transform may be performed only if the horizontal transform core and the vertical transform core are the same. Alternatively, the secondary transform may be performed only if the horizontal transform core and the vertical transform core are different.

[0470] Alternatively, a secondary transform is allowed only if the horizontal and vertical transforms utilize a predefined transform core, for example, a secondary transform may be allowed if the horizontal and vertical transforms use a DCT2 transform core.

[0471] Alternatively, whether to perform the secondary transform may be determined based on the number of non-zero transform coefficients of the current block. For example, if the non-zero transform coefficients of the current block are equal to or less than a threshold, the secondary transform may be set not to be used, and if the non-zero transform coefficients of the current block are greater than the threshold, the secondary transform may be set to be used. Furthermore, the secondary transform may be set to be used only when the current block is coded using intra prediction.

[0472] Based on the shape of the current block, the size or shape of the sub-block on which the secondary transformation is to be performed may be determined.

[0473] 36 and 37 are diagrams showing sub-blocks that are intended to perform secondary transformation.

[0474] If the current block is square, after the primary transform is performed, a secondary transform may be performed on the upper left sub-block of size NxN of the current block. For example, if the current block is an 8x8 coding block, after the primary transform is performed on the current block, a secondary transform may be performed on the upper left sub-block of size 4x4 of the current block (see FIG. 36).

[0475] If the current block is non-square, i.e., the width is four times greater than the height, then a secondary transform may be performed on the upper left sub-block of size (kN) × (4kN) of the current block after the primary transform is performed. For example, if the current block is non-square and has a size of 16 × 4, then a secondary transform may be performed on the upper left sub-block of size 2 × 8 of the current block (see FIG. 37(a)).

[0476] If the current block is non-square, i.e., the height is four times greater than the width, then a secondary transform may be performed on the upper left sub-block of size (4kN) x (kN) of the current block after the primary transform is performed. For example, if the current block is non-square and has a size of 16x4, then a secondary transform may be performed on the upper left sub-block of size 2x8 of the current block (see FIG. 37(b)).

[0477] The decoder may perform an inverse transform (second inverse transform) of the secondary transform, and then perform an inverse transform (first inverse transform) of the primary transform on the result. As a result of performing the second inverse transform and the first inverse transform, a residual signal of the current block can be obtained.

[0478] Information indicating the transform type of the current block may be transmitted by a signal via the stream, and the information may be index information tu_mts_idx indicating one of a combination of horizontal and vertical transform types.

[0479] The vertical and horizontal transform cores may be determined based on the transform type candidates specified based on the index information tu_mts_idx. Tables 11 and 12 show the transform type combinations based on tu_mts_idx.

[0480] [Table 11]

[0481] [Table 12]

[0482] The transform type may be determined as any one of DCT2, DST7, DCT8 or skip transform. Alternatively, other than the transform skip, only the transform core may be used to configure the transform type combination candidates.

[0483] When using Table 11, if tu_mts_idx is 0, a transform skip may be applied horizontally and vertically. If tu_mts_idx is 1, a DCT2 may be applied horizontally and vertically. If tu_mts_idx is 3, a DCT8 may be applied horizontally and a DCT7 may be applied vertically.

[0484] When using Table 12, if tu_mts_idx is 0, DCT2 may be applied horizontally and vertically. If tu_mts_idx is 1, transform skip may be applied horizontally and vertically. If tu_mts_idx is 3, DCT8 may be applied horizontally and DCT7 may be applied vertically.

[0485] Whether to encode the index information may be determined based on at least one of the size, shape, or number of non-zero coefficients of the current block. For example, if the number of non-zero coefficients is equal to or less than a threshold, the index information may not be signaled and a default transform type may be applied to the current block. The default transform type may be DST7. Alternatively, the default mode may vary depending on the size, shape, or intra-prediction mode of the current block.

[0486] The threshold may be determined based on the size or shape of the current block. For example, if the size of the current block is equal to or smaller than 32x32, the threshold may be set as 2, and if the current block is larger than 32x32 (e.g., if the current block is a coding block of size 32x64 or 64x32), the threshold may be set as 4.

[0487] The plurality of lookup tables may be pre-stored in the encoder / decoder. At least one of the index values ​​assigned to the transform type combination candidates, the types of the transform type combination candidates, and the number of the transform type combination candidates may be different among the plurality of lookup tables.

[0488] The lookup table for the current block may be selected based on at least one of the size, shape, predictive coding mode, intra prediction mode of the current block, whether to apply a second transform, or whether to apply a transform skip to the neighboring block.

[0489] For example, if the size of the current block is 4x4 or less or the current block is encoded using inter prediction, the lookup table in Table 11 may be used; if the size of the current block is greater than 4x4 or the current block is encoded using intra prediction, the lookup table in Table 12 may be used.

[0490] Alternatively, information indicating one of a plurality of lookup tables may be transmitted via a signal over the stream, and the decoder may select the lookup table for the current block based on the information.

[0491] As another example, an index assigned to a candidate transform type combination may be adaptively determined based on at least one of the size, shape, predictive coding mode, intra prediction mode, whether a second transform is performed, or whether a transform skip is applied to a neighboring block. For example, if the size of the current block is 4×4, an index assigned to the transform skip may have a smaller value than an index assigned to the transform skip if the size of the current block is larger than 4×4. Specifically, if the size of the current block is 4×4, an index of 0 may be assigned to the transform skip, and if the current block is larger than 4×4 but smaller than or equal to 16×16, an index greater than 0 (e.g., index 1) may be assigned to the transform skip. If the current block is larger than 16×16, a maximum value (e.g., 5) may be assigned as the index of the transform skip.

[0492] Alternatively, if the current block is coded using inter prediction, the transform skip may be assigned an index of 0. If the current block is coded using intra prediction, the transform skip may be assigned an index greater than 0 (e.g., index 1).

[0493] Alternatively, if the current block is a block of size 4x4 that is coded using inter prediction, the transform skip may be assigned an index of 0. On the other hand, if the current block is not coded using inter prediction or is larger than 4x4, the transform skip may be assigned an index of a value greater than 0 (e.g., index 1).

[0494] Transform type combination candidates different from the transform type combination candidates listed in Tables 11 and 12 may be defined and used. For example, transform type combination candidates to which a transform core such as DCT7, DCT8, or DST2 is applied may be used to apply transform skip to horizontal or vertical transforms and other transforms. In this case, whether to use transform skip as a horizontal or vertical transform type candidate may be determined based on at least one of the size (e.g., width and / or height), shape, predictive coding mode, or intra prediction mode of the current block.

[0495] Alternatively, information indicating whether a specified candidate transform type is available may be transmitted via a signal through the stream. For example, a flag indicating whether a transform skip can be used as a candidate transform type for the horizontal and vertical directions may be transmitted via a signal. Based on the flag, it may be determined whether the specified candidate transform type combination is included in the multiple candidate transform type combinations.

[0496] Alternatively, information indicating whether a candidate transform type is to be applied to the current block may be transmitted by a signal via the stream. For example, a flag cu_mts_flag indicating whether DCT2 is to be applied in the horizontal and vertical directions may be transmitted by a signal. If the value of cu_mts_flag is 1, DCT2 may be set as the transform core for the vertical and horizontal directions. If the value of cu_mts_flag is 0, DCT8 or DST7 may be set as the transform core for the vertical and horizontal directions. Alternatively, if the value of cu_mts_flag is 0, information tu_mts_idx specifying one of a plurality of candidate transform type combinations may be transmitted by a signal.

[0497] If the current block is a non-square block with width greater than height or height greater than width, further encoding of cu_mts_flag may be omitted and the value of cu_mts_flag may be considered as 0.

[0498] The number of available transform type combination candidates may be set differently based on the size, shape, or intra-prediction mode of the current block. For example, if the current block is square, three or more transform type combination candidates may be available, and if the current block is non-square, two transform type combination candidates may be available. Alternatively, if the current block is square, only transform type combination candidates with different horizontal and vertical transform types may be available.

[0499] If there are three or more transform type combination candidates available for the current block, index information tu_mts_idx indicating one of the transform type combination candidates may be transmitted by a signal. On the other hand, if there are two transform type combination candidates available for the current block, a flag mts_flag indicating one of the transform type combination candidates may be transmitted by a signal. Table 13 below shows a process of encoding information specifying a transform type combination candidate based on the shape of the current block.

[0500] [Table 13]

[0501] The indexes of the candidate transform type combinations are sorted (or reordered) based on the shape of the current block. For example, the indexes assigned to the candidate transform type combinations when the current block is square may be different from the indexes assigned to the candidate transform type combinations when the current block is non-square. For example, when the current block is square, the transform type combinations may be selected based on Table 14 below, and when the current block is non-square, the transform type combinations may be selected based on Table 15 below.

[0502] [Table 14]

[0503] [Table 15]

[0504] The transform type may be determined based on the number of horizontal non-zero coefficients or the number of vertical non-zero coefficients of the current block. The number of horizontal non-zero coefficients indicates the number of non-zero coefficients contained in 1×N (where N is the width of the current block), and the number of vertical non-zero coefficients indicates the number of non-zero coefficients contained in N×1 (where N is the height of the current block). If the maximum number of horizontal non-zero coefficients is less than or equal to a threshold, a primary transform type may be applied in the horizontal direction; if the maximum number of horizontal non-zero coefficients is greater than the threshold, a secondary transform type may be applied in the horizontal direction. If the maximum number of vertical non-zero coefficients is less than or equal to a threshold, a primary transform type may be applied in the vertical direction; if the maximum number of vertical non-zero coefficients is greater than the threshold, a secondary transform type may be applied in the vertical direction.

[0505] FIG. 38 is a diagram for explaining an example of determining the transformation type of the current block.

[0506] For example, if the current block is coded using intra prediction and the maximum value of the horizontal non-zero coefficients of the current block is 2 or less (see FIG. 38(a)), the horizontal transform type may be determined as DST7.

[0507] If the current block is coded using intra prediction and the maximum value of the vertical non-zero coefficients of the current block is greater than 2 (see FIG. 38(b)), the vertical transform type may be determined as DCT2 or DCT8.

[0508] A signal may be sent via the stream to indicate whether the transform type of the current block is to be explicitly determined based on the information sent by the signal from the stream. For example, a signal may be sent at the sequence level to indicate whether a block coded using intra prediction allows explicit transform type determination, such as sps_explicit_intra_mts_flag, and / or whether a block coded using inter prediction allows explicit transform type determination, such as sps_explicit_inter_mts_flag.

[0509] If explicit transform type determination is allowed, the transform type of the current block may be determined based on index information tu_mts_idx signaled from the stream. On the other hand, if explicit transform type determination is not allowed, the transform type may be determined based on at least one of the size of the current block, the shape of the current block, whether sub-block unit transforms are allowed, and the positions of sub-blocks containing non-zero transform coefficients. For example, the horizontal transform type of the current block may be determined based on the width of the current block, and the vertical transform type of the current block may be determined based on the height of the current block. For example, if the width of the current block is less than 4 or greater than 16, the horizontal transform type may be determined as DCT2. Otherwise, the horizontal transform type may be determined as DST7. If the height of the current block is less than 4 or greater than 16, the vertical transform type may be determined as DCT2. Otherwise, the vertical transform type may be determined as DST7. Here, in order to determine the horizontal transformation type and the vertical transformation type, thresholds to be compared with the width and height may be determined based on at least one of the size, shape, and intra prediction mode of the current block.

[0510] Alternatively, if the current block is square with the same height and width, the horizontal transform type and the vertical transform type may be set to be the same, but if the current block is non-square with different heights and widths, the horizontal transform type and the vertical transform type may be set to be different. For example, if the width of the current block is greater than the height, the horizontal transform type may be determined as DST7 and the vertical transform type may be determined as DCT2. If the height of the current block is greater than the width, the vertical transform type may be determined as DST7 and the horizontal transform type may be determined as DCT2.

[0511] Depending on whether explicit transform type determination is allowed, the number and / or types of transform type candidates or the number and / or types of transform type combination candidates may differ. For example, if explicit transform type determination is allowed, DCT2, DST7, and DCT8 may be used as transform type candidates. Therefore, each of the horizontal transform type and the vertical transform type may be set as DCT2, DST8, or DCT8. If explicit transform type determination is not allowed, DCT2 and DST7 are used as transform type candidates only. Therefore, each of the horizontal transform type and the vertical transform type may be determined as DCT2 or DST7.

[0512] If the encoder performs the transform and quantization, the decoder can obtain the residual block by inverse quantization and inverse transform. The decoder can add the predicted block and the residual block to obtain a reconstructed block of the current block.

[0513] When the reconstructed block of the current block is obtained, in-loop filtering can reduce information loss that occurs during the quantization and coding process. The in-loop filter is a deblocking filter. The in-loop filter may include at least one of a sample adaptive offset filter (SAO), a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before applying the in-loop filter is referred to as a first reconstructed block, and the reconstructed block after applying the in-loop filter is referred to as a second reconstructed block.

[0514] At least one of a deblocking filter, SAO, or ALF may be applied to the first reconstructed block to obtain a second reconstructed block, in which case the SAO or ALF may be applied after the deblocking filter is applied.

[0515] The deblocking filter is used to reduce image quality degradation (blocking artifacts) that occur at block boundaries when quantization is performed on a block-by-block basis. To apply the deblocking filter, a blocking strength (BS) between a first reconstructed block and an adjacent reconstructed block may be determined.

[0516] FIG. 39 is a flowchart showing the process of determining the blocking strength.

[0517] In the example shown in Figure 39, P denotes the first reconstructed block and Q denotes the adjacent reconstructed block, which may be adjacent to the left or above the current block.

[0518] The example shown in Figure 39 shows that the blocking strength is determined taking into account the predictive coding modes of P and Q, whether they contain non-zero transform coefficients, whether inter-prediction is performed using the same reference picture, and whether the difference in motion vectors is greater than or equal to a threshold.

[0519] Whether to apply a deblocking filter may be determined based on the blocking strength, for example, if the blocking strength is 0, no filtering may be performed.

[0520] SAO is used to reduce ringing artifacts that occur when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset determined based on the pattern of the first reconstructed picture. Methods for determining the offset include edge offset (EO) and band offset. EO refers to a method of determining the offset of a current sample based on the pattern of surrounding pixels. BO refers to a method of applying the same offset to a set of pixels with similar luminance values ​​in a region. Specifically, pixel luminance is divided into 32 equal intervals, and pixels with similar luminance values ​​are grouped together. For example, four adjacent bands out of 32 bands may be grouped together, and the same offset may be applied to samples belonging to the four bands.

[0521] ALF is a method of generating a second reconstructed picture by applying a filter of a predefined size / shape to a first reconstructed picture or a reconstructed picture to which a deblocking filter is applied. Equation 16 below shows an example of applying ALF.

[0522] Equation 16 JPEG2025183407000032.jpg18150

[0523] Any one of predefined filter candidates may be selected for each picture, coding tree unit, coding block, prediction block, or transform block, and each filter candidate may have a different size or shape.

[0524] FIG. 40 is a diagram showing predefined filter candidates.

[0525] In the example shown in FIG. 40, at least one of a 5x5, 7x7 and 9x9 diamond may be selected.

[0526] Only diamonds of size 5x5 can be used for the chromaticity components.

[0527] It is within the scope of the present invention to use an embodiment described with a focus on a decoding process or an encoding process in an encoding process or a decoding process. It is also within the scope of the present invention to modify the embodiments described according to a predetermined order in an order other than the order in which they are described.

[0528] Although the embodiments have been described based on a series of steps or flowcharts, the time sequence of the invention is not limited, and the steps may be executed simultaneously or in another order as needed. In the above embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagrams may also be implemented as hardware devices or software, and multiple structural elements may be combined into a single hardware device or software device. The embodiments may be implemented in the form of program instructions, which may be executed by various computer components and stored on a computer-readable storage medium. The computer-readable storage medium may include program instructions, data files, data structures, etc., independently or in combination. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. The hardware devices may be configured to operate as one or more software modules to perform the processes of the present invention, or vice versa. [Industrial Applicability]

[0529] The present invention may be applied to electronic devices that encode / decode video.

Claims

1. 1. A video decoding method comprising: determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained, the first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; obtaining a third predicted block of the current block by performing a weighted sum operation on the first predicted block and the second predicted block; When the combined prediction mode is applied to the current block, triangulation is not applied to the current block. Video decoding methods.

2. 1. A video encoding method comprising: determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained, the first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; obtaining a third predicted block of the current block by performing a weighted sum operation on the first predicted block and the second predicted block; When the combined prediction mode is applied to the current block, triangulation is not applied to the current block. Video coding methods.

3. 1. A video decoding apparatus comprising: a prediction unit; The prediction unit determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block; obtaining a third predicted block of the current block by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; When the combined prediction mode is applied to the current block, triangulation is not applied to the current block. Video decoding device.

4. A video encoding device comprising: a prediction unit; The prediction unit determining whether to apply a combined prediction mode to a current block; When the combined prediction mode is applied to the current block, obtaining a first prediction block and a second prediction block of the current block; obtaining a third predicted block of the current block by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; When the combined prediction mode is applied to the current block, triangulation is not applied to the current block. Video coding device.

5. 1. A video decoder comprising: a processor; and a memory configured to store a computer program executable by said processor; The processor: determining whether to apply a combined prediction mode to the current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained; a third predicted block of the current block is obtained by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; When the combined prediction mode is applied to the current block, triangulation is not applied to the current block. Video decoder.

6. 1. A video encoder comprising: a processor; and a memory configured to store a computer program executable by the processor; The processor: determining whether to apply a combined prediction mode to the current block; When the combined prediction mode is applied to the current block, a first predicted block and a second predicted block of the current block are obtained; a third predicted block of the current block is obtained by performing a weighted sum operation of the first predicted block and the second predicted block; The first predicted block is obtained based on motion information of a merging candidate of the current block, and the second predicted block is obtained based on an intra prediction mode of the current block, wherein the intra prediction mode of the current block is a planar mode; When the combined prediction mode is applied to the current block, triangulation is not applied to the current block. Video encoder.

7. A computer-readable storage medium having a computer program and a bitstream stored thereon, The computer program, when executed by a processor, performs the method of claim 2 to generate the bitstream. A computer-readable storage medium.

Images