Video signal encoding / decoding method and device

Abstract

To provide a video signal encoding / decoding method and a device that improve inter-prediction efficiency by dividing an encoded block that improve inter prediction efficiency into a plurality of prediction blocks and deriving motion information for each prediction block from among a plurality of predictions.SOLUTION: A decryption method includes the steps of dividing a coded block into a first prediction unit PU1 and a second prediction unit PU2, deriving a merging candidate list of the coded blocks, deriving first motion information of a first prediction unit and second motion information of a second prediction unit using the merge candidate list, and obtaining predicted samples in the encoded block on the basis of the first motion information and the second motion information.SELECTED DRAWING: Figure 23

Description

[Technical field]

[0001] The present invention relates to a video signal encoding / decoding method and an apparatus therefor. [Background technology]

[0002] As display panels become larger and larger, video services with higher image quality are required. The biggest problem with high-definition video services is the large increase in data volume. To solve this problem, active studies on improving video compression rates are underway. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding), which was approved on January 25, 2013. Its compression performance is about twice that of H.264 / AVC. With the rapid growth of high-definition video services, the performance limitations of HEVC are gradually appearing. Summary of the Invention [Problem to be solved by the invention]

[0003] It is an object of the present invention to provide a method for dividing a coding block into a number of prediction blocks when encoding / decoding a video signal, and an apparatus for implementing said method.

[0004] It is an object of the present invention to provide a method for deriving motion information for each predictive block of a plurality of predictive blocks when encoding / decoding a video signal, and an apparatus for implementing said method.

[0005] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a method for deriving merging candidates using an inter motion information list when encoding / decoding a video signal, and an apparatus for performing said method.

[0006] The technical problems that the present invention aims to achieve are not limited to the technical problems mentioned above, and a person having ordinary skill in the art to which the present invention pertains will clearly understand other technical problems not mentioned from the following description. [Means for solving the problem]

[0007] A video signal decoding / encoding method according to the present invention includes the steps of: dividing a coding block into a first prediction unit and a second prediction unit; deriving a merge candidate list for the coding block; deriving first motion information of the first prediction unit and second motion information of the second prediction unit using the merge candidate list; and obtaining a prediction sample in the coding block based on the first motion information and the second motion information. In this case, whether to divide the coding block is determined based on the size of the coding block. Also, the first motion information of the first prediction unit is derived from a first merge candidate in the merge candidate list, and the second motion information of the second prediction unit is derived from a second merge candidate different from the first merge candidate.

[0008] In the video signal decoding / encoding method according to the present invention, if at least one of the width and height of the coding block is greater than a threshold, splitting the coding block may not be allowed.

[0009] In a video signal decoding / encoding method according to the present invention, the method further includes a step of decoding, from a bitstream, first index information for specifying the first merging candidate and second index information for specifying the second merging candidate, wherein if a value of the second index information is greater than or equal to a value of the first index information, the second merging candidate has, as an index, a value obtained by adding 1 to the value of the second index information.

[0010] In the video signal encoding / decoding method of the present invention, when the prediction sample is included in the boundary area between the first prediction unit and the second prediction unit, the prediction sample can be derived based on a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information.

[0011] In the video signal encoding / decoding method of the present invention, a first weighting value applied to the first predicted sample may be determined based on an x-axis coordinate and a y-axis coordinate of the predicted sample.

[0012] In the video signal encoding / decoding method of the present invention, the second weighting value applied to the second predicted sample may be derived by subtracting the first weighting value from a constant.

[0013] In the video signal decoding / encoding method according to the present invention, the maximum number of merging candidates included in the merging candidate list can be determined based on whether the coding block is divided into the first prediction unit and the second prediction unit.

[0014] The above brief summary features of the present invention are merely exemplary embodiments of the detailed description of the invention that follows and are not intended to limit the scope of the invention. Effect of the Invention

[0015] According to the present invention, by providing a method for dividing a coding block into a plurality of prediction blocks and deriving motion information of each prediction block among the plurality of predictions, it is possible to improve inter prediction efficiency.

[0016] According to the present invention, by providing a method for deriving merging candidates using an inter motion information list, the inter prediction efficiency can be improved.

[0017] The effects obtainable by the present invention are not limited to the above effects, and a person having ordinary skill in the art to which the present invention pertains will clearly understand other effects not mentioned from the following description. [Brief description of the drawings]

[0018] [Figure 1] FIG. 2 is a block diagram illustrating a video encoder according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram illustrating a video decoder according to an embodiment of the present invention. [Diagram 3] FIG. 2 illustrates a basic coding tree unit according to an embodiment of the present invention. [Figure 4] A diagram showing multiple division types of a coding block. [Diagram 5] A diagram showing the division modes of a coding tree unit. [Figure 6] 2 is a flowchart illustrating an inter-prediction method according to an embodiment of the present invention. [Figure 7] FIG. 2 illustrates non-linear motion of an object. [Figure 8] 2 is a flowchart illustrating an affine motion based inter prediction method according to an embodiment of the present invention. [Figure 9] FIG. 2 is a diagram showing examples of affine seed vectors for each affine motion model. [Figure 10] FIG. 2 is a diagram showing examples of affine vectors of sub-blocks in a four-parameter motion model. [Figure 11]11 is a flowchart illustrating a process of deriving motion information for a current block in merge mode. [Figure 12] FIG. 13 illustrates an example of candidate blocks for deriving merging candidates. [Figure 13] FIG. 13 illustrates the location of a reference sample. [Figure 14] FIG. 13 illustrates an example of candidate blocks for deriving merging candidates. [Figure 15] 13 is a flowchart illustrating an update state of an inter-motion information list. [Figure 16] FIG. 13 illustrates an example of updating an inter-merge candidate list. [Figure 17] FIG. 13 is a diagram showing an example in which the indexes of stored inter-merge candidates are updated. [Figure 18] FIG. 2 shows the location of a representative sub-block. [Figure 19] FIG. 13 is a diagram illustrating an example of generating inter motion information lists for different inter prediction modes. [Figure 20] 13 is a diagram showing an example of adding an inter-merge candidate included in a long-term motion information list to a merge candidate list. FIG. [Figure 21] FIG. 13 illustrates an example of performing redundancy detection on only some of the merging candidates. [Figure 22] FIG. 13 illustrates an example of skipping redundancy detection for a particular merging candidate. [Diagram 23] FIG. 13 is a diagram showing an example of dividing a coding block into multiple prediction units using diagonals. [Figure 24] FIG. 13 is a diagram showing an example of dividing a coding block into two prediction units. [Diagram 25] FIG. 1 is a diagram showing an example of dividing a coding block into multiple prediction blocks having different sizes. [Figure 26] FIG. 13 illustrates adjacent blocks for deriving triangle merging candidates. [Figure 27] FIG. 13 illustrates an example for determining the availability of neighboring blocks for each triangular prediction unit. [Figure 28] FIG. 13 is a diagram illustrating an example of deriving a predicted sample based on a weighted sum operation of a first predicted sample and a second predicted sample. [Figure 29] FIG. 13 is a diagram illustrating an example of deriving a predicted sample based on a weighted sum operation of a first predicted sample and a second predicted sample. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0020] Video encoding and decoding is performed on a block-by-block basis. For example, encoding / decoding operations such as transform, quantization, prediction, loop filtering, or reconstruction may be performed on a coding block, a transform block, or a predictive block.

[0021] Hereinafter, the block to be coded / decoded is called a “current block.” For example, according to a current coding / decoding process step, the current block can represent a coding block, a transformation block or a prediction block.

[0022] In addition, the term "unit" as used in this specification may be understood to represent a basic unit for performing a specific encoding / decoding process, and "block" may be understood to represent a sample array of a certain size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described below, a coding block and a coding unit may be understood to have the same meaning.

[0023] FIG. 1 is a block diagram illustrating a video encoder according to an embodiment of the present invention.

[0024] Referring to FIG. 1, the video encoding device 100 may include an image division unit 110, a prediction unit 120, 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150 and a memory 155.

[0025] Each component shown in Fig. 1 is shown alone to represent different characteristic functions in a video encoding device, but does not represent that each component is composed of separate hardware or a single software assembly. That is, for ease of explanation, each component is arranged so that at least two of the components are combined to form one component, or one component is divided into multiple components to perform the function. As long as it does not deviate from the essence of the present invention, such an embodiment in which each component is integrated and an embodiment in which each component is separated also fall within the scope of the present invention.

[0026] In addition, some of the components are not essential components for performing the essential functions of the present invention, but are only optional components for improving performance. The present invention may be implemented by including only the members necessary to realize the essence of the present invention other than the components used only to improve performance, and a structure including only essential components other than the optional components used only to improve performance also falls within the scope of the present invention.

[0027] The image division unit 110 can divide an input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image division unit 110 divides one image into a combination of multiple coding units, prediction units, and transform units. The image can be coded by selecting a combination of a coding unit, a prediction unit, and a transform unit based on a predetermined criterion (e.g., a cost function).

[0028] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used to divide a video or largest coding unit into other coding units as a root. The other coding units may have child nodes whose number is equal to the number of divided coding units. A coding unit that is not divided due to some restrictions becomes a leaf node. That is, if it is assumed that a coding unit can only realize a square division, a coding unit can be divided into a maximum of four other coding units.

[0029] Hereinafter, in the embodiments of the present invention, the encoding unit may refer to a unit that performs encoding, or may refer to a unit that performs decoding.

[0030] The prediction units in one coding unit may be divided into shapes such as at least one of a square or a rectangle of the same size, and one prediction unit in one coding unit may also be divided into a shape and / or size that differs from another prediction unit.

[0031] Performing intra prediction based on a coding unit If the prediction unit is not the smallest coding unit, intra prediction can be performed without the need to split into multiple prediction units NxN.

[0032] The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It may be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference image, etc.) may be determined based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction unit may determine the prediction method and prediction mode, and the prediction may be performed by the conversion unit. A residual value (residual block) between the generated prediction block and the original block may be input to the conversion unit 130. In addition, prediction mode information, motion vector information, etc. for prediction may be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When a specific encoding mode is used, the prediction unit 120 and 125 may not generate a prediction block, but the original block may be directly encoded and transmitted to the decoder.

[0033] The inter prediction unit 120 may predict a prediction unit based on information of at least one image, either an image immediately before or an image immediately after the current image. In some cases, the prediction unit may also be predicted based on information of a part of an encoded region in the current image. The inter prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0034] The reference image interpolation unit receives reference image information from the memory 155, and can generate pixel information of integer pixels or less from the reference image. For luminance pixels, an 8-tap interpolation filter based on DCT with different filter coefficients can be used to generate pixel information of integer pixels or less in units of 1 / 4 pixels. For chrominance signals, a 4-tap interpolation filter based on DCT with different filter coefficients can be used to generate pixel information of integer pixels or less in units of 1 / 8 pixels.

[0035] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. As a method for calculating a motion vector, multiple methods such as a full search-based block matching algorithm (FBMA), a three step search method (TSS), a new three-step search algorithm (NTS), etc. can be used. According to the interpolated pixel, the motion vector may have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel. In the motion prediction unit, a current prediction unit can be predicted by using different motion prediction methods. As a motion prediction method, multiple methods such as a skip method, a merge method, an advanced motion vector prediction method (AMVP), an intra block copy method, etc. can be used.

[0036] The intra prediction unit 125 can generate a prediction unit based on the neighboring reference pixel information of the current block, which is pixel information in the current image. When the neighboring block of the current prediction unit is a block on which inter prediction has been performed, and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be the reference pixel information of the neighboring block on which intra prediction has been performed. In other words, when the reference pixel is unavailable, at least one reference pixel of the available reference pixels can be used instead of the unavailable reference pixel information.

[0037] In intra prediction, the prediction mode may include an angular prediction mode that uses reference pixel information based on the prediction direction and a non-angular mode that does not use direction information when performing prediction. The mode for predicting luma information may be different from the mode for predicting chroma information. To predict chroma information, intra prediction mode information for predicting luma information or predicted luma signal information may be used.

[0038] When performing intra prediction, if the size of prediction unit is the same as the size of transform unit, intra prediction can be performed on prediction unit based on the pixel located on the left side, the pixel located on the top left, and the pixel located on the top.However, when performing intra prediction, if the size of prediction unit is different from the size of transform unit, intra prediction can be performed based on the reference pixel of transform unit.Note that intra prediction using N×N division can be applied only to the smallest coding unit.

[0039] After applying an Adaptive Intra Smoothing (AIS) filter to the reference pixels based on the prediction mode, a prediction block can be generated by an intra prediction method. The type of the adaptive intra smoothing filter applied to the reference pixels can be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted based on the intra prediction mode of a prediction unit located in the vicinity of the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted from the surrounding prediction units, if the intra prediction mode of the current prediction unit is the same as the intra prediction mode of the surrounding prediction units, information indicating that the prediction mode of the current prediction unit is the same as the surrounding prediction units can be transmitted using predetermined flag information. If the prediction mode of the current prediction unit is different from the prediction mode of the surrounding prediction units, the prediction mode information of the current block can be encoded by performing entropy encoding.

[0040] In addition, a residual block including residual value information that is a difference value between a prediction unit that performs prediction based on the prediction unit generated by the prediction unit 120, 125 and an original block of the prediction unit can be generated. The generated residual block can be input to the conversion unit 130.

[0041] The transform unit 130 may perform transform on the original block and the residual block including residual value information between the prediction units generated by the prediction units 120 and 125 using a transform method such as a discrete cosine transform (DCT) or a discrete sine transform (DST). Here, the DCT transform kernel includes at least one of DCT2 or DCT8, and the DSTc includes DST7. Whether to apply DCT or DST to transform the residual block may be determined based on intra prediction mode information of the prediction unit for generating the residual block. Transformation of the residual block may also be skipped. A flag indicating whether to skip transformation of the residual block may be coded. Transformation may be skipped for residual blocks whose magnitude is equal to or less than a threshold, luma component, or chroma component (4:4:4 format or less).

[0042] The quantization unit 135 may quantize the values ​​transformed into the frequency domain by the transformation unit 130. The quantization coefficient may vary depending on the importance of the block or image. The values ​​calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0043] The rearrangement unit 160 may perform rearrangement of coefficient values ​​on the quantized residual values.

[0044] The rearrangement unit 160 may change two-dimensional block shape coefficients into one-dimensional vector form using a coefficient scanning method. For example, the rearrangement unit 160 may scan DC coefficients or high-frequency region coefficients using a zig-zag scan method and convert them into one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of zig-zag scanning, vertical scanning for scanning two-dimensional block shape coefficients along a column direction and horizontal scanning for scanning two-dimensional block shape coefficients along a row direction may be used. That is, it is possible to determine which of zig-zag scanning, vertical scanning, and horizontal scanning to use, depending on the size of the transform unit and the intra prediction mode.

[0045] The entropy coding unit 165 may perform entropy coding based on the value calculated by the rearrangement unit 160. For example, the entropy coding may use a plurality of coding methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0046] The entropy coding unit 165 can code a variety of information such as residual value coefficient information and block type information of the coding unit from the rearrangement unit 160 and the prediction units 120 and 125, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0047] The entropy coding unit 165 may perform entropy coding on the coefficient values ​​of the coding unit input from the rearrangement unit 160 .

[0048] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the multiple values ​​quantized by the quantization unit 135, and perform inverse transform on the values ​​transformed by the transform unit 130. A reconstructed block can be generated by merging residual values ​​generated by the inverse quantization unit 140 and the inverse transform unit 145 with prediction units predicted by the motion prediction unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125.

[0049] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0050] The deblocking filter can remove block artifacts generated in a reconstructed image due to boundaries between blocks. In order to determine whether to perform deblocking, it can be determined whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied based on a required deblocking filtering strength. In addition, when performing vertical filtering or horizontal filtering in the process of using the deblocking filter, horizontal filtering and vertical filtering can be performed synchronously.

[0051] The offset correction unit can correct the offset between the deblocked video and the original video on a pixel-by-pixel basis. The offset correction can be performed on a specified image in the following manner: After dividing the pixels included in the video into a predetermined number of regions, the regions that require offset execution are determined, and an offset is applied to the corresponding region or an offset is applied taking into account edge information of each pixel.

[0052] Adaptive loop filtering (ALF) can be performed based on a comparison value between the filtered reconstructed image and the original video. After dividing the pixels included in the video into predetermined groups, a filter to be used for the corresponding group is determined, so that filtering can be performed differentially for each group. Information regarding whether adaptive loop filtering is applied and luminance information can be transmitted according to a coding unit (CU). The shape and filter coefficients of the adaptive loop filter to be applied vary depending on each block. It is also possible to apply the same type (constant type) of adaptive loop filter regardless of the characteristics of the block to which it is applied.

[0053] The memory 155 can store the reconstructed block or image calculated by the filter unit 150 and can provide the stored reconstructed block or image to the prediction units 120, 125 when performing inter prediction.

[0054] FIG. 2 is a block diagram illustrating a video decoder according to an embodiment of the present invention.

[0055] Referring to FIG. 2, the video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0056] When a video bitstream is input from a video encoder, the input bitstream can be decoded in steps that are the reverse of those taken by the video encoder.

[0057] The entropy decoding unit 210 may perform entropy decoding in a step that is the reverse of the step of performing entropy encoding in the entropy encoding unit of the video encoder. For example, a number of methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc. may be applied to correspond to the method performed in the video encoder.

[0058] The entropy decoding unit 210 can perform decoding on information related to intra prediction and inter prediction performed by the encoder.

[0059] The rearrangement unit 215 may perform rearrangement by rearranging a bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. The rearrangement may be performed by reconstructing a plurality of coefficients represented in a one-dimensional vector form into coefficients in a two-dimensional block shape. The rearrangement unit 215 may perform rearrangement by receiving information related to a coefficient scan performed by the encoding unit and performing a reverse scan according to the scan order performed by the corresponding encoding unit.

[0060] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values ​​of the rearranged block.

[0061] The inverse transform unit 225 may perform an inverse discrete cosine transform and an inverse discrete sine transform on the quantization result performed by the video encoder. The inverse discrete cosine transform and the inverse discrete sine transform are inverse transforms of the transform performed by the transform unit, that is, inverse transforms of the discrete cosine transform and the discrete sine transform. Here, the DCT transform kernel may include at least one of DCT2 and DCT8, and the DST transform kernel may include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform unit 225 may not perform the inverse transform. The inverse transform may be performed by a transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder may selectively perform a transform method (e.g., DCT or DST) based on a plurality of pieces of information such as a prediction method, a size of a current block, and a prediction direction.

[0062] The prediction units 230, 235 can generate a prediction block based on information related to the generation of the prediction block provided by the entropy decoding unit 210 and previously decoded block or image information provided by the memory 245.

[0063] As mentioned above, when performing intra prediction in the same manner as the operation manner in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixel located to the left of the prediction unit, the pixel located to the upper left, and the pixel located above.When performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed based on the reference pixel of the transform unit.Note that intra prediction using N×N division can be applied only to the minimum coding unit.

[0064] The prediction unit 230, 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives a plurality of pieces of information, such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and classifies the prediction unit based on the current encoding unit, and determines whether the prediction unit is performing inter prediction or intra prediction. The inter prediction unit 230 can perform inter prediction on the current prediction unit based on information included in at least one image of the image immediately before or immediately after the current image to which the current prediction unit belongs, using information necessary for performing inter prediction of the current prediction unit provided by the video encoder. Alternatively, inter prediction can be performed based on information of a reconstructed part of the area in the current image to which the current prediction unit belongs.

[0065] To perform inter prediction, it is possible to determine, based on the coding unit, whether the motion prediction method of the prediction unit included in the corresponding coding unit is skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), or intra block duplication mode.

[0066] The intra prediction unit 235 may generate a prediction block based on pixel information in the current image. If the prediction unit is a prediction unit that has performed intra prediction, the intra prediction may be performed based on intra prediction mode information of the prediction unit provided from the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra smoothing filter is a part that performs filtering on reference pixels of the current block, and may determine whether to apply a filter based on the prediction mode of the current prediction unit. Adaptive intra smoothing filtering may be performed on reference pixels of the current block using the prediction mode of the prediction unit and adaptive intra smoothing filter information provided from the video encoder. If the prediction mode of the current block is a mode that does not perform adaptive intra smoothing filtering, the adaptive intra smoothing filter may not be applied.

[0067] Regarding the reference pixel interpolation unit, if the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values ​​to be interpolated for the reference pixels, the reference pixels can be generated by performing interpolation on the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without performing interpolation on the reference pixels, the reference pixels do not need to be interpolated. If the prediction mode of the current block is a DC mode, the DC filter can generate a prediction block by filtering.

[0068] The reconstructed block or image may be provided to a filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0069] From the video encoder, information regarding whether to apply a deblocking filter to a corresponding block or image and information regarding whether to use a strong filter or a weak filter when applying the deblocking filter may be received. The deblocking filter of the video decoder may receive the information regarding the deblocking filter provided by the video encoder, and the video decoder may perform deblocking filtering on the corresponding block.

[0070] The offset compensation unit may perform offset compensation on the reconstructed image based on the offset information and the type of offset compensation applied to the video when encoding.

[0071] ALF can be applied to a coding unit based on information provided by the encoder regarding whether to apply ALF, ALF coefficient information, etc. Such ALF information can be provided by being included in a particular parameter set.

[0072] The memory 245 is capable of storing the reconstructed image or block, making said image or block available as a reference image or block, and of providing the reconstructed image to an output.

[0073] FIG. 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

[0074] The largest coding block can be defined as a coding tree block. An image may be divided into multiple coding tree units (a coding tree unit is a coding tree unit (CTU) in size). The largest coding unit may be called a largest coding unit (LCU). Figure 3 shows an example of dividing an image into multiple coding tree units.

[0075] The size of the coding tree unit may be defined at the picture level or at the sequence level, so that the picture parameter set or the sequence parameter set can be used to signal the information indicating the size of the coding tree unit.

[0076] For example, the size of the coding tree unit for all images in a sequence may be 128 x 128. Alternatively, the size of the coding tree unit may be determined to be one of 128 x 128 or 256 x 256 at the image level. For example, the size of the coding tree unit in a first image may be 128 x 128, and the size of the coding tree unit in a second image may be 256 x 256.

[0077] A coding block can be generated by dividing the coding tree unit. The coding block represents a basic unit for performing coding / decoding processing. For example, prediction or transformation can be performed according to different coding blocks, or a predictive coding mode can be determined according to different coding blocks. Here, the predictive coding mode represents a method for generating a predicted image. For example, the predictive coding mode may include intra prediction (Intra Prediction), inter prediction (Inter Prediction), current picture referencing (CPR), intra block copy (IBC), or combined prediction (Combined Prediction). For a coding block, a predictive block related to the coding block can be generated using at least one predictive coding mode among intra prediction, inter prediction, current picture referencing, or combined prediction.

[0078] Information indicating the predictive coding mode of the current block may be transmitted in a signal via the bitstream. For example, the information may be a one-bit flag indicating whether the predictive coding mode is an intra mode or an inter mode. Only if it is determined that the predictive coding mode of the current block is an inter mode, can the current image reference or combined prediction be used.

[0079] The current image reference is used to obtain a prediction block of the current block from an encoded / decoded area in the current image by using the current image as a reference image. Here, the current image means an image including the current block. Information indicating whether the current image reference is applied to the current block may be transmitted as a signal via the bitstream. For example, the information may be a 1-bit flag. If the flag is true, the prediction coding mode of the current block may be determined as the current image reference. If the flag is false, the prediction mode of the current block may be determined as the inter prediction.

[0080] Alternatively, the predictive coding mode of the current block may be determined based on the reference image index. For example, if the reference image index points to the current image, the predictive coding mode of the current block may be determined as current image reference. If the reference image index points to an image other than the current image, the predictive coding mode of the current block may be determined as inter prediction. That is, the current image reference is a prediction method that uses information of an encoded / decoded area in the current image, and the inter prediction is a prediction method that uses information of another encoded / decoded image.

[0081] The combined prediction is a coding mode formed by combining two or more of intra prediction, inter prediction, and current image reference. For example, when the combined prediction is applied, a first predicted block may be generated based on one of intra prediction, inter prediction, and current image reference, and a second predicted block may be generated based on the other. When the first predicted block and the second predicted block are generated, a final predicted block may be generated by averaging and weighted summing the first predicted block and the second predicted block. Information indicating whether the combined prediction is applied may be transmitted in a signal via the bitstream. The information may be a 1-bit flag.

[0082] FIG. 4 is a diagram showing a number of division types of a coding block.

[0083] A coding block can be divided into a plurality of coding blocks based on a quadtree division, a binary tree division, or a ternary tree division. Also, a divided coding block can be further divided into a plurality of coding blocks based on a quadtree division, a binary tree division, or a ternary tree division.

[0084] Quadtree partitioning is a partitioning technique that divides the current block into four blocks. As a result of the quadtree partitioning, the current block can be divided into four square partitions (see “SPLIT_QT” in (a) of FIG. 4).

[0085] Binary tree partitioning is a partitioning technique that divides a current block into two blocks. The process of dividing the current block into two blocks along the vertical direction (i.e., using a vertical line that crosses the current block) can be called vertical binary tree partitioning, and the process of dividing the current block into two blocks along the horizontal direction (i.e., using a horizontal line that crosses the current block) can be called horizontal binary tree partitioning. After binary tree partitioning, the current block can be divided into two non-square partitions. "SPLIT_BT_VER" in (b) of FIG. 4 represents the vertical binary tree partitioning result, and "SPLIT_BT_HOR" in (c) of FIG. 4 represents the horizontal binary tree partitioning result.

[0086] Ternary tree partitioning is a partitioning technique that partitions the current block into three blocks. The process of partitioning the current block into three blocks along the vertical direction (i.e., using two vertical lines that cross the current block) can be called vertical ternary tree partitioning, and the process of partitioning the current block into three blocks along the horizontal direction (i.e., using two horizontal lines that cross the current block) can be called horizontal ternary tree partitioning. After ternary tree partitioning, the current block can be partitioned into three non-square partitions. In this case, the width / height of the partition located at the center of the current block may be twice the width / height of the other partitions. "SPLIT_TT_VER" in FIG. 4(d) represents the vertical ternary tree partitioning result, and "SPLIT_TT_HOR" in FIG. 4(e) represents the horizontal ternary tree partitioning result.

[0087] The number of times a coding tree unit is divided can be defined as a partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at a sequence or image level. Therefore, the maximum partitioning depth of a coding tree unit may differ depending on a sequence or image.

[0088] Alternatively, a maximum partitioning depth can be determined for each of multiple partitioning techniques independently. For example, the maximum partitioning depth that allows for quadtree partitioning may be different from the maximum partitioning depth that allows for binary tree partitioning and / or ternary tree partitioning.

[0089] The encoder may transmit information indicative of at least one of the partition shape or partition depth of the current block in a signal via the bitstream, and the decoder may determine the partition shape and partition depth of the coding tree unit based on the information parsed from the bitstream.

[0090] FIG. 5 is a diagram illustrating an example of division of a coding tree unit.

[0091] The process of dividing a coding block using a division technique such as quad-tree division, binary-tree division and / or ternary-tree division can be called multi-tree partitioning.

[0092] The coding blocks generated by applying the multi-tree partitioning to the coding block can be called downstream coding blocks. If the partitioning depth of the coding block is k, the partitioning depth of the downstream coding blocks is k+1.

[0093] On the other hand, for a plurality of coding blocks with a division depth of k+1, the coding block with a division depth of k can be called an upstream coding block.

[0094] The division type of the current coding block may be determined based on at least one of the division shape of the upstream coding block or the division type of the adjacent coding block, where the adjacent coding block is adjacent to the current coding block and may include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the current coding block, where the division type may include at least one of whether to perform quadtree division, whether to perform binary tree division, the binary tree division direction, whether to perform ternary tree division, or the ternary tree division direction.

[0095] To determine the partition shape of the coding block, information indicating whether the coding block is partitioned can be transmitted as a signal via the bitstream. The information is a 1-bit flag "split_cu_flag", which indicates that the coding block is partitioned using a multi-tree partitioning technique if the flag is true.

[0096] If "split_cu_flag" is true, a signal may be sent via the bitstream indicating whether the coding block has been quadtree split. The information is a 1-bit flag "split_qt_flag", and if the flag is true, the coding block may be split into 4 blocks.

[0097] For example, in the example shown in Fig. 5, the coding tree unit is quadtree-divided to generate four coding blocks with a division depth of 1. Note that the diagram illustrates the application of quadtree division to the first and fourth coding blocks among the four coding blocks generated as a result of the quadtree division. Finally, four coding blocks with a division depth of 2 can be generated.

[0098] In addition, by applying quadtree partitioning again to a coding block with a partition depth of 2, a coding block with a partition depth of 3 can be generated.

[0099] When quadtree partitioning is not applied to the coding block, it is possible to determine whether to perform binary tree partitioning or ternary tree partitioning on the coding block by considering at least one of the following: the size of the coding block, whether the coding block is located on the boundary of the image, the maximum partition depth, or the partition shape of the adjacent block. When it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating a partitioning direction can be transmitted by a signal via a bit stream. The information may be a one-bit flag "mtt_split_cu_vertical_flag". Based on the flag, it can be determined whether the partitioning direction is vertical or horizontal. It is to be noted that information indicating whether binary tree partitioning or ternary tree partitioning is applied to the coding block can be transmitted by a signal via a bit stream. The information may be a one-bit flag "mtt_split_cu_binary_flag". Based on the flag, it can be determined whether to apply binary tree partitioning or ternary tree partitioning to the coding block.

[0100] For example, in the example shown in Fig. 5, vertical binary tree partitioning is applied to a coding block with a partition depth of 1. Of the coding blocks generated as a result of the partitioning, vertical ternary tree partitioning is applied to the left coding block, and vertical binary tree partitioning is applied to the right coding block.

[0101] Inter prediction is a predictive coding mode that predicts a current block using information of a previous image. For example, a block in the previous image at the same position as the current block (hereinafter, referred to as a collocated block) can be used as a prediction block for the current block. Hereinafter, a prediction block generated based on a block that is located at the same position as the current block is referred to as a collocated prediction block.

[0102] On the other hand, if an object in the previous image moves to another position in the current image, the current block can be effectively predicted based on the object's motion. For example, if the moving direction and size of the object can be understood by comparing the previous image with the current image, a predicted block (or predicted image) of the current block can be generated taking into account the object's motion information. Hereinafter, the predicted block generated using the motion information can be referred to as a motion predicted block.

[0103] A residual block can be generated by subtracting the predictive block from the current block. In this case, in the presence of target motion, the energy of the residual block can be reduced by using a motion predictive block instead of a co-located predictive block, improving the compression performance of the residual block.

[0104] As described above, a process of generating a prediction block using motion information may be called motion compensated prediction. In most inter predictions, a prediction block may be generated based on motion compensated prediction.

[0105] The motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weight index. The motion vector represents the moving direction and size of an object. The reference image index specifies a reference image of the current block among multiple reference images included in a reference image list. The prediction direction indicates one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). At least one of the motion information in the L0 direction or the motion information in the L1 direction can be used based on the prediction direction of the current block. The bidirectional weight index specifies a weight applied to the L0 prediction block and a weight applied to the L1 prediction block.

[0106] FIG. 6 is a flowchart illustrating an inter prediction method according to an embodiment of the present invention.

[0107] Referring to FIG. 6, the inter prediction method includes a step of determining an inter prediction mode of a current block (S601), a step of obtaining motion information of the current block based on the determined inter prediction mode (S602), and a step of performing motion compensation prediction on the current block based on the obtained motion information (S603).

[0108] Here, the inter prediction mode represents a number of techniques for determining motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information may include a merge mode and an advanced motion vector prediction mode. The inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. According to the inter prediction mode, the motion information of the current block may be determined based on information analyzed from a neighboring block or a bitstream adjacent to the current block.

[0109] The inter prediction method using affine motion information will now be described in detail.

[0110] FIG. 7 is a diagram illustrating the nonlinear movement of an object.

[0111] The motion of an object in a video may be nonlinear. For example, as shown in the example of FIG. 7, nonlinear motion of an object may occur, such as camera zoom-in, zoom-out, rotation, or affine transformation. When nonlinear motion of an object occurs, the motion of the object cannot be effectively represented by a translational motion vector. Therefore, in the part where nonlinear motion of an object occurs, affine motion is used instead of translational motion, thereby improving coding efficiency.

[0112] FIG. 8 is a flow chart illustrating an affine motion based inter prediction method according to an embodiment of the present invention.

[0113] According to the information analyzed from the bitstream, it can determine whether to apply an affine motion-based inter prediction technique to the current block. Specifically, it can determine whether to apply an affine motion-based inter prediction technique to the current block according to at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0114] When applying an affine motion-based inter prediction technique to a current block, an affine motion model of the current block can be determined (S1101). The affine motion model may be determined by at least one of a six-parameter affine motion model or a four-parameter affine motion model. The six-parameter affine motion model represents affine motion with six parameters, and the four-parameter affine motion model represents affine motion with four parameters.

[0115] Equation 1 is a case where affine motion is expressed by six parameters. The affine motion represents the translational motion of a given region determined by an affine seed vector.

number

[0116] When affine motion is represented by six parameters, complex motion can be represented, but the number of bits required for coding each parameter increases, which reduces coding efficiency. Therefore, affine motion can also be represented by four parameters. Equation 2 is the case when affine motion is represented by four parameters.

number

[0117] Information for determining an affine motion model of a current block may be coded and transmitted as a signal through a bitstream. For example, the information may be a 1-bit flag "affine_type_flag". When the value of the flag is 0, it indicates that a 4-parameter affine motion model is applied. When the value of the flag is 1, it indicates that a 6-parameter affine motion model is applied. The flag may be coded in units of slices, tiles, or blocks (e.g., coding blocks or coding tree units). When a flag is transmitted using a signal at a slice level, the affine motion model determined at the slice level may be applied to all blocks belonging to the slice.

[0118] Alternatively, an affine motion model of the current block may be determined based on an affine inter prediction mode of the current block. For example, when an affine merge mode is applied, an affine motion model of the current block may be determined as a four-parameter motion model. On the other hand, when an affine motion vector prediction mode is applied, information for determining an affine motion model of the current block may be coded and transmitted as a signal via a bitstream. For example, when an affine motion vector prediction mode is applied to the current block, an affine motion model of the current block may be determined based on a 1-bit flag "affine_type_flag".

[0119] Next, an affine seed vector for the current block may be derived (S1102). If a four-parameter affine motion model is selected, motion vectors may be derived at two control points of the current block. However, if a six-parameter affine motion model is selected, motion vectors may be derived at three control points of the current block. The motion vectors at the control points may be referred to as affine seed vectors. The control points may include at least one of the top left corner, the top right corner, or the bottom left corner of the current block.

[0120] FIG. 9 is a diagram showing examples of affine seed vectors for each affine motion model.

[0121] In a four-parameter affine motion model, an affine seed vector can be derived for two of the top left corner, the top right corner, or the bottom left corner. For example, as shown in FIG. 9A, when a four-parameter affine motion model is selected, an affine seed vector sv 0 and the affine seed vector sv associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)). 1 Alternatively, instead of the affine seed vector for the upper left corner, an affine seed vector for the lower left corner can be used, or instead of the affine seed vector for the upper right corner, an affine seed vector for the lower left corner can be used.

[0122] In a six-parameter affine motion model, affine seed vectors for the top left corner, top right corner, and bottom left corner can be derived. For example, as shown in FIG. 9B, when a six-parameter affine motion model is selected, an affine seed vector sv for the top left corner of the current block (e.g., the top left sample (x0, y0)) can be derived. 0 , the affine seed vector sv associated with the top-right corner of the current block (e.g., the top-right sample (x1, y1)). 1 and the affine seed vector sv associated with the top-left corner of the current block (e.g., the top-left sample (x2, y2)). 2 The affine vector can be derived using:

[0123] In the embodiment described below, in the four-parameter affine motion model, the affine seed vectors of the top-left control point and the top-right control point are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiment described below that uses the first affine seed vector and the second affine seed vector, at least one of the first affine seed vector and the second affine seed vector can be replaced with the affine seed vector of the bottom-left control point (the third affine seed vector) or the affine seed vector of the bottom-right control point (the fourth affine seed vector).

[0124] In the six-parameter affine motion model, the affine seed vectors of the top-left control point, the top-right control point, and the bottom-left control point are referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector, respectively. In an embodiment using the first affine seed vector, the second affine seed vector, and the third affine seed vector, described below, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with the affine seed vector of the bottom-right control point (fourth affine seed vector).

[0125] The affine seed vector can be used to derive affine vectors according to different sub-blocks (S803), where the affine vectors represent translational motion vectors derived based on the affine seed vectors. The affine vectors of a sub-block can be called affine sub-block motion vectors or sub-block motion vectors.

[0126] FIG. 10 is a diagram showing an example of affine vectors of sub-blocks in a four-parameter motion model.

[0127] The affine vector of the sub-block can be derived based on the control point position, the sub-block position, and the affine seed vector. For example, Equation 3 shows an example of derivation of the affine sub-block vector.

number

[0128] In the above formula 3, (x, y) represents the position of the subblock. Here, the position of the subblock represents the position of the reference sample included in the subblock. The reference sample may be a sample located at the upper left corner of the subblock, or may be at least one sample located at the center position in the x-axis or y-axis coordinate. (x 0 ,y 0 ) represents the position of the first control point, and (sv 0x ,sv 0y ) represents the first affine seed vector. Note that (x 1 ,y 1 ) represents the position of the second control point, and (sv 1x ,sv 1y ) represents the second affine seed vector.

[0129] If the first control point and the second control point correspond to the upper left corner and the upper right corner of the current block, respectively, then x 1 -x 0 can be set to a value that is equal to the width of the current block.

[0130] Then, motion compensation prediction can be performed for each subblock using the affine vector of each subblock (S1104). After performing motion compensation prediction, a prediction block for each subblock can be generated. The prediction block of the subblock can be set as the prediction block of the current block.

[0131] Next, the inter prediction method using translational motion information will be described in detail.

[0132] The motion information of the current block may be derived from the motion information of another block of the current block. Here, the other block may be a block that is preferentially encoded / decoded by inter prediction rather than the current block. A merge mode may be defined as a case where the motion information of the current block is the same as the motion information of another block. Also, a motion vector prediction mode may be defined as a case where the motion vector of the other block is set as a predicted value of the motion vector of the current block.

[0133] FIG. 11 is a flow chart illustrating a process of deriving motion information for a current block in merge mode.

[0134] Merge candidates for a current block may be derived (S1101). The merging candidates for the current block are derived from a block that is before the current block and that is coded / decoded using inter prediction.

[0135] FIG. 12 is a diagram showing examples of candidate blocks for deriving merging candidates.

[0136] The candidate block may include at least one of adjacent blocks including samples adjacent to the current block or non-adjacent blocks including samples not adjacent to the current block. Hereinafter, the samples for determining the candidate block are designated as reference samples. Note that the reference samples adjacent to the current block are called adjacent reference samples, and the reference samples not adjacent to the current block are called non-adjacent reference samples.

[0137] The neighboring reference sample may be included in the neighboring column of the leftmost column of the current block or the neighboring row of the topmost row of the current block. For example, if the coordinates of the top left sample of the current block are (0,0), one of the block including the reference sample at the (-1,H-1) position, the block including the reference sample at the (W-1,-1) position, the block including the reference sample at the (W,-1) position, the block including the reference sample at the (-1,H) position, or the block including the reference sample at the (-1,-1) position may be used as a candidate block. As shown in the drawing, the neighboring blocks with indexes 0 to 4 may be used as candidate blocks.

[0138] The non-adjacent reference sample represents a sample having at least one of the x-axis distance or y-axis distance from a reference sample adjacent to the current block having a predefined value. For example, one of a block including a reference sample having a predefined x-axis distance from a left reference sample, a block including a non-adjacent sample having a predefined y-axis distance from an upper reference sample, or a block including a non-adjacent sample having predefined x-axis and y-axis distances from an upper left reference sample can be used as a candidate block. The predefined value may be an integer such as 4, 8, 12, 16, etc. As shown in the drawing, at least one of blocks having indexes from 5 to 26 can be used as a candidate block.

[0139] A non-adjacent reference sample may be a sample that is not located on the same vertical, horizontal, or diagonal line as an adjacent reference sample.

[0140] FIG. 13 is a diagram showing the positions of the reference samples.

[0141] As shown in the example of Fig. 13, the x coordinate of the upper non-adjacent reference sample may be set to be different from the x coordinate of the upper adjacent reference sample. For example, when the position of the upper adjacent reference sample is (W-1, -1), the position of the upper non-adjacent reference sample that is N away from the upper adjacent reference sample along the y axis may be ((W / 2)-1, -1-N), and the position of the upper non-adjacent reference sample that is 2N away from the upper adjacent reference sample along the y axis may be (0, -1-2N). That is, the position of the non-adjacent reference sample may be determined based on the position of the adjacent reference sample and the distance from the adjacent reference sample.

[0142] Hereinafter, among the candidate blocks, a candidate block including adjacent reference samples will be referred to as an adjacent block, and a block including non-adjacent reference samples will be referred to as a non-adjacent block.

[0143] If the distance between the current block and the candidate block is equal to or greater than a threshold, the candidate block may be set as unavailable as a merging candidate. The threshold may be determined according to the size of the coding tree unit. For example, the threshold may be the height of the coding tree unit (ctu_height) or a value obtained by adding or subtracting an offset value to the height of the coding tree unit (e.g., ctu_height ± N). The offset value N is a value predefined in the encoder and decoder, and may be 4, 8, 16, 32, or ctu_height.

[0144] If the difference value between the y-axis coordinate of the current block and the y-axis coordinate of a sample included in the candidate block is greater than a threshold, the candidate block may be determined to be unavailable as a merging candidate.

[0145] Alternatively, a candidate block that does not belong to the same coding tree unit as the current block can be set as unavailable as a merging candidate, for example, if the reference sample is beyond the upper boundary of the coding tree unit to which the current block belongs, the candidate block containing the reference sample can be set as unavailable as a merging candidate.

[0146] When the upper boundary of the current block is adjacent to the upper boundary of a coding tree unit, if multiple candidate blocks are set as unavailable as merge candidates, the encoding / decoding efficiency of the current block will be reduced. To solve the above problem, the number of candidate blocks located above the current block can be made greater than the number of candidate blocks located to the left of the current block by setting the candidate blocks.

[0147] FIG. 14 is a diagram showing examples of candidate blocks for deriving merging candidates.

[0148] 14, the candidate blocks can be an upper block belonging to the N block rows above the current block and a left block belonging to the M block rows to the left of the current block. In this case, by making M larger than N, the number of left candidate blocks can be made larger than the number of above candidate blocks.

[0149] For example, the difference between the y-axis coordinate of a reference sample in the current block and the y-axis coordinate of an upper block that can be used as a candidate block can be set to N times the height of the current block or less, while the difference between the x-axis coordinate of a reference sample in the current block and the x-axis coordinate of a left block that can be used as a candidate block can be set to M times the width of the current block or less.

[0150] For example, in the example shown in FIG. 14, the blocks belonging to the two block columns above the current block and the blocks belonging to the five block columns to the left of the current block are set as candidate blocks.

[0151] Merging candidates can also be derived from temporally neighboring blocks in a different image than the current block, for example, from collocated blocks in a collocated image.

[0152] The motion information of the merging candidate may be set to be the same as the motion information of the candidate block, for example, at least one of the motion vector, the reference image index, the prediction direction, or the bidirectional weight index of the candidate block may be the motion information of the merging candidate.

[0153] A merge candidate list including merge candidates may be generated (S1102). The merge candidates may be classified into adjacent merge candidates derived from adjacent blocks adjacent to the current block and non-adjacent merge candidates derived from non-adjacent blocks.

[0154] Indices of the merge candidates in the merge candidate list may be assigned according to a predetermined order. For example, indices assigned to adjacent merge candidates may have lower values ​​than indices assigned to non-adjacent merge candidates. Alternatively, an index may be assigned to each merge candidate based on the index of each block shown in FIG. 12 or FIG. 14.

[0155] If the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates may be selected (S1103). In this case, information indicating whether motion information of the current block is derived from an adjacent merge candidate may be transmitted in a signal via the bitstream. The information may be a 1-bit flag. For example, a syntax element isAdjancentMergeFlag indicating whether motion information of the current block is derived from an adjacent merge candidate may be transmitted in a signal via the bitstream. If the syntax element isAdjancentMergeFlag has a value of 1, the motion information of the current block may be derived based on the adjacent merge candidate. On the other hand, if the syntax element isAdjancentMergeFlag has a value of 0, the motion information of the current block may be derived based on a non-adjacent merge candidate.

[0156] Information for specifying one of the multiple merging candidates may be transmitted in a signal via the bitstream, for example, information indicating an index of one of the merging candidates included in the merging candidate list may be transmitted in a signal via the bitstream.

[0157] When isAdjacentMergeflag is 1, a signal may be used to transmit a syntax element merge_idx for determining one of the adjacent merge candidates. The maximum value of the syntax element merge_idx may be a value whose difference value from the number of adjacent merge candidates is 1.

[0158] If isAdjacentMergeflag is 0, the signal may be used to send a syntax element NA_merge_idx for determining one of the non-adjacent merge candidates. The syntax element NA_merge_idx indicates a value obtained by subtracting the index of the non-adjacent merge candidate from the number of adjacent merge candidates. The decoder may select the non-adjacent merge candidate by adding the number of adjacent merge candidates to the index based on NA_merge_idx.

[0159] If the number of merging candidates included in the merging candidate list is less than a threshold, merging candidates included in the inter motion information list may be added to the merging candidate list, where the threshold may be the maximum number of merging candidates included in the merging candidate list or the maximum number of merging candidates minus an offset value. The offset value may be an integer such as 1 or 2. The inter motion information list may include merging candidates derived based on blocks encoded / decoded before the current block.

[0160] The inter motion information list includes merge candidates derived from blocks in a current image that are coded / decoded based on inter prediction. For example, the motion information of the merge candidates included in the inter motion information list may be set to be equal to the motion information of the blocks that are coded / decoded based on inter prediction. Here, the motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weight index.

[0161] For ease of interpretation, the merge candidates included in the inter motion information list are called inter merge candidates.

[0162] The encoder and decoder can predefine the maximum number of merging candidates that can be included in the inter motion information list, for example, the maximum number of merging candidates that can be included in the inter motion information list can be 1, 2, 3, 4, 5, 6, 7, 8, or more (e.g., 16).

[0163] Alternatively, information can be signaled via the bitstream indicating the maximum number of merging candidates that can be included in the inter motion information list, said information being signaled at a sequence level, a picture level or a slice level.

[0164] Alternatively, the maximum number of merge candidates included in the inter motion information list can be determined according to the size of the image, the size of the slice, or the size of the coding tree unit.

[0165] The inter motion information list may be initialized in units of an image, a slice, a tile, a brick, a coding tree unit, or a coding tree unit line (row or column). For example, when a slice is initialized, the inter motion information list is also initialized, and the inter motion information list may not include any merge candidates.

[0166] Alternatively, a signal may be sent via the bitstream indicating whether to initialize the inter motion information list. The information may be sent by a signal at a slice level, a tile level, a brick level, or a block level. The inter motion information list located before the information indicates the initialization of the inter motion information list may be used.

[0167] Alternatively, information regarding inter merge candidates can be signaled via a picture parameter set or slice header. A slice may be initialized and an initial inter merge candidate may be included in the inter motion information list. Thus, the inter merge candidate is used for the earliest coded / decoded block in the slice.

[0168] The blocks are coded / decoded according to the coding / decoding order. Furthermore, the blocks coded / decoded based on inter prediction may be set as prediction region merging candidates according to the coding / decoding order.

[0169] FIG. 15 is a flowchart showing update information of the inter motion information list.

[0170] When performing inter prediction on a current block (S1501), an inter merge candidate may be derived based on the current block (S1502). Motion information of the inter merge candidate may be set equal to the motion information of the current block.

[0171] If the inter motion information list is empty (S1503), an inter merge candidate derived based on the current block may be added to the inter motion information list (S1504).

[0172] If the inter merge candidate is included in the inter motion information list (S1503), redundancy detection may be performed on the motion information of the current block (or on the inter merge candidate derived based on the current block) (S1505). The redundancy detection is used to determine whether the motion information of the inter merge candidate stored in the inter motion information list is the same as the motion information of the current block. Redundancy detection may be performed on all inter merge candidates stored in the inter motion information list. Alternatively, redundancy detection may be performed on inter merge candidates whose indexes are greater than or less than a threshold value among the inter merge candidates stored in the inter motion information list.

[0173] If the inter merge candidate does not include an inter merge candidate having motion information that is the same as the motion information of the current block, the inter merge candidate derived based on the current block may be added to the inter motion information list (S1508). It may be determined whether the inter merge candidates are the same based on whether their motion information (e.g., motion vectors and / or reference image indexes, etc.) is the same.

[0174] In this case, if the maximum number of inter merge candidates is stored in the inter motion information list (S1506), the oldest inter merge candidate is deleted (S1507), and an inter merge candidate derived based on the current block can be added to the inter motion information list (S1508).

[0175] Any of the multiple inter merge candidates may be labeled with a respective index. When adding an inter merge candidate derived from the current block to the inter motion information list, the smallest index (e.g., 0) may be assigned to the inter merge candidate, and the index of the stored inter merge candidate may be increased by 1. In this case, when the maximum number of inter prediction merge candidates is stored in the inter motion information list, the inter merge candidate with the highest index is removed.

[0176] Alternatively, when an inter merge candidate derived from the current block is added to the inter motion information list, the largest index may be assigned to the inter merge candidate. For example, if the number of inter merge candidates stored in the inter motion information list is less than a maximum value, an index whose value is equal to the number of stored inter merge candidates may be assigned to the inter merge candidate. Alternatively, if the number of inter merge candidates stored in the inter motion information list is equal to the maximum value, an index obtained by subtracting 1 from the maximum value may be assigned to the inter merge candidate. In addition, the inter merge candidate with the smallest index is deleted. Also, the indexes of the remaining stored inter merge candidates may be decreased by 1.

[0177] FIG. 16 is a diagram illustrating an embodiment of updating the inter-merge candidate list.

[0178] Assume that an intermerge candidate derived from the current block is added to the intermerge candidate list and the highest index is assigned to the intermerge candidate. Assume also that the maximum number of intermerge candidates is stored in the intermerge candidate list.

[0179] When adding the inter merge candidate HmvpCand[n+1] derived from the current block to the prediction region merge candidate list HmvpCandList, the inter merge candidate HmvpCand[0] with the smallest index among the stored inter merge candidates is deleted. Also, the indexes of the remaining inter merge candidates can be decreased by 1. Note that the index of the inter merge candidate HmvpCand[n+1] derived from the current block can be set to the maximum value (n in the example shown in FIG. 16).

[0180] If an inter merge candidate that is the same as the inter merge candidate derived based on the current block is stored (S1505), the inter merge candidate derived based on the current block does not need to be added to the inter motion information list (S1509).

[0181] Alternatively, when adding an inter merge candidate derived based on the current block to the inter motion information list, a stored inter merge candidate that is the same as the inter merge candidate can be deleted, in which case the same effect as updating the index of the stored inter merge candidate can be obtained.

[0182] FIG. 17 is a diagram showing an example of updating the index of stored inter-merge candidates.

[0183] If the index of the stored inter prediction merge candidate that is the same as the inter merge candidate mvCand derived based on the current block is hIdx, the index of the stored inter merge candidate is deleted. Also, the index of the inter merge candidate whose index is greater than hIdx can be decremented by 1. For example, in the example shown in FIG. 17, it is illustrated that HmvpCand[2] that is the same as mvCand is deleted from the inter motion information list HvmpCand, and the indexes of HmvpCand[3] to HmvpCand[n] are decremented by 1, respectively.

[0184] Note that the inter merge candidate mvCand derived from the current block can be added to the end of the inter motion information list.

[0185] Alternatively, the index assigned to a stored inter merge candidate that is the same as the inter merge candidate derived based on the current block can be updated, for example, the index of the stored inter merge candidate can be changed to a minimum or maximum value.

[0186] It may be set not to add the motion information of the block included in a predetermined area to the inter motion information list. For example, the inter merge candidate derived based on the motion information of the block included in the merge processing area cannot be added to the inter motion information list. Since the encoding / decoding order of the blocks included in the merge processing area is not defined, it is inappropriate to use the motion information of any one of these blocks for the inter prediction of other blocks. Therefore, there is a possibility that the inter merge candidate derived based on the block included in the merge processing area is not added to the inter motion information list.

[0187] When motion compensation prediction is performed on a sub-block basis, an inter-merge candidate may be derived based on motion information of a representative sub-block among a plurality of sub-blocks included in a current block. For example, when a sub-block merging candidate is used for a current block, an inter-merge candidate may be derived based on motion information of a representative sub-block among the sub-blocks.

[0188] The motion vector of a sub-block may be derived in the following order. First, any one of the merging candidates included in the merging candidate list of the current block is selected, and an initial shift vector (shVector) may be derived based on the motion vector of the selected merging candidate. In addition, a shift sub-block whose reference sample position is (xColSb, yColSb) may be derived by adding the initial shift vector to the position (xSb, ySb) of a reference sample (e.g., the top left sample or the middle position sample) of each sub-block in the coding block. The following Equation 4 shows an equation for deriving the shift sub-block.

number

[0189] Next, the motion vector of the collocated block corresponding to the center position of the sub-block including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0190] The representative sub-block may refer to the sub-block that includes the top-left sample or the center sample of the current block.

[0191] FIG. 18 is a diagram showing the locations of representative sub-blocks.

[0192] 20(a) shows an example in which a sub-block located at the top left of a current block is set as a representative sub-block. FIG. 20(b) shows an example in which a sub-block located at the center of a current block is set as a representative sub-block. When motion compensation prediction is performed in units of sub-blocks, inter-merge candidates for a current block can be derived based on a motion vector of a sub-block including a top left sample of the current block or a sub-block including a center sample of the current block.

[0193] It may be determined whether to use the current block as an inter merge candidate based on the inter prediction mode of the current block. For example, a block encoded / decoded based on an affine motion model may be set as unavailable as an inter merge candidate. Thus, even if the current block is encoded / decoded by inter prediction, if the inter prediction mode of the current block is an affine prediction mode, the inter prediction motion information list is not updated based on the current block.

[0194] Alternatively, the inter-merge candidate may be derived based on at least one sub-block vector of the sub-blocks included in the block encoded / decoded based on the affine motion model. For example, the inter-merge candidate may be derived using a sub-block located at the top left of the current block, a sub-block located at the center of the current block, or a sub-block located at the top right of the current block. Alternatively, the average value of the sub-block vectors of multiple sub-blocks may be set as the motion vector of the inter-merge candidate.

[0195] Alternatively, the inter-merge candidate can be derived by the average value of the affine seed vectors of the blocks encoded / decoded based on the affine motion model. For example, the average value of at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector of the current block can be set as the motion vector of the inter-merge candidate.

[0196] Alternatively, inter motion information lists can be arranged for different inter prediction modes. For example, at least one of an inter motion information list for a block coded / decoded by intra block duplication, an inter motion information list for a block coded / decoded based on a translational motion model, and an inter motion information list for a block coded / decoded based on an affine motion model can be defined. Any one of the multiple inter motion information lists can be selected based on the inter prediction mode of the current block.

[0197] FIG. 19 is a diagram showing an example of generating inter motion information lists for different inter prediction modes.

[0198] When a block is coded / decoded based on a non-affine motion model, an inter merge candidate mvCand derived based on the block can be added to the inter non-affine motion information list HmvpCandList, whereas when a block is coded / decoded based on an affine motion model, an inter merge candidate mvAfCand derived based on the block can be added to the inter-affine motion information list HmvpAfCandList.

[0199] The affine seed vector of a block can be stored in an inter merge candidate derived from a block that is encoded / decoded based on an affine motion model, and the inter merge candidate can then be used as a merge candidate for deriving the affine seed vector of the current block.

[0200] In addition to the described inter motion information list, another inter motion information list can also be defined. In addition to the above inter motion information list (hereinafter referred to as the first inter motion information list), a long-term motion information list (hereinafter referred to as the second inter motion information list) can also be defined. Here, the long-term motion information list includes long-term merge candidates.

[0201] If both the first inter motion information list and the second inter motion information list are empty, the inter merge candidate can be added to the second inter motion information list first. Only after the number of available inter merge candidates in the second inter motion information list reaches the maximum number, the inter merge candidate can be added to the first inter motion information list.

[0202] Alternatively, one inter merge candidate can be added to both the second inter motion information list and the first inter motion information list.

[0203] In this case, the second inter motion information list that has been arranged may not be updated. Alternatively, when the decoding area is equal to or larger than a predetermined ratio of the slice, the second inter motion information list may be updated. Alternatively, the second inter motion information list may be updated every N coding tree unit lines.

[0204] On the other hand, the first inter motion information list can be updated every time a block encoded / decoded by inter prediction is generated. However, the inter merge candidate added to the second inter motion information list may be set not to be used for updating the first inter motion information list.

[0205] A signal may be sent via the bitstream to transmit information for selecting one of the first inter motion information list and the second inter motion information list, and if the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter motion information list indicated by the information may be added to the merge candidate list.

[0206] Alternatively, the inter motion information list may be selected based on the size and shape of the current block, the inter prediction mode, whether bidirectional prediction is enabled, whether motion vector refinement is enabled, or whether triangulation is enabled.

[0207] Alternatively, if an inter merge candidate included in the first inter motion information list is added but the number of merge candidates included in the merge candidate list is less than the maximum number of merge candidates, an inter merge candidate included in the second inter motion information list can be added to the merge candidate list.

[0208] FIG. 20 is a diagram showing an example of adding an inter-merge candidate included in the long-term motion information list to the merge candidate list.

[0209] If the number of merge candidates included in the merge candidate list is less than the maximum number, the inter merge candidates included in the first inter motion information list HmvpCandList can be added to the merge candidate list. If the number of merge candidates included in the merge candidate list is also less than the maximum number even after the inter merge candidates included in the first inter motion information list are added to the merge candidate list, the inter merge candidates included in the long term motion information list HmvpLTCandList can be added to the merge candidate list.

[0210] The inter merge candidates may be set to include additional information in addition to the motion information. For example, the size, shape, or division information of the block may be added to the inter merge candidates and stored. When constructing a merge candidate list for the current block, only inter merge candidates whose size, shape, or division information is the same or similar to that of the current block are used among the inter merge candidates. Alternatively, inter merge candidates whose size, shape, or division information is the same or similar to that of the current block may be preferentially added to the merge candidate list.

[0211] Alternatively, an inter motion information list may be generated for each of the size, shape, or partition information of a block. A merge candidate list for the current block may be generated by using an inter motion information list corresponding to the shape, size, or partition information of the current block among the multiple inter motion information lists.

[0212] If the number of merge candidates included in the merge candidate list of the current block is less than a threshold, the inter merge candidates included in the inter motion information list may be added to the merge candidate list. The adding process may be performed according to ascending or descending order of indexes. For example, the inter merge candidate with the largest index may be added to the merge candidate list first.

[0213] When it is desired to add an inter merge candidate included in the inter motion information list to the merge candidate list, redundancy detection can be performed between the inter merge candidate and multiple merge candidates stored in the merge candidate list.

[0214] Redundancy detection can be performed only on some inter merge candidates included in the inter motion information list. For example, redundancy detection can be performed only on inter merge candidates whose index is equal to or greater than a threshold. Alternatively, redundancy detection can be performed only on the N merge candidates with the highest indexes or the N merge candidates with the lowest indexes.

[0215] Alternatively, redundancy detection may be performed on only a portion of the merge candidates stored in the merge candidate list, for example, on only merge candidates whose index is greater than or equal to a threshold or less than or equal to a threshold, or on merge candidates derived from a block at a particular location, where the particular location may include at least one of the left neighboring block, the above neighboring block, the top-right neighboring block, or the bottom-left neighboring block of the current block.

[0216] FIG. 21 is a diagram showing an example in which redundancy detection is performed on only some of the merging candidates.

[0217] When it is desired to add an inter-merge candidate HmvpCand[j] to the merge candidate list, a redundancy detection can be performed on the inter-merge candidate and the two merge candidates with maximum indices mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], where NumMerge can represent the number of available spatial and temporal merge candidates.

[0218] Different from the illustrated example, when it is desired to add an inter merge candidate HmvpCand[j] to the merge candidate list, redundancy detection can be performed on the inter merge candidate and the two merge candidates with the smallest index. For example, it can be checked whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, redundancy detection can be performed only on merge candidates derived from a specific position. For example, redundancy detection can be performed only on at least one of the merge candidates derived from the neighboring block located to the left of the current block or the merge candidates derived from the neighboring block located above the current block. If there is no merge candidate derived from a specific position in the merge candidate list, the inter merge candidate can be added to the merge candidate list without performing redundancy detection.

[0219] When finding a merge candidate that is the same as a first intermerge candidate and performing a redundant detection on a second intermerge candidate, the redundant detection of the merge candidate that is the same as the first intermerge candidate can be skipped.

[0220] FIG. 22 is a diagram showing an example of skipping redundancy detection for a specific merging candidate.

[0221] When it is desired to add an inter merge candidate HmvpCand[i] with index i to the merge candidate list, redundancy detection is performed between the inter merge candidate and the merge candidates stored in the merge candidate list. In this case, if a merge candidate mergeCandList[j] is found that is the same as the inter merge candidate HmvpCand[i], redundancy detection can be performed between the inter merge candidate HmvpCand[i-1] with index i-1 and the merge candidates without adding the inter merge candidate HmvpCand[i] to the merge candidate list. In this case, redundancy detection can be skipped between the inter merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j].

[0222] For example, in the example shown in Figure 22, it is determined that HmvpCand[i] and mergeCandList[2] are the same. This allows HmvpCand[i] to not be added to the merge candidate list, and redundancy detection to be performed on HmvpCand[i-1]. In this case, redundancy detection between HmvpCand[i-1] and mergeCandList[2] can be skipped.

[0223] If the number of merge candidates included in the merge candidate list of the current block is less than a threshold, in addition to the inter merge candidate, it may further include at least one of a paired merge candidate and a zero merge candidate. A paired merge candidate is a merge candidate whose motion vector is the average of the motion vectors of two or more merge candidates, and a zero merge candidate is a merge candidate whose motion vector is 0.

[0224] Merge candidates can be added to the current block's merge candidate list in the following order:

[0225] Spatial merge candidates - Temporal merge candidates - Inter merge candidates - (Inter affine merge candidates) - Paired merge candidates - Zero merge candidates The spatial merge candidates are merge candidates derived from at least one of adjacent blocks or non-adjacent blocks, and the temporal merge candidates are merge candidates derived from an immediately preceding reference image. The prediction domain affine merge candidates represent prediction domain merge candidates derived from blocks encoded / decoded by an affine motion model.

[0226] The inter motion information list may be used in a motion vector prediction mode. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold, the inter merge candidate included in the inter motion information list can be set as the motion vector prediction candidate related to the current block. Specifically, the motion vector of the inter merge candidate can be set as the motion vector prediction candidate.

[0227] When any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as the motion vector prediction value of the current block. Then, the motion vector residual value of the current block is decoded, and the motion vector prediction value and the motion vector residual value are added to obtain the motion vector of the current block.

[0228] The motion vector prediction candidate list for the current block can be constructed according to the following order:

[0229] Spatial motion vector prediction candidate-Temporal motion vector prediction candidate-Inter decoding region merge candidate-(Inter decoding region affine merge candidate)-Zero motion vector prediction candidate The spatial motion vector prediction candidate is a motion vector prediction candidate derived from at least one of adjacent blocks or non-adjacent blocks, and the temporal motion vector prediction candidate is a motion vector prediction candidate derived from an immediately preceding reference image. The inter-affine merge candidate represents an inter motion vector prediction candidate derived from a block encoded / decoded by an affine motion model. The zero motion vector prediction candidate represents a candidate whose motion vector value is 0.

[0230] A coding block may be divided into multiple prediction units, and prediction may be performed on each of the divided prediction units, where a prediction unit represents a basic unit for prediction.

[0231] The coding block may be divided by at least one of vertical lines, horizontal lines, diagonal lines, or diagonal lines. Information for determining at least one of the number, angle, or position of lines dividing the coding block may be transmitted in a signal via the bit stream. For example, information representing any one of the division type candidates of the coding block may be transmitted in a signal via the bit stream, or information specifying any one of a plurality of line candidates dividing the coding block may be transmitted in a signal via the bit stream. Alternatively, information for determining the number or type of line candidates dividing the coding block may be transmitted in a signal via the bit stream. For example, a one-bit flag may be used to determine whether a diagonal line with an angle larger than a diagonal and / or a diagonal line with an angle smaller than a diagonal can be used as a line candidate.

[0232] Alternatively, at least one of the number, angle or position of lines dividing the coding block may be adaptively determined based on at least one of the intra prediction mode, inter prediction mode, position of available merging candidates or division type of adjacent blocks of the coding block.

[0233] When a coding block is divided into multiple prediction units, intra prediction or inter prediction can be performed on the divided multiple prediction units.

[0234] FIG. 23 is a diagram showing an example of dividing a coding block into a plurality of prediction units using diagonals.

[0235] In the examples shown in Figures 23(a) and 23(b), the coding block can be divided into two triangular prediction units using a diagonal.

[0236] In Figures 23(a) and 23(b), the division of the coding block into two prediction units is illustrated using a diagonal line connecting two vertices of the coding block. However, the coding block can be divided into two prediction units using a diagonal line, at least one of whose ends does not pass through a vertex of the coding block.

[0237] FIG. 24 is a diagram showing an example of dividing a coding block into two prediction units.

[0238] For example, in the examples shown in Figures 24(a) and 24(b), the coding block can be divided into two prediction units using a diagonal line whose ends touch the upper and lower boundaries of the coding block, respectively.

[0239] Alternatively, in the examples shown in Figures 24(c) and 24(d), the coding block can be divided into two prediction units using a diagonal line whose ends touch the left and right boundaries of the coding block, respectively.

[0240] Alternatively, the coding block can be divided into two prediction blocks of different sizes. For example, the diagonal line dividing the coding block is set to contact two boundary surfaces that form one vertex. This allows the coding block to be divided into two prediction units of different sizes.

[0241] FIG. 25 is a diagram showing an example of dividing a coding block into a plurality of prediction blocks having different sizes.

[0242] For example, in the examples shown in Figures 25(a) and 25(b), the diagonal line connecting at the upper left or lower right corner of the coding block can be set to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block without passing through the upper left or lower right corner of the coding block, thereby dividing the coding block into two prediction units having different sizes.

[0243] Alternatively, in the examples shown in Figures 25(c) and 25(d), the coding block can be divided into two prediction units having different sizes by setting the diagonal line connecting at the upper right corner or lower left corner of the coding block to pass through the left boundary, right boundary, upper boundary or lower boundary of the coding block without passing through the upper left corner or lower right corner of the coding block.

[0244] Each prediction unit generated by the division of the coding is called "Nth prediction unit". For example, in the examples shown in Figures 23 to 25, PU1 may be defined as the first prediction unit, and PU2 may be defined as the second prediction unit. The first prediction unit may be a prediction unit including a sample located at the lower left or upper left of the coding block, and the second prediction unit may be a prediction unit including a sample located at the upper right or lower right of the coding block.

[0245] Conversely to the above, a prediction unit including a sample located at the top right or bottom right of a coding block can be defined as a first prediction unit, and a prediction unit including a sample located at the bottom left or top left of a coding block can be defined as a second prediction unit.

[0246] In the following embodiment, the example of division using a diagonal line is mainly described. In particular, the process of dividing a coding block into two prediction units using a diagonal line is called diagonal division or triangular division, and the prediction unit generated based on the diagonal division is called triangular prediction unit. However, in the following embodiment, division can also be performed using a diagonal line whose angle is different from the vertical line, horizontal line or diagonal angle.

[0247] It may be determined whether to apply a diagonal split to a coding block based on at least one of the slice type, the maximum number of merge candidates that can be included in a merge candidate list, the size of the coding block, the shape of the coding block, the predictive coding mode of the coding block, or the split type of the parent node.

[0248] For example, it may be determined whether to apply diagonal splitting to a coding block based on whether the current slice is of type B. Diagonal splitting is only permissible if the current slice is of type B.

[0249] Alternatively, it may be determined whether to apply a diagonal split to a coding block based on whether the maximum number of merging candidates in the merging candidate list is greater than or equal to 1. A diagonal split is permissible only if the maximum number of merging candidates in the merging candidate list is greater than or equal to 1.

[0250] Alternatively, there is a drawback that when at least one of the width or height of the hardware is greater than 64, the data processing unit with a size of 64×64 is accessed redundantly. Therefore, when at least one of the width or height of the coding block is greater than a threshold, the coding block may not be divided into multiple prediction blocks. For example, when at least one of the height and width of the coding block is greater than 64 (for example, when at least one of the width and height is 128), diagonal division may not be used.

[0251] Alternatively, considering the maximum number of samples that are processed simultaneously in a hardware implementation, diagonal splitting may not be used for coding blocks with a number of samples larger than a threshold, e.g., diagonal splitting may not be allowed for coding tree blocks with a number of samples larger than 4096.

[0252] Alternatively, diagonal division may not be allowed for a coding block that contains less than a threshold number of samples, for example, if the number of samples contained in the coding block is less than 64, diagonal division may not be applied to the coding block.

[0253] Alternatively, it may be determined whether to apply diagonal division to the coding block based on whether the ratio of the width to the height of the coding block is less than a first threshold or whether the ratio of the width to the height of the coding block is greater than a second threshold. Here, the ratio of the width to the height of the coding block, whRatio, may be determined as the ratio of the width CbW to the height CbH of the coding block. For example, as shown in the following Equation 5.

number

[0254] The second threshold may be the reciprocal of the first threshold, for example, if the first threshold is k, the second threshold may be 1 / k.

[0255] Diagonal splitting can be applied to the coding only if the ratio of the width to the height of the coding block is between a first threshold and a second threshold.

[0256] Alternatively, triangular partitioning may be used only if the ratio of width to height of the coding block is less than a first threshold or greater than a second threshold, e.g., if the first threshold is 16, diagonal partitioning may not be allowed for coding blocks of size 64x4 or 4x64.

[0257] Alternatively, it may be determined whether or not diagonal splitting is allowed based on the split type of the parent node. For example, if the coding block of the parent node is split based on quadtree splitting, diagonal splitting may be applied to the coding block of the leaf node. On the other hand, if the coding block of the parent node is split based on quadtree splitting or ternary tree splitting, the coding block of the leaf node is set to not allow diagonal splitting.

[0258] Alternatively, it may be determined whether to allow diagonal splitting based on the predictive coding mode of the coding block. For example, if the coding block is coded in an intra prediction mode, diagonal splitting may be allowed only if the coding block is coded in an intra prediction mode or if the coding block is coded in a predefined inter prediction mode. Here, the predefined inter prediction mode may represent at least one of a merge mode, an advanced motion vector prediction mode, an affine merge mode, or an affine motion vector prediction mode.

[0259] Alternatively, it may be possible to determine whether to allow diagonal splitting based on the size of the parallel processing region. For example, if the size of the coding block is larger than the size of the parallel processing region, diagonal splitting may not be used.

[0260] Two or more of the above listed conditions may be taken into consideration in determining whether to apply diagonal splitting to a coding block.

[0261] As another example, a signal may be sent via the bitstream indicating whether a diagonal partition is applied to a coding block. The information may be sent by a signal at a sequence level, an image level, a slice level, or a block level. For example, at the coding block level, a signal may send a flag triangle_partition_flag indicating whether triangulation is applied to the coding block.

[0262] If it is decided to apply a diagonal division to the coding block, then information can be transmitted via the bitstream in a signal indicating the number of lines or the position of the lines dividing the coding block.

[0263] For example, when a coding block is divided by a diagonal, information indicating the direction of the diagonal dividing the coding block may be transmitted by a signal via the bitstream. For example, a flag triangle_partition_type_flag indicating the direction of the diagonal may be transmitted by a signal via the bitstream. The flag indicates whether the coding block is divided by a diagonal connecting the upper left and lower right or by a diagonal connecting the upper right and lower left.

[0264] When a coding block is divided by a diagonal connecting the upper left and lower right, it may be called a left triangular division type. When a coding block is divided by a diagonal connecting the upper right and lower left, it may be called a right triangular division type. For example, when the value of the flag is 0, it indicates that the division type of the coding block is a left triangular division type, and when the value of the flag is 1, it indicates that the division type of the coding block is a right triangular division type.

[0265] It should be noted that, via the bitstream, a signal may transmit information indicating whether the sizes of the prediction units are the same or information indicating the position of a diagonal for dividing the coding block. For example, if the information indicating the size of the prediction units indicates that the sizes of the prediction units are the same, the coding for the information indicating the position of the diagonal may be skipped, and the coding block may be divided into two prediction units using a diagonal passing through two vertices of the coding block. On the other hand, if the information indicating the size of the prediction units indicates that the sizes of the prediction units are different, the position of the diagonal for dividing the coding block may be determined based on the information indicating the position of the diagonal. For example, when a left triangular division type is applied to the coding block, the position information may indicate whether the diagonal touches the left and bottom boundaries or the top and right boundaries of the coding block. Alternatively, when a right triangular division type is applied to the coding block, the position information may indicate whether the diagonal touches the right and bottom boundaries or the top and left boundaries of the coding block.

[0266] Information indicative of the partition type of a coding block may be signaled at the coding block level, allowing the partition type to be determined for different coding blocks to which diagonal partitioning is applied.

[0267] As another example, a signal may transmit information indicating a partition type for a sequence, image, slice, tile, or coding tree unit, where the partition type of a coding block to which a diagonal partition is applied may be set in the same way in the sequence, image, slice, tile, or coding tree unit.

[0268] Alternatively, the first coding unit in the coding tree unit that applies diagonal partitioning is coded with information for determining the partition type and signaled, and the second and subsequent coding units that apply diagonal partitioning are configured to use the same partition type as the first coding unit.

[0269] As another example, the partition type of the coding block may be determined based on the partition type of the neighboring block. Here, the neighboring block may include at least one of the neighboring block adjacent to the upper left corner of the coding block, the neighboring block adjacent to the upper right corner, the neighboring block adjacent to the lower left corner, the neighboring block located above, or the neighboring block located to the left. For example, the partition type of the current block may be set to be the same as the partition type of the neighboring block. Alternatively, the partition type of the current block may be determined based on whether to apply a left triangular partition type to the upper left neighboring block, and whether to apply a right triangular partition type to the upper right neighboring block or the lower left neighboring block.

[0270] In order to perform motion prediction compensation on the first triangular prediction unit and the second triangular prediction unit, motion information of each of the first triangular prediction unit and the second triangular prediction unit may be derived. In this case, the motion information of the first triangular prediction unit and the second triangular prediction unit may be derived from merge candidates included in the merge candidate list. In order to distinguish between a general merge candidate list and a merge candidate list used when deriving motion information of a triangular prediction unit, a merge candidate list for deriving motion information of a triangular prediction unit is called a triangle merge candidate list, and a merge candidate included in the triangle merge candidate list is called a triangle merge candidate. However, the present invention also includes the use of the merge candidate derivation method and the merge candidate list construction method for triangle merge candidates and a triangle merge candidate list construction direction.

[0271] Through the bitstream, a signal may transmit information for determining a maximum number of triangle merge candidates that may be included in a triangle merge candidate list, and the information may represent a difference between the maximum number of merge candidates that may be included in a triangle merge candidate list and the maximum number of triangle merge candidates that may be included in a triangle merge candidate list.

[0272] Triangle merging candidates can be derived from the spatial and temporal neighboring blocks of a coding block.

[0273] FIG. 26 is a diagram showing adjacent blocks for deriving triangle merging candidates.

[0274] The triangle merging candidates can be derived using at least one of the neighboring blocks located above the coding block, the neighboring blocks located to the left of the coding block, or the co-located blocks included in a different image from the coding block. The upper neighboring blocks may include at least one of the following blocks: a block including a sample (xCb+CbW-1, yCb-1) located above the coding block, a block including a sample (xCb+CbW, yCb-1) located above the coding block, or a block including a sample (xCb-1, yCb-1) located above the coding block. The left neighboring blocks may include at least one of the following blocks: a block including a sample (xCb-1, yCb+CbH-1) located to the left of the coding block, or a block including a sample (xCb-1, yCb + CbH) located to the left of the coding block. The collocated block may be determined to be either a block containing samples (xCb+CbW, yCb+CbH) adjacent to the upper right corner of the coding block in the collocated image or a block containing samples (xCb / 2, yCb / 2) located at the center of the coding block.

[0275] The neighboring blocks may be searched according to a predefined order, and the triangle merge candidate list may be formed from the triangle merge candidates according to the predefined order. For example, the triangle merge candidates may be searched according to the order of B1, A1, B0, A0, C0, B2, and C1 to form the triangle merge candidate list.

[0276] The motion information of the triangular prediction units may be derived based on the triangle merging candidate list, that is, the triangular prediction units may share one triangle merging candidate list.

[0277] To derive the motion information of the triangle merging unit, information for specifying at least one of the triangle merging candidates included in the triangle merging candidate list may be sent in a signal via the bitstream, for example, index information merge_triangle_idx for specifying at least one of the triangle merging candidates may be sent in a signal via the bitstream.

[0278] The index information may specify a combination of merging candidates of the first triangular prediction unit and the second triangular prediction unit. For example, Table 1 below shows an example of a combination of merging candidates based on index information merge_triangle_idx. [Table 1]

[0279] When the value of the index information merge_triangle_idx is 1, it indicates that the motion information of the first triangular prediction unit is derived from the merge candidate with index 1, and the motion information of the second triangular prediction unit is derived from the merge candidate with index 0. The index information merge_triangle_idx makes it possible to determine the triangle merge candidate for deriving the motion information of the first triangular prediction unit and the triangle merge candidate for deriving the motion information of the second triangular prediction unit.

[0280] The partition type of the coding block to which the diagonal partition is applied can also be determined based on the index information. That is, the index information can specify a combination of merge candidates of the first triangular prediction unit, merge candidates of the second triangular prediction unit, and a partition direction of the coding block. When the partition type of the coding block is determined based on the index information, it is not necessary to code the information triangle_partition_type_flag indicating the direction of the diagonal line that divides the coding block. Table 2 shows the partition type of the coding block related to the index information merge_triangle_idx. [Table 2]

[0281] When the variable TriangleDir is 0, it indicates that a left triangular partition type is applied to the coding block. When the variable TriangleDir is 1, it indicates that a right triangular partition type is applied to the coding block. By combining Table 1 and Table 2, it may be set so as to specify a combination of merging candidates for the first triangular prediction unit, merging candidates for the second triangular prediction unit, and the division direction of the coding block based on the index information merge_triangle_idx.

[0282] As another example, index information used for only one of the first triangular prediction unit and the second triangular prediction unit may be transmitted by a signal. In addition, an index used for the other triangle merge candidate of the first triangular prediction unit and the second triangular prediction unit may be determined based on the index information. For example, a triangle merge candidate of the first triangular prediction unit may be determined based on index information merge_triangle_idx representing an index of one of the triangle merge candidates. In addition, a triangle merge candidate of the second triangular prediction unit may be specified based on the merge_triangle_idx. For example, a triangle merge candidate of the second triangular prediction unit may be derived by adding or subtracting an offset value to or from the index information merge_triangle_idx. The offset value may be an integer such as 1 or 2. For example, a triangle merge candidate of the second triangular prediction unit may be determined as a triangle merge candidate having an index obtained by adding 1 to merge_traingle_idx. When merge_triangle_idx indicates the triangle merge candidate with the largest index value among the triangle merge candidates, motion information of the second triangular prediction unit can be derived from the triangle merge candidate with index 0 or the triangle merge candidate whose index is the value obtained by subtracting 1 from merge_triangle_idx.

[0283] Alternatively, the motion information of the second triangular prediction unit may be derived from a triangle merge candidate having a reference image that is the same as the triangle merge candidate of the first triangular prediction unit specified by the index information. Here, the triangle merge candidate having a reference image that is the same as the triangle merge candidate of the first triangular prediction unit may represent at least one of an L0 reference image or an L1 reference image having the triangle merge candidate of the first triangular prediction unit. When there are multiple triangle merge candidates having the same reference image as the triangle merge candidate of the first triangular prediction unit, one of them may be selected based on whether the merge candidate includes at least one of bidirectional motion information or a difference value between the index of the merge candidate and the index information.

[0284] As another example, index information may be transmitted by a signal for each of the first triangular prediction unit and the second triangular prediction unit. For example, first index information 1st_merge_idx for determining a triangle merge candidate for the first triangular prediction unit and second index information 2nd_merge_idx for determining a triangle merge candidate for the second triangular prediction unit may be transmitted by a signal via a bitstream. Motion information of the first triangular prediction unit is derived from the triangle merge candidate determined based on the first index information 1st_merge_idx, and motion information of the second triangular prediction unit is derived from the triangle merge candidate determined based on the second index information 2nd_merge_idx.

[0285] The first index information 1st_merge_idx may represent an index of any one of the triangle merge candidates included in the triangle merge candidate list. The triangle merge candidate of the first triangular prediction unit may be determined to be the triangle merge candidate indicated by the first index information 1st_merge_idx.

[0286] The triangle merge candidate indicated by the first index information 1st_merge_idx is set as unavailable as a triangle merge candidate of the second triangular prediction unit. As a result, the second index information 2nd_merge_idx of the second triangular prediction unit may indicate any one index of the remaining triangle merge candidates other than the triangle merge candidate indicated by the first index information. When the value of the second index information 2nd_merge_idx is smaller than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangular prediction unit may be determined as a triangle merge candidate having index information represented by the second index information 2nd_merge_idx. On the other hand, when the value of the second index information 2nd_merge_idx is equal to or greater than the value of the first index information 1st_merge_idx, the triangle merge candidate of the second triangular prediction unit may be determined as a triangle merge candidate having, as an index, a value obtained by adding 1 to the value of the second index information 2nd_merge_idx.

[0287] Alternatively, it may be determined whether to transmit the second index information in the signal based on the number of triangle merge candidates included in the triangle merge candidate list. For example, if the maximum number of triangle merges included in the triangle merge candidate list is less than or equal to 2, the second index information may be skipped. If the second index information is skipped, the second triangle merge candidate may be derived by adding or subtracting an offset value to the first index information. For example, if the maximum number of triangle merge candidates included in the triangle merge candidate list is 2 and the first index information is index 0, the second triangle merge candidate may be derived by adding 1 to the first index information. Alternatively, if the maximum number of triangle merge candidates included in the triangle merge candidate list is 2 and the first index information is 1, the second triangle merge candidate may be derived by subtracting 1 from the first index information.

[0288] Alternatively, when skipping transmitting the second index information in the signal, the second index information can be a default value, where the default value may be 0. The second triangle merging candidate can be derived by comparing the first index information with the second index information. For example, when the second index information is smaller than the first index information, the merging candidate with index 0 is determined as the second triangle merging candidate, and when the second index information is equal to or larger than the first index information, the merging candidate with index 1 is determined as the second triangle merging candidate.

[0289] If the triangle merge candidate has one-way motion information, the one-way motion information of the triangle merge candidate is the motion information of the triangle prediction unit. On the other hand, if the triangle merge candidate has two-way motion information, only one of the L0 motion information or the L1 motion information is the motion information of the triangle prediction unit. It can be determined whether to obtain the L0 motion information or the L1 motion information according to the index of the triangle merge candidate or the motion information of another triangle prediction unit.

[0290] For example, when the index of the triangle merge candidate is an even number, the L0 motion information of the triangle prediction unit is set to 0, and the L1 motion information of the triangle merge candidate is set to the L1 motion information of the triangle prediction unit. On the other hand, when the index of the triangle merge candidate is an odd number, the L1 motion information of the triangle prediction unit is set to 0, and the L0 motion information of the triangle merge candidate is set to 0. Conversely, when the index of the triangle merge candidate is an even number, the L0 motion information of the triangle merge candidate can be set to the L0 motion information of the triangle prediction unit, and when the index of the triangle merge candidate is an odd number, the L1 motion information of the triangle merge candidate can be set to the L1 motion information of the triangle prediction unit. Alternatively, when it comes to the first triangle prediction unit, when the triangle merge candidate is an even number, the L0 motion information of the triangle merge candidate is set to the L0 motion information of the first triangle prediction unit. On the other hand, when it comes to the second triangle prediction unit, when the triangle merge candidate is an odd number, the L1 motion information of the triangle merge candidate is set to the L1 motion information of the second triangle prediction unit.

[0291] Alternatively, if the first triangle prediction unit has L0 motion information, the L0 motion information of the second triangle prediction unit is set to 0, and the L1 motion information of the triangle merging candidate is set to the L1 information of the second triangle prediction unit; on the other hand, if the first triangle prediction unit has L1 motion information, the L1 motion information of the second triangle prediction unit is set to 0, and the L0 motion information of the triangle merging candidate is set to the L0 motion signal of the second triangle prediction unit.

[0292] The triangle merging candidate list for deriving the motion information of the first triangular prediction unit and the triangle merging candidate list for deriving the motion information of the second triangular prediction unit may also be set differently.

[0293] For example, when a triangle merge candidate for deriving motion information of the first triangular prediction unit in a triangle merge candidate list is specified based on index information related to the first triangular prediction unit, the motion information of the second triangular prediction unit may be derived using a triangle merge candidate list including the remaining triangle merge candidates other than the triangle merge candidate indicated by the index information. Specifically, the motion information of the second triangular prediction unit may be derived from any one of the remaining triangle merge candidates.

[0294] As a result, the maximum number of triangle merge candidates included in the triangle merge candidate list of the first triangular prediction unit is different from the maximum number of triangle merge candidates included in the triangle merge candidate list of the second triangular prediction unit. For example, when the triangle merge candidate list of the first triangular prediction unit includes M merge candidates, the triangle merge candidate list of the second triangular prediction unit may include M-1 merge candidates other than the triangle merge candidate indicated by the index information of the first triangular prediction unit.

[0295] As another example, merging candidates for each triangular prediction unit can be derived from neighboring blocks adjacent to the coding block, and the availability of the neighboring blocks can be determined by taking into account the shape or position of the triangular prediction unit.

[0296] FIG. 27 is a diagram illustrating an example for determining the availability of neighboring blocks for each triangular prediction unit.

[0297] An adjacent block that is not adjacent to the first triangular prediction unit can be set as an adjacent block that is unavailable for the first triangular prediction unit, and an adjacent block that is not adjacent to the second triangular prediction unit can be set as an adjacent block that is unavailable for the second triangular prediction unit.

[0298] 27(a), when the left triangulation type is applied to the coding block, it may be determined that, among the adjacent blocks adjacent to the coding block, blocks A1, A0, and A2 adjacent to the first triangular prediction unit can be used for the first triangular prediction unit, and blocks B0 and B1 cannot be used for the first triangular prediction unit. As a result, the triangle merge candidate list for the first triangular prediction unit includes triangle merge candidates derived from blocks A1, A0, and A2, but does not include corner merge candidates derived from blocks B0 and B1.

[0299] 27(b), when the left triangulation type is applied to the coding block, it may be determined that blocks B0 and B1 adjacent to the second triangular prediction unit can be used for the second triangular prediction unit, and blocks A1, A0, and A2 cannot be used for the second triangular prediction unit, so that the triangle merge candidate list for the second triangular prediction unit includes triangle merge candidates derived from blocks B0 and B1, but does not include triangle merge candidates derived from blocks A1, A0, and A2.

[0300] This makes it possible to determine the number of triangle merging candidates or the range of triangle merging candidates available to a triangular prediction unit based on at least one of the position of the triangular prediction unit or the division type of the coding block.

[0301] As another example, a merge mode may be applied to only one of the first triangular prediction unit and the second triangular prediction unit, and motion information of the other of the first triangular prediction unit and the second triangular prediction unit may be set to be the same as the motion information of the triangular prediction unit to which the merge mode is applied, or may be derived by refining the motion information of the triangular prediction unit to which the merge candidate is applied.

[0302] For example, a motion vector and a reference image index of the first triangular prediction unit can be derived based on the triangle merging candidates. A motion vector of the second triangular prediction unit can be derived by performing refinement on the motion information of the first triangular prediction unit. For example, a motion vector of the second triangular prediction unit can be derived by adding or subtracting a fine motion vector {Rx, Ry} to or from a motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. A reference image index of the second triangular prediction unit can be set to be the same as a reference image index of the first triangular prediction unit.

[0303] The signal may transmit information for determining a fine motion vector representing a difference between a motion vector of the first triangular prediction unit and a motion vector of the second triangular prediction unit via the bitstream, the information including at least one of information representing a magnitude of the fine motion vector or information representing a symbol of the fine motion vector.

[0304] Alternatively, the fine motion vector symbols may be derived based on at least one of the position, index, or partition type applied to the coding block of the triangular prediction unit.

[0305] As another example, a signal may transmit a motion vector and a reference image index of one of the first triangular prediction unit and the second triangular prediction unit, and the motion vector of the other of the first triangular prediction unit and the second triangular prediction unit may be derived by performing refinement on the motion vector transmitted by the signal.

[0306] For example, a motion vector and a reference image index of the first triangular prediction unit can be determined based on information transmitted in a signal in a bitstream. A motion vector of the second triangular prediction unit can be derived by performing refinement on the motion vector of the first triangular prediction unit. For example, a motion vector of the second triangular prediction unit can be derived by adding or subtracting a fine motion vector {Rx, Ry} to or from a motion vector {mvD1LXx, mvD1LXy} of the first triangular prediction unit. A reference image index of the second triangular prediction unit can be set to be the same as a reference image index of the first triangular prediction unit.

[0307] Based on the motion information of the first triangular prediction unit and the motion information of the second triangular prediction unit, a motion compensation prediction may be performed on the coding block. In this case, image quality may be degraded at the boundary between the first triangular prediction unit and the second triangular prediction unit. For example, image quality may be continuously degraded near an edge at the boundary between the first triangular prediction unit and the second triangular prediction unit. In order to reduce image quality degradation at the boundary, a prediction sample may be derived by smoothing filtering or weighted prediction.

[0308] A prediction sample for applying a diagonal division in a coding block may be derived based on a weighted sum operation between a first prediction sample obtained based on the motion information of the first triangular prediction unit and a second prediction sample obtained based on the motion information of the second triangular prediction unit. Alternatively, a prediction sample of the first triangular prediction unit may be derived from a first prediction block determined based on the motion information of the first triangular prediction unit, a prediction sample of the second triangular prediction unit may be derived from a second prediction block determined based on the motion information of the second triangular prediction unit, and a prediction sample located in a boundary area between the first triangular prediction unit and the second triangular prediction unit may be derived based on a weighted sum operation between the first prediction sample included in the first prediction block and the second prediction sample included in the second prediction block. For example, the following Equation 6 shows an example of deriving a prediction sample of the first triangular prediction unit and the second triangular prediction unit.

number

[0309] In Equation 6, P1 represents the first predicted sample, P2 represents the second predicted sample, w1 represents the weighting applied to the first predicted sample, and (1-w1) represents the weighting applied to the second predicted sample. In the example shown in Equation 6, the weighting applied to the second predicted sample can be derived by subtracting the weighting applied to the first predicted sample from a constant.

[0310] When applying a left triangulation type to a coding block, the boundary region may include prediction samples with identical x-axis and y-axis coordinates, whereas when applying a right triangulation type to a coding block, the boundary region may include prediction samples whose sum of x-axis and y-axis coordinates is greater than or equal to a first threshold and less than or equal to a second threshold.

[0311] The size of the boundary area can be determined based on at least one of the size of the coding block, the shape of the coding block, the motion information of the triangular prediction unit, the motion vector differential value of the triangular prediction unit, the output order of the reference image, or the differential value between the first prediction sample and the second prediction sample at the diagonal boundary.

[0312] Fig. 28 is a diagram showing an example of deriving a predicted sample based on a weighted sum operation of a first predicted sample and a second predicted sample. Fig. 28 shows an example of applying a left triangular partition type to a coding block, and Fig. 29 shows an example of applying a right triangular partition type to a coding block. Note that Figs. 28(a) and 29(a) show prediction patterns for the luma component, and Figs. 28(b) and 29(b) show prediction patterns for the chroma component.

[0313] In the drawings, the numbers written in the prediction samples near the boundary between the first prediction unit and the second prediction unit represent the weights applied to the first prediction sample. For example, if the number written in the prediction sample is N, a weight of N / 8 is applied to the first prediction sample, and a weight of (1-(N / 8)) is applied to the second prediction sample. In this way, the prediction samples can be derived.

[0314] In a non-boundary region, the first prediction sample or the second prediction sample may be determined as a prediction sample. With reference to the example shown in FIG. 28, in a region belonging to a first triangular prediction unit in a region where the absolute value of the difference between the x-axis coordinate and the y-coordinate is greater than a threshold, the first prediction sample derived based on the motion information of the first triangular prediction unit may be determined as a prediction sample. On the other hand, in a region belonging to a second triangular prediction unit in a region where the difference between the x-axis coordinate and the y-axis coordinate is greater than a threshold, the second prediction sample derived based on the motion information of the second triangular prediction unit may be determined as a prediction sample.

[0315] 29, in an area where the sum of the x-axis coordinate and the y-axis coordinate is less than a first threshold, a first prediction sample derived based on the motion information of a first triangular prediction unit can be determined as a prediction sample, while in an area where the sum of the x-axis coordinate and the y-axis coordinate is greater than a second threshold, a second prediction sample derived based on the motion information of a second triangular prediction unit can be determined as a prediction sample.

[0316] A threshold for determining a non-border region can be determined based on at least one of the size of the coding block, the shape of the coding block, and the color component. For example, when the threshold for the luminance component is N, the threshold for the chrominance component can be N / 2.

[0317] A predicted sample included in the boundary region may be derived based on a weighted sum operation of the first predicted sample and the second predicted sample, where weights applied to the first predicted sample and the second predicted sample may be determined based on at least one of a position of the predicted sample, a size of the coding block, a shape of the coding block, or a color component.

[0318] For example, in the example shown in Fig. 28(a), by applying the same weight to the first predicted sample and the second predicted sample, a predicted sample at a position having the same x-axis coordinate and y-axis coordinate can be derived. By applying a ratio of weights applied to the first predicted sample and the second predicted sample of (3:1) or (1:3), a predicted sample in which the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 1 can be derived. Furthermore, by applying a ratio of weights applied to the first predicted sample and the second predicted sample of (7:1) or (1:7), a predicted sample in which the absolute value of the difference between the x-axis coordinate and the y-axis coordinate is 2 can be derived.

[0319] Alternatively, in the example shown in Figure 28(b), by applying the same weighting value to the first predicted sample and the second predicted sample, a predicted sample at a position having the same x-axis coordinate and y-axis coordinate can be derived, and by setting the ratio of the weighting values ​​applied to the first predicted sample and the second predicted sample to (7:1) or (1:7), a predicted sample whose absolute value of the difference between the x-axis coordinate and the y coordinate is 1 can be derived.

[0320] For example, in the example shown in FIG. 29(a), by applying the same weighting value to the first predicted sample and the second predicted sample, it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 less than the width or height of the coding block. By setting the ratio of the weighting values ​​applied to the first predicted sample and the second predicted sample to (3:1) or (1:3), it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as the width or height of the coding block or is 2 less than the width or height of the coding block. By setting the ratio of the weighting values ​​applied to the first predicted sample and the second predicted sample to (7:1) or (1:7), it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is 1 more than the width or height of the coding block or is 3 less than the width or height of the coding block.

[0321] Alternatively, in the example shown in Fig. 29(b), by applying the same weighting value to the first predicted sample and the second predicted sample, it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is smaller than the width or height of the coding block by 1. By setting the ratio of the weighting values ​​applied to the first predicted sample and the second predicted sample to (7:1) or (1:7), it is possible to derive a predicted sample whose sum of the x-axis coordinate and the y-axis coordinate is the same as the width or height of the coding block or is smaller than the width or height of the coding block by 2.

[0322] As another example, the weights can be determined by considering the position of the prediction sample or the shape of the coding block. Equations 7 to 9 show an example of deriving the weights when applying a left triangulation type to the coding block. Equation 7 shows an example of deriving the weights to be applied to the first prediction sample when the coding block is square shaped.

number

[0323] In Equation 7, x and y represent the positions of the predicted samples. If the coding block has a non-square shape, the weight applied to the first predicted sample can be derived as shown in Equation 8 or 9. Equation 8 shows the case where the width of the coding block is greater than the height, and Equation 9 shows the case where the width of the coding block is less than the height.

number

number

[0324] When a right triangulation type is applied to a coding block, the weights applied to the first predicted sample may be determined as shown in Equations 10 to 12. Equation 10 shows an example of deriving the weights applied to the first predicted sample when the coding block is square shaped.

number

[0325] In Equation 10, CbW represents the width of the coding block. If the coding block has a non-square shape, the weighting value applied to the first predicted sample can be derived as shown in Equation 11 or 12. Equation 11 shows the case where the width of the coding block is greater than the height, and Equation 12 shows the case where the width of the coding block is smaller than the height.

number

number

[0326] In Equation 11, CbH represents the height of the coding block.

[0327] As shown in the above example, for prediction samples in the boundary region, a first prediction sample is weighted higher than a second prediction sample to derive samples included in the first triangular prediction unit, and a second prediction sample is weighted higher than the first prediction sample to derive samples included in the second triangular prediction unit.

[0328] When diagonal division is applied to a coding block, it is possible to set the coding block to not apply a combined prediction mode, which is a combination of an intra prediction mode and a merge mode.

[0329] Intra prediction is to predict a current block using reconstructed samples that have been coded / decoded around the current block. In this case, the intra prediction of the current block can use reconstructed samples before applying an in-loop filter.

[0330] The intra prediction technique includes intra prediction based on a matrix and general intra prediction considering the directionality with surrounding reconstructed samples. Information indicating the intra prediction technique of the current block may be transmitted in a signal via a bit stream. The information may be a 1-bit flag. Alternatively, the intra prediction technique of the current block may be determined based on at least one of the position, size, shape, or intra prediction technique of a neighboring block of the current block. For example, if the current block is located across an image boundary, the current block is set to not apply intra prediction based on a matrix.

[0331] Matrix-based intra prediction is a method of obtaining a prediction block of a current block based on matrix multiplication of a matrix stored in an encoder and a decoder and a reconstructed sample around the current block. Information indicating one of a plurality of stored matrices can be transmitted in a signal via a bitstream. The decoder can determine a matrix to be used for intra prediction of the current block based on the information and the size of the current block.

[0332] General intra prediction is a method of obtaining a predicted block related to a current block based on a non-angular intra prediction mode or an angular intra prediction mode.

[0333] A residual image can be derived by subtracting the original image from the predicted image. In this case, when the residual image is transformed into the frequency domain, removing high frequency components from the frequency components does not significantly degrade the subjective image quality of the video. Therefore, reducing the value of the high frequency components or setting the value of the high frequency components to 0 has the effect of improving compression efficiency without causing obvious visual distortion. In order to reflect the above characteristics, the residual image can be decomposed into two-dimensional frequency components by transforming the current block. The transformation can be performed using a transformation technique such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST).

[0334] In the residual image, the 2D image transform may not be performed for some blocks. Not performing the 2D image transform may be called a transform skip. When applying the transform skip, quantization may be applied to the residual values ​​on which the transform has not been performed.

[0335] After transforming a current block using DCT or DST, a transform can be performed again on the transformed current block. In this case, the transform based on DCT or DST is defined as a primary transform, and the process of performing a transform again on the block to which the primary transform is applied is called a secondary transform.

[0336] The main transform can be performed using any one of a number of possible transform kernels, for example, the main transform can be performed using any one of DCT2, DCT8, or DCT7.

[0337] Different transform kernels can be used for the horizontal and vertical directions. A signal can also be sent via the bitstream indicating a combination of a horizontal transform kernel and a vertical transform kernel.

[0338] The execution units of the primary transform and the secondary transform are different. For example, the primary transform can be performed on an 8×8 block, and the secondary transform can be performed on a sub-block of the transformed 8×8 block that has a size of 4×4. In this case, the transform coefficients of the excess area that is not subjected to the secondary transform can be set to 0.

[0339] Alternatively, a primary transform can be performed on the 4x4 block and a secondary transform can be performed on a region of size 8x8 that contains the transformed 4x4 block.

[0340] Via the bitstream, a signal may transmit information indicating whether or not to perform a secondary transformation.

[0341] Alternatively, a determination can be made as to whether to perform a secondary transform based on whether the horizontal and vertical transform kernels are the same. For example, a secondary transform can be performed only if the horizontal and vertical transform kernels are the same. Alternatively, a secondary transform can be performed only if the horizontal and vertical transform kernels are different.

[0342] Alternatively, the secondary transform can be performed only if the horizontal and vertical transforms apply a predefined transform kernel, for example, allowing the secondary transform only if the horizontal and vertical transforms use a DCT2 transform kernel.

[0343] Alternatively, it may be determined whether to perform the secondary transform based on the number of non-zero transform coefficients of the current block. For example, if the non-zero coefficients of the current block are less than or equal to a threshold, the secondary transform may be set not to be used. If the non-zero transform coefficients of the current block are greater than a threshold, the secondary transform may be set to be used. The secondary transform may be set to be used only if the current block is coded using intra prediction.

[0344] In the decoder, the inverse transform (second inverse transform) of the second transform can be performed, and the inverse transform (first inverse transform) of the first transform can be performed on the result. As a result of performing the second inverse transform and the first inverse transform, a residual signal of the current block can be obtained.

[0345] After performing the transform and quantization in the encoder, the decoder can obtain the residual block by inverse quantization and inverse transform. At the decoder, the predictive block and the residual block can be added to obtain the reconstructed block of the current block.

[0346] Once a reconstructed block of the current block is obtained, in-loop filtering can be used to reduce information loss that occurs in the quantization and encoding processes. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

[0347] It is within the scope of the present invention to use the embodiments described with emphasis on a decoding or encoding process in an encoding or decoding process. It is also within the scope of the present invention to modify the embodiments described in a given order in a different order than that described.

[0348] Although the embodiments have been described based on a series of steps or flow charts, this does not limit the time order of the invention. Furthermore, the steps may be executed simultaneously or in a different order, as necessary. In the above embodiments, each of the components (e.g., units, modules, etc.) constituting the block diagrams may be realized as a hardware device or software. Furthermore, a plurality of components may be combined and executed as a single hardware device or software. The embodiments may be executed in the form of program instructions. The program instructions may be executed by various computer components and recorded on a computer-readable storage medium. The computer-readable storage medium may include program instructions, data files, data structures, etc. alone or in combination. Examples of computer-readable storage media may include magnetic media such as hard disks, flexible disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices that are specifically arranged in a manner to store and execute program instructions, such as ROMs, RAMs, flash memories, etc. The hardware devices may be configured to operate as one or more software modules to execute the processes according to the present invention, or vice versa. [Industrial Applicability]

[0349] The present invention is applicable to electronic devices that perform encoding / decoding on video.

Claims

1. 1. A video decoding method, comprising: deriving a merge candidate list for the current block; deriving first motion information of a first prediction unit based on a first merging candidate included in the merging candidate list, the first merging candidate being determined based on first index information; deriving a second motion information of a second prediction unit based on a second merging candidate included in the merging candidate list, the second merging candidate being determined based on second index information; determining a prediction sample of the current block based on a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information; A video decoding method, wherein the first index information and the second index information are obtained by decoding from a bitstream, and if a value of the second index information is greater than or equal to a value of the first index information, the second merging candidate is determined to be a merging candidate having an index obtained by adding 1 to the value of the second index information.

2. 2 . The video decoding method of claim 1 , wherein if a value of the second index information is smaller than a value of the first index information, the second merging candidate is determined as the merging candidate having an index indicated by the second index information.

3. The video decoding method of claim 1 or 2, wherein whether to apply a partition-based mode to the current block is determined based on at least one of a slice type, a size of the current block, a shape of the current block, and a prediction mode of the current block.

4. The video decoding method of claim 3 , wherein whether to apply the partition-based mode to the current block is determined based on whether a ratio of a width to a height of the current block is less than a first threshold.

5. The video decoding method of claim 3 , wherein whether or not to apply a partition-based mode to the current block is determined based on information indicating whether or not to apply a partition-based mode to the current block, the information being parsed at a sequence level from the bitstream.

6. The video decoding method of claim 1 or 2, further comprising: determining a partition type of the current block based on information indicating a partition type of the current block, the information being parsed at a current block level from the bitstream.

7. 1. A video encoding method, comprising: deriving a merge candidate list for the current block; deriving first motion information of a first prediction unit based on a first merging candidate included in the merging candidate list, where the first merging candidate corresponds to first index information; determining second motion information of a second prediction unit based on a second merging candidate included in the merging candidate list, where the second merging candidate corresponds to second index information; determining a prediction sample of the current block based on a weighted sum operation of a first prediction sample derived based on the first motion information and a second prediction sample derived based on the second motion information; encoding the first index information and the second index information into a bitstream, and if a value of the second index information is greater than or equal to a value of the first index information, encoding the second index information using a value obtained by subtracting 1 from an index of the second merging candidate.

8. The video encoding method of claim 7 , wherein if a value of the second index information is smaller than a value of the first index information, the second index information is encoded using a value equal to an index of the second merging candidate.

9. The video encoding method of claim 7 or 8, wherein whether to apply a partition-based mode to the current block is determined based on at least one of a slice type, a size of the current block, a shape of the current block, and a prediction mode of the current block.

10. The video encoding method of claim 9 , wherein whether to apply the partition-based mode to the current block is determined based on whether a ratio of width to height of the current block is less than a first threshold.

11. The video encoding method of claim 9 , wherein information indicating whether the partition-based mode is applied to the current block is signaled in the bitstream at a sequence level.

12. 9. The video encoding method of claim 7 or 8, wherein information indicating the partition type of the current block is signaled in the bitstream at the current block level.

13. 1. A video decoding device, comprising: A video decoding device comprising an intra prediction unit configured to perform the video decoding method according to any one of claims 1 to 6.

14. 1. A video encoding device, comprising:

13. Video encoding device comprising an intra prediction unit configured to perform a video encoding method according to any one of claims 7 to 12.

15. 1. A computer-readable storage medium, comprising: A computer-readable storage medium storing a computer program for causing a computer to execute the video decoding method according to any one of claims 1 to 6 or the video encoding method according to any one of claims 7 to 12.

Images