Video encoding / decoding method for improving merge mode and method for transmitting bitstream - Patents.com

Abstract

A video encoding / decoding method and apparatus for improving merge mode is provided. [Solution] The video decoding method according to the present invention includes a step of determining a first mode based on at least one of a Combined Inter Intra Prediction (CIIP) mode, a geometric partitioning mode, and a merge mode using a motion vector differential, a step of determining a motion vector from a merge candidate list, and a step of generating a prediction block of a current block based on the first mode and the motion vector.

Description

[Technical field]

[0001] The present invention relates to a video encoding / decoding method and apparatus for improving a merge mode, and more particularly, to a video encoding / decoding method and apparatus for generating a prediction block of a current block by adding a new type of merge mode to a general merge mode. [Background technology]

[0002] The following description merely provides background information related to the present embodiment and does not constitute prior art.

[0003] Video data has a larger amount of data than audio data or still image data, and therefore requires a large amount of hardware resources, including memory, to store or transmit the video data without compression processing.

[0004] Therefore, in general, when video data is stored or transmitted, an encoder is used to compress the video data before storing or transmitting it, and a decoder receives the compressed video data, decompresses it, and plays it back. Examples of such video compression technologies include H.264 / AVC, High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), which has improved coding efficiency by about 30% or more compared to HEVC.

[0005] However, as video size, resolution, and frame rate gradually increase, the amount of data to be encoded also increases accordingly, and therefore new compression technologies that offer better encoding efficiency and greater image quality improvement than existing compression technologies are required.

[0006] The combined inter / intra prediction (CIIP) mode is a method of generating a predicted block of a current block by weighting an intra prediction signal and an inter prediction signal. The geometric partition mode is a method of dividing one coding unit (CU) into two regions, performing inter prediction independently on the two divided regions, and generating a predicted block of a current block by weighting two inter prediction signals. The merge mode using motion vector difference adds a motion vector difference to a motion vector induced in the general merge mode to correct the motion vector. It is necessary to combine the CIIP mode, the geometric partition mode, or the merge mode using motion vector difference to improve the accuracy of the merge mode and improve the coding efficiency. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention has been made in consideration of the above-mentioned conventional technology, and an object of the present invention is to provide a method for generating a predicted block of a current block based on a combined inter / intra prediction (CIIP) mode, a geometric partition mode, and a merge mode using a motion vector difference, and a recording medium storing a bitstream generated by the method. [Means for solving the problem]

[0008] In order to achieve the above object, a video decoding method according to one aspect of the present invention includes a step of determining a first mode based on at least one of a Combined Inter Intra Prediction (CIIP) mode, a geometric partitioning mode, and a merge mode using a motion vector differential, a step of determining a motion vector from a merge candidate list, and a step of generating a prediction block of a current block based on the first mode and the motion vector.

[0009] In order to achieve the above object, a video encoding method according to another aspect of the present invention includes a step of determining a first mode based on at least one of a CIIP mode, a geometric partition mode, and a merge mode using a motion vector differential, a step of determining a motion vector from a merge candidate list, and a step of generating a prediction block of a current block based on the first mode and the motion vector.

[0010] According to an embodiment, there is also provided a method for transmitting a bitstream generated by a video encoding method or apparatus according to the present invention.

[0011] According to an embodiment, there is provided a recording medium storing a bitstream generated by the video encoding method or apparatus according to the present invention.

[0012] Further, according to one embodiment, there is provided a recording medium storing a bitstream received by the video decoding device according to the present invention, decoded, and used to reconstruct a video. Effect of the Invention

[0013] According to an embodiment of the present invention, there is provided a method and apparatus for generating a predicted block of a current block based on a combined inter / intra prediction (CIIP) mode and a geometric partition mode.

[0014] Furthermore, the present invention provides a method and apparatus for generating a predicted block of a current block based on a CIIP mode and a merge mode using a motion vector difference.

[0015] In addition, the present invention provides a method and apparatus for generating a predicted block of a current block based on a geometric partition mode and a merge mode using a motion vector difference.

[0016] Also, according to the present invention, a method and apparatus for generating a predicted block of a current block based on a CIIP mode, a geometric partition mode, and a merge mode using a motion vector difference can be provided.

[0017] Furthermore, the present invention provides a method and apparatus for improving video encoding / decoding efficiency.

[0018] The effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those having ordinary skill in the art to which the present invention pertains from the description below. [Brief description of the drawings]

[0019] [Figure 1] 1 is an exemplary block diagram of a video encoding device embodying techniques of the present invention; [Diagram 2] FIG. 13 is a diagram for explaining a method of dividing blocks using the QTBTTT structure. [Figure 3a] FIG. 2 is a diagram illustrating multiple intra-prediction modes. [Figure 3b] FIG. 2 is a diagram illustrating multiple intra-prediction modes, including a wide-angle intra-prediction mode. [Figure 4] FIG. 13 is an example diagram of neighboring blocks of a current block; [Diagram 5] 1 is an exemplary block diagram of a video decoding device embodying the techniques of the present invention; [Figure 6] 1 is a diagram illustrating a method of generating a predicted block of a current block in a combined inter / intra prediction (CIIP) mode. [Figure 7] 11 is a diagram illustrating neighboring blocks referenced for determining weights in a combined intra-frame inter-frame prediction mode according to an embodiment of the present invention; FIG. [Figure 8] 11 is a diagram illustrating a method for determining weights in a combined intra-picture inter-picture prediction mode according to an embodiment of the present invention. [Figure 9] FIG. 2 is a diagram for explaining a method of applying a geometric partitioning mode to a 32×32 block according to an embodiment of the present invention. [Figure 10a] FIG. 13 is a diagram for explaining angle parameters in a geometric division mode according to an embodiment of the present invention. [Figure 10b] FIG. 13 is a diagram illustrating distance parameters in a geometric division mode according to an embodiment of the present invention. [Figure 11] 11 is a diagram illustrating a lookup table of division direction information in a geometric division mode according to an embodiment of the present invention. FIG. [Figure 12] 11 is a diagram illustrating directional information of a motion vector difference according to an embodiment of the present invention; FIG. [Figure 13] 5 is a diagram illustrating distance information of a motion vector difference according to an embodiment of the present invention; FIG. [Figure 14] 4 is a flowchart of applying inter prediction according to an embodiment of the present invention. [Figure 15] 4 is a flowchart illustrating a new mode application in inter prediction according to an embodiment of the present invention. [Figure 16] 11 is a flowchart illustrating a method for applying a new mode in inter prediction according to another embodiment of the present invention. [Figure 17] 11 is a flowchart illustrating a method for applying a new mode in inter prediction according to another embodiment of the present invention. [Figure 18] 1 is a diagram illustrating a new mode that combines a geometric division mode and a CIIP mode according to an embodiment of the present invention. [Figure 19] 11 is a diagram illustrating a new mode that combines a CIIP mode and a merge mode using a motion vector difference according to an embodiment of the present invention. [Figure 20] 11 is a diagram illustrating a new mode that combines a geometric partitioning mode and a merge mode using a motion vector difference according to an embodiment of the present invention. [Figure 21]11 is a diagram illustrating a new mode that combines a geometric partition mode, a CIIP mode, and a merge mode using a motion vector difference according to an embodiment of the present invention. [Figure 22] 4 is a flowchart illustrating a video decoding process according to an embodiment of the present invention. [Figure 23] 4 is a flowchart illustrating a video encoding process according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific examples of the embodiments of the present invention will be described in detail with reference to the drawings.

[0021] It should be noted that when adding reference numerals to components in each drawing, the same components are assigned the same numerals as much as possible even if they are displayed in different drawings. In addition, when describing the present embodiment, if a detailed description of related publicly known configurations or functions is determined to make the gist of the present embodiment unclear, the detailed description will be omitted.

[0022] 1 is an exemplary block diagram of a video encoding device embodying the technology of the present invention. Hereinafter, the video encoding device and its sub-components will be described with reference to FIG.

[0023] The video encoding device is configured to include a picture division unit 110, a prediction unit 120, a subtractor 130, a transformation unit 140, a quantization unit 145, a rearrangement unit 150, an entropy encoding unit 155, an inverse quantization unit 160, an inverse transform unit 165, an adder 170, a loop filter unit 180, and a memory 190.

[0024] Each component of the video encoding device may be implemented in hardware or software, or a combination of hardware and software. Further, the function of each component may be implemented in software, and a microprocessor may execute the software function corresponding to each component.

[0025] A video is composed of one or more sequences including a plurality of pictures. Each picture is divided into a plurality of regions, and coding is performed for each region. For example, a picture is divided into one or more tiles and / or slices. Here, one or more tiles are defined as a tile group. Each tile and / or slice is divided into one or more coding tree units (CTUs). Each CTU is divided into one or more coding units (CUs) according to a tree structure. Information applied to each CU is coded as a CU syntax, and information commonly applied to CUs included in one CTU is coded as a CTU syntax. Furthermore, information commonly applied to all blocks in one slice is coded as a slice header syntax, and information commonly applied to all blocks constituting one or more pictures is coded in a picture parameter set (PPS) or a picture header. Furthermore, information commonly referred to by a plurality of pictures is coded in a sequence parameter set (SPS). Then, information commonly referenced by one or more SPSs is coded into a Video Parameter Set (VPS). Furthermore, information commonly applied to one tile or tile group is coded as syntax in a tile or tile group header. The syntax included in an SPS, PPS, slice header, tile, or tile group header is called high level syntax.

[0026] The picture division unit 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is coded as an SPS or PPS syntax and transmitted to a video decoding apparatus.

[0027] The picture division unit 110 divides each picture constituting a video into a plurality of coding tree units (CTUs) having a preset size, and then recursively divides the CTUs using a tree structure. Leaf nodes in the tree structure become coding units (CUs), which are basic units of encoding.

[0028] The tree structure may be a quad tree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a binary tree (BT) in which an upper node is divided into two lower nodes, or a ternary tree (TT) in which an upper node is divided into three lower nodes in a ratio of 1:2:1, or a structure that combines two or more of the QT, BT, and TT structures. For example, a QTBT (Quad Tree plus Binary Tree) structure or a QTBTTT (Quad Tree plus Binary Tree Ternary Tree) structure may be used. Here, BTTT and TT are collectively referred to as MTT (Multiple-Type Tree).

[0029] 2 is a diagram for explaining a method of dividing a block using the QTBTTT structure, FIGs. 3a and 3b are diagrams illustrating a plurality of intra prediction modes including a wide-angle intra prediction mode, and FIG. 4 is an example diagram of neighboring blocks of a current block.

[0030] As shown in FIG. 2, the CTU is first split into a QT structure. The quadtree split is repeated until the size of a splitting block reaches the minimum block size MinQTSize of a leaf node allowed in QT. A first flag QT_split_flag indicating whether each node of the QT structure is split into four nodes of a lower layer is coded by the entropy coding unit 155 and signaled to the video decoding device. If the leaf node of the QT is not larger than the maximum block size MaxBTSize of a root node allowed in BT, it is further split into one or more of a BT structure or a TT structure. In the BT structure and / or the TT structure, there are multiple splitting directions. For example, there are two directions, that is, a direction in which the block of the corresponding node is split horizontally and a direction in which the block of the corresponding node is split vertically. As shown in FIG. 2, when MTT splitting is initiated, a second flag mtt_split_flag indicating whether the node is split or not, and a flag indicating the split direction (vertical or horizontal) and / or a flag indicating the split type (Binary or Ternary) after splitting are additionally coded by the entropy coding unit 155 and signaled to the video decoding device.

[0031] Alternatively, before encoding the first flag QT_split_flag indicating whether each node is split into four nodes in the lower layer, a CU split flag split_cu_flag indicating whether the node is split may be encoded. If the CU split flag split_cu_flag value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a coding unit (CU), which is a basic unit of coding. If the CU split flag split_cu_flag value indicates that the node is split, the video coding apparatus starts coding from the first flag in the above-mentioned manner.

[0032] When QTBT is used as another example of a tree structure, there are two types: a type in which a block of a corresponding node is split horizontally into two blocks of the same size (i.e., symmetric horizontal splitting) and a type in which a block of a corresponding node is split vertically (i.e., symmetric vertical splitting). A split flag split_flag indicating whether each node of the BT structure is split into a block of a lower layer and split type information indicating the type of split are coded by the entropy coding unit 155 and transmitted to the video decoding device. Meanwhile, there may be an additional type in which a block of a corresponding node is split into two blocks of asymmetric shape. The asymmetric type includes a type in which a block of a corresponding node is split into two rectangular blocks having a size ratio of 1:3, or a type in which a block of a corresponding node is split diagonally.

[0033] The CU has various sizes according to the QTBT or QTBTTT division from the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of QTBTTT) is referred to as a "current block". Depending on the QTBTTT division adopted, the shape of the current block may be rectangular as well as square.

[0034] The prediction unit 120 predicts the current block to generate a predicted block. The prediction unit 120 includes an intra prediction unit 122 and an inter prediction unit .

[0035] In general, each current block in a picture is predictively coded. In general, prediction of the current block is performed using intra prediction techniques (using data from the picture containing the current block) or inter prediction techniques (using data from a picture coded prior to the picture containing the current block). Inter prediction includes both unidirectional prediction and bidirectional prediction.

[0036] The intra prediction unit 122 predicts pixels in a current block using pixels (reference pixels) located around the current block in a current picture including the current block. There are a number of intra prediction modes depending on the prediction direction. For example, as shown in FIG. 3a, the multiple intra prediction modes include two non-directional modes including a planar mode and a DC mode, and 65 directional modes. Different neighboring pixels and calculation formulas are defined to be used depending on each prediction mode.

[0037] For efficient directional prediction of a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dashed arrows in FIG. 3b are additionally used. These are called "wide angle intra-prediction modes". In FIG. 3b, the arrows indicate corresponding reference samples used for prediction, but do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. The wide angle intra-prediction modes are modes that perform prediction in the opposite direction to a specific directional mode without additional bit transmission when the current block is rectangular. In this case, some wide angle intra-prediction modes available for the current block are determined from among the wide angle intra-prediction modes according to the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having angles smaller than 45 degrees can be used when the current block has a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having angles larger than -135 degrees can be used when the current block has a rectangular shape whose width is larger than its height.

[0038] The intra prediction unit 122 determines the intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit 122 encodes the current block using various intra prediction modes and selects an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit 122 calculates a bit-rate-distortion value using a bit-rate-distortion analysis for the various tested intra prediction modes and selects the intra prediction mode with the best bit-rate-distortion characteristics among the tested modes.

[0039] The intra prediction unit 122 selects one intra prediction mode from among a plurality of intra prediction modes, and predicts a current block using surrounding pixels (reference pixels) and an arithmetic equation determined by the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit 155 and transmitted to the video decoding device.

[0040] The inter prediction unit 124 generates a prediction block for a current block using a motion compensation process. The inter prediction unit 124 searches for a block most similar to the current block in a reference picture that has been coded and decoded prior to the current picture, and generates a prediction block for the current block using the searched block. Then, it generates a motion vector (MV) corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture. In general, motion estimation is performed on a luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. Motion information including information on the reference picture used to predict the current block and information on the motion vector is coded by the entropy coding unit 155 and transmitted to the video decoding device.

[0041] The inter prediction unit 124 performs interpolation on the reference picture or the reference block to improve the accuracy of prediction. That is, a sub-sample between two consecutive integer samples is interpolated by applying a filter coefficient to a plurality of consecutive integer samples including the two integer samples. When a step of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector is expressed to a precision of a decimal unit, not a precision of an integer sample unit. The precision or resolution of the motion vector is set differently for each unit of the target region to be encoded, such as a slice, a tile, a CTU, a CU, etc. When such an adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied for each CU is signaled. The information on the motion vector resolution is information indicating the precision of a differential motion vector, which will be described later.

[0042] Meanwhile, the inter prediction unit 124 performs inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors representing the position of a block most similar to the current block in each reference picture are used. The inter prediction unit 124 selects a first reference picture and a second reference picture from a reference picture list 0 (RefPicList0) and a reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block in each reference picture to generate a first reference block and a second reference block. Then, the inter prediction unit 124 averages or weightedly averages the first reference block and the second reference block to generate a prediction block for the current block. Then, the inter prediction unit 124 transmits motion information including information on the two reference pictures used to predict the current block and information on the two motion vectors to the encoding unit 150. Here, the reference picture list 0 is composed of pictures before the current picture in display order among pre-reconstructed pictures, and the reference picture list 1 is composed of pictures after the current picture in display order among pre-reconstructed pictures. However, this is not necessarily limited to the above, and a previously restored picture after the current picture in display order may be additionally included in reference picture list 0, and conversely, a previously restored picture before the current picture may be additionally included in reference picture list 1.

[0043] Various methods are used to minimize the amount of bits required to code the motion information.

[0044] For example, if the reference picture and motion vector of the current block are the same as those of a neighboring block, the motion information of the current block is transmitted to the video decoding device by encoding information identifying the neighboring block. This method is called a "merge mode."

[0045] In the merge mode, the inter prediction unit 124 selects a preset number of merge candidate blocks (hereinafter, referred to as "merge candidates") from the neighboring blocks of the current block.

[0046] As shown in FIG. 4, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current picture are used as neighboring blocks for deriving merging candidates. Furthermore, blocks located in a reference picture (which may be the same or different from the reference picture used to predict the current block) rather than the current picture in which the current block is located are used as merging candidates. For example, a co-located block with the current block in the reference picture or a block adjacent to the co-located block is additionally used as a merging candidate. If the number of merging candidates selected by the above-described method is smaller than a preset number, a zero vector is added to the merging candidates.

[0047] The inter prediction unit 124 uses the neighboring blocks to construct a merge list including a preset number of merge candidates. The inter prediction unit 124 selects a merge candidate to be used as motion information of the current block from among the merge candidates included in the merge list, and generates merge index information for identifying the selected candidate. The generated merge index information is encoded by the encoding unit 150 and transmitted to the video decoding device.

[0048] Merge skip mode is a special case of merge mode, and when all the transform coefficients for entropy coding are close to zero after quantization, only the neighboring block selection information is transmitted without transmitting the residual signal. By using the merge skip mode, it is possible to achieve relatively high coding efficiency for low-motion images, still images, screen content images, etc.

[0049] Hereinafter, the merge mode and the merge skip mode will be collectively referred to as the merge / skip mode.

[0050] Yet another method for encoding motion information is the Advanced Motion Vector Prediction (AMVP) mode.

[0051] In the AMVP mode, the inter prediction unit 124 derives a predicted motion vector candidate for the motion vector of the current block using a peripheral block of the current block. As the peripheral blocks used to derive the predicted motion vector candidate, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current picture shown in FIG. 4 are used. Furthermore, a block located in a reference picture (which may be the same or different from the reference picture used to predict the current block) rather than the current picture in which the current block is located is used as the peripheral block used to derive the predicted motion vector candidate. For example, a block located at the same position as the current block in the reference picture (collocated block) or a block adjacent to the block at the same position is used. If the number of motion vector candidates is smaller than a preset number by the method described above, a zero vector is added to the motion vector candidates.

[0052] The inter prediction unit 124 derives motion vector predictor candidates using the motion vectors of the neighboring blocks, determines a motion vector predictor for the motion vector of the current block using the motion vector predictor candidates, and calculates a difference motion vector by subtracting the motion vector predictor from the motion vector of the current block.

[0053] The motion vector predictor is obtained by applying a predefined function (e.g., median, average calculation, etc.) to the motion vector predictor candidate. In this case, the video decoding apparatus also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector predictor candidate are blocks that have already been encoded and decoded, the video decoding apparatus also already knows the motion vectors of the neighboring blocks. Therefore, the video encoding apparatus does not need to encode information for identifying the motion vector predictor candidate. Therefore, in this case, information on the differential motion vector and information on the reference picture used to predict the current block are encoded.

[0054] Meanwhile, the motion vector predictor may be determined by selecting one of the motion vector predictor candidates, in which case information for identifying the selected motion vector predictor candidate is additionally encoded together with information on the differential motion vector and information on the reference picture used to predict the current block.

[0055] The subtractor 130 subtracts the predicted block generated by the intra predictor 122 or the inter predictor 124 from the current block to generate a residual block.

[0056] The transform unit 140 transforms the residual signal in the residual block having pixel values ​​in the spatial domain into transform coefficients in the frequency domain. The transform unit 140 transforms the residual signal in the residual block using the entire size of the residual block as a transform unit, or divides the residual block into a plurality of sub-blocks and transforms the sub-blocks as transform units. Alternatively, the transform domain is divided into two sub-blocks, a transformed domain and a non-transformed domain, and transforms the residual signal using only the transformed domain sub-block as a transform unit. Here, the transformed domain sub-block is one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, a flag cu_sbt_flag indicating that only the sub-block has been transformed, directional information cu_sbt_horizontal_flag, and / or position information cu_sbt_pos_flag are coded by the entropy coding unit 155 and signaled to the video decoding apparatus. In addition, the size of the transform domain sub-block has a size ratio of 1:3 on the horizontal axis (or vertical axis) basis, and in this case, a flag cu_sbt_quad_flag that distinguishes the corresponding division is additionally coded by the entropy coding unit 155 and signaled to the video decoding device.

[0057] Meanwhile, the transform unit 140 performs transforms on the residual block in the horizontal and vertical directions separately. Various types of transform functions or transform matrices are used for the transform. For example, a pair of transform functions for horizontal and vertical transforms is defined as a Multiple Transform Set (MTS). The transform unit 140 selects one transform function pair having the highest transform efficiency from the MTS, and transforms the residual block in the horizontal and vertical directions, respectively. Information mts_idx about the transform function pair selected from the MTS is coded by the entropy coding unit 155 and signaled to the video decoding device.

[0058] The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 using the quantization parameter, and outputs the quantized transform coefficients to the entropy coding unit 155. For any block or frame, the quantization unit 145 immediately quantizes the associated residual block without transformation. The quantization unit 145 applies different quantization coefficients (scaling values) depending on the position of the transform coefficients in the transform block. The quantization matrix applied to the two-dimensionally arranged quantized transform coefficients is coded and signaled to the video decoding device.

[0059] The reordering unit 150 performs reordering of the coefficient values ​​on the quantized residual values.

[0060] The sorting unit 150 converts a two-dimensional coefficient array into a one-dimensional coefficient sequence using coefficient scanning. For example, the sorting unit 150 uses a zig-zag scan or a diagonal scan to scan from a DC coefficient to a high-frequency region coefficient and outputs a one-dimensional coefficient sequence. Depending on the size of the transform unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction and a horizontal scan that scans the two-dimensional block shape coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transform unit and the intra prediction mode, a scan method to be used is determined from among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0061] The entropy coding unit 155 generates a bitstream by coding the sequence of one-dimensional quantized transform coefficients output from the reordering unit 150 using various coding schemes such as Context-based Adaptive Binary Arithmetic Code (CABAC) or Exponential Golomb.

[0062] The entropy coding unit 155 also codes information related to block division, such as a CTU size, a CU division flag, a QT division flag, an MTT division type, and an MTT division direction, so that the video decoding device can divide the block in the same manner as the video coding device. The entropy coding unit 155 also codes information related to a prediction type indicating whether the current block is coded by intra prediction or inter prediction, and codes intra prediction information (i.e., information related to an intra prediction mode) or inter prediction information (information related to a coding mode of motion information (merge mode or AMVP mode), a merge index in the case of the merge mode, and information related to a reference picture index and a difference motion vector in the case of the AMVP mode) according to the prediction type. The entropy coding unit 155 also codes information related to quantization, i.e., information related to a quantization parameter and information related to a quantization matrix.

[0063] The inverse quantization unit 160 generates transform coefficients by inverse quantizing the quantized transform coefficients output from the quantization unit 145. The inverse transform unit 165 transforms the transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to restore the residual block.

[0064] The adder 170 reconstructs the current block by adding the reconstructed residual block and the prediction block generated by the prediction unit 120. The pixels in the reconstructed current block are used as reference pixels when intra-predicting the next block in the order.

[0065] The loop filter unit 180 performs filtering on the restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc., that occur due to block-based prediction and transform / quantization. The filter unit 180 includes all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186 as in-loop filters.

[0066] The deblocking filter 182 filters the boundaries between reconstructed blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter 184 and alf 186 perform additional filtering on the deblock-filtered image. The SAO filter 184 and alf 186 are filters used to compensate for differences between reconstructed pixels and original pixels caused by lossy coding. The SAO filter 184 applies an offset in units of CTUs to improve not only subjective image quality but also coding efficiency. In contrast, the ALF 186 performs block-based filtering and applies different filters to distinguish edges and degree of change of a corresponding block to compensate for distortion. Information about filter coefficients used in the ALF is coded and signaled to a video decoder.

[0067] The reconstructed blocks filtered through the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in a memory 190. When all blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter-predicting blocks in a picture to be coded later.

[0068] 5 is an exemplary block diagram of a video decoding apparatus embodying the technology of the present invention. The video decoding apparatus and its sub-components will be described below with reference to FIG.

[0069] The video decoding apparatus includes an entropy decoding unit 510 , a reordering unit 515 , an inverse quantization unit 520 , an inverse transform unit 530 , a prediction unit 540 , an adder 550 , a loop filter unit 560 , and a memory 570 .

[0070] 1, each component of the video decoding apparatus may be implemented in hardware or software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, and a microprocessor may execute the software functions corresponding to each component.

[0071] The entropy decoding unit 510 determines a current block to be decoded by decoding a bitstream generated by a video encoding device and extracting information related to block division, and extracts prediction information, information related to a residual signal, etc., required to restore the current block.

[0072] The entropy decoding unit 510 extracts information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS) to determine the size of the CTU, and divides the picture into CTUs of the determined size. The entropy decoding unit 510 then determines the CTU as the top layer of a tree structure, i.e., the root node, and extracts division information about the CTU, thereby dividing the CTU using the tree structure.

[0073] For example, when splitting a CTU using the QTBTTT structure, first, the first flag QT_split_flag related to the split of QT is extracted, and each node is split into four nodes in the lower layer. Then, for a node corresponding to a leaf node of QT, the second flag MTT_split_flag related to the split of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf node is split into an MTT structure. As a result, each node below the leaf node of QT is recursively split into a BT or TT structure.

[0074] As another example, when splitting a CTU using the QTBTTT structure, first extract a CU split flag split_cu_flag indicating whether the CU can be split, and if the corresponding block is split, extract the first flag QT_split_flag. During the splitting process, each node undergoes 0 or more repeated QT splits followed by 0 or more repeated MTT splits. For example, a CTU may immediately undergo MTT splits, or conversely, may only undergo multiple QT splits.

[0075] As another example, when splitting a CTU using the QTBT structure, the first flag QT_split_flag related to the split of the QT is extracted and each node is split into four nodes in the lower layer. Then, for nodes that correspond to leaf nodes of the QT, the split flag split_flag indicating whether or not the node is further split by the BT and the split direction information are extracted.

[0076] Meanwhile, when the entropy decoding unit 510 determines the current block to be decoded using the division of the tree structure, it extracts information on a prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit 510 extracts syntax elements related to intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit 510 extracts syntax elements related to the inter-prediction information, i.e., information indicating a motion vector and a reference picture referred to by the motion vector.

[0077] In addition, the entropy decoding unit 510 extracts information related to the quantization and information related to the quantized transform coefficients of the current block as information related to the residual signal.

[0078] The reordering unit 515 rearranges the sequence of one-dimensional quantized transform coefficients entropy decoded by the entropy decoding unit 510 back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning order performed by the video encoding device.

[0079] The inverse quantization unit 520 inverse quantizes the quantized transform coefficients using a quantization parameter. The inverse quantization unit 520 applies different quantization coefficients (scaling values) to the two-dimensionally arranged quantized transform coefficients. The inverse quantization unit 520 performs inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to the two-dimensional array of quantized transform coefficients.

[0080] The inverse transform unit 530 inversely transforms the dequantized transform coefficients from the frequency domain to the spatial domain to restore a residual signal, thereby generating a residual block for the current block.

[0081] In addition, when the inverse transform unit 530 inverse transforms only a portion (sub-block) of the transform block, it extracts a flag cu_sbt_flag indicating that only the sub-block of the transform block has been transformed, directional information (vertical / horizontal) of the sub-block cu_sbt_horizontal_flag, and / or position information of the sub-block cu_sbt_pos_flag, and restores the residual signal by inverse transforming the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain, and generates a final residual block for the current block by filling the residual signal with ``0'' values ​​for areas that are not inverse transformed.

[0082] In addition, when MTS is applied, the inverse transform unit 530 determines a transform function or transform matrix to be applied in the horizontal and vertical directions, respectively, using the MTS information mts_idx signaled from the video encoding device, and performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions using the determined transform function.

[0083] The prediction unit 540 includes an intra prediction unit 542 and an inter prediction unit 544. The intra prediction unit 542 is activated when the prediction type of the current block is intra prediction, and the inter prediction unit 544 is activated when the prediction type of the current block is inter prediction.

[0084] The intra prediction unit 542 determines an intra prediction mode of the current block from among multiple intra prediction modes from a syntax element for the intra prediction mode extracted from the entropy decoding unit 510, and predicts the current block using reference pixels surrounding the current block according to the intra prediction mode.

[0085] The inter prediction unit 544 determines a motion vector of the current block and a reference picture to which the motion vector refers using syntax elements for the inter prediction mode extracted from the entropy decoding unit 510, and predicts the current block using the motion vector and the reference picture.

[0086] The adder 550 reconstructs a current block by adding the residual block output from the inverse transform unit and the predicted block output from the inter prediction unit or the intra prediction unit. The reconstructed pixels in the current block are used as reference pixels when intra predicting a block to be decoded later.

[0087] The loop filter unit 560 includes a deblocking filter 562, an SAO filter 564, and an ALF 566 as in-loop filters. The deblocking filter 562 deblocks the boundaries between reconstructed blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblock filtering to compensate for differences between reconstructed pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information on the filter coefficients decoded from the bitstream.

[0088] The reconstructed blocks filtered through the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in a memory 570. Once all blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter-predicting blocks in a picture to be coded later.

[0089] FIG. 6 is a diagram illustrating a method of generating a predicted block of a current block in a combined inter / intra prediction (CIIP) mode.

[0090] The intra prediction mode has the same meaning as the intra prediction mode. The intra prediction mode and the intra prediction mode are mixed. The inter prediction mode has the same meaning as the inter prediction mode. The inter prediction mode and the inter prediction mode are mixed. The combined intra inter prediction mode has the same meaning as the combined intra inter prediction mode. The combined intra inter prediction mode and the CIIP mode are mixed. In the CIIP mode, the inter prediction block is generated in the same manner as in the general merge mode. The intra prediction block is generated by applying a planar mode to reference pixels adjacent to the periphery of the current block. A final CIIP-based prediction block is generated by applying a weighted value to the generated inter prediction block and the intra prediction block.

[0091] Referring to FIG. 6, a reference block P inter is derived. The planar mode is applied to the reference pixels adjacent to the current block to generate an intra-prediction block P Planar The corresponding reference block P inter and intra prediction block P Planar Applying weights to the CIIP-based prediction block P CIIP is generated. Reference block P inter The weighting applied to is W inter The intra prediction block P Planar The weighting applied to is W intra CIIP-based prediction block P CIIP is generated using the following Equation 1.

[0092]

number

[0093] FIG. 7 is a diagram illustrating neighboring blocks referenced for determining weights in a combined intra-frame inter-frame prediction mode according to an embodiment of the present invention.

[0094] In the CIIP mode, the weight is determined taking into consideration whether the neighboring blocks adjacent to the current block are coded in intra prediction mode.

[0095] Referring to FIG. 7, in the CIIP mode, the weight is determined in consideration of whether the upper peripheral block A and the left peripheral block L adjacent to the current block are intra-prediction mode coded.

[0096] FIG. 8 is a diagram illustrating a method for determining weights in a combined intra-frame inter-frame prediction mode according to an embodiment of the present invention.

[0097] If the neighboring blocks adjacent to the current block code many intra-prediction modes, a large weight is assigned to the intra-prediction block, whereas if the neighboring blocks adjacent to the current block code few intra-prediction modes, a small weight is assigned to the intra-prediction block.

[0098] Referring to FIG. 8, when the upper peripheral block A and the left peripheral block L of the current block in FIG. 7 code an intra prediction mode, a weight assigned to the intra prediction block corresponds to 3. When the upper peripheral block A codes an intra prediction mode and the left peripheral block L does not code an intra prediction mode, a weight assigned to the intra prediction block corresponds to 2. When the upper peripheral block A does not code an intra prediction mode and the left peripheral block L codes an intra prediction mode, a weight assigned to the intra prediction block corresponds to 2. When the upper peripheral block A does not code an intra prediction mode and the left peripheral block L does not code an intra prediction mode, a weight assigned to the intra prediction block corresponds to 1.

[0099] FIG. 9 is a diagram illustrating a method of applying a geometric partitioning mode to a 32×32 block according to an embodiment of the present invention.

[0100] In the geometric partition mode, one coding unit is divided into two regions by a straight line partition boundary. Inter prediction is performed for each of the two divided regions using different motion information. An inter prediction block is generated for each of the two divided regions. A weighted average of the two generated inter prediction blocks is then generated to generate a final prediction block of the geometric partition mode. The geometric partition mode sets a partition boundary region defined by a straight line using angle parameters and distance parameters. The weighted average has the same meaning as the weighted sum.

[0101] 9, a 32×32 block is divided into two regions. Inter prediction is performed on each of the two divided regions. φ corresponds to an angle parameter. ρ corresponds to a distance parameter. A line dividing the 32×32 block is set using the angle parameter and the distance parameter.

[0102] 10a and 10b are diagrams for explaining angle parameters and distance parameters in a geometric division mode according to an embodiment of the present invention.

[0103] Referring to FIG. 10a, the angle parameters are defined as a total of 20 quantized angles by symmetrically dividing a range of 360 degrees within a coding unit.

[0104] Referring to Fig. 10b, the distance parameter is defined by four quantized distances. Of the total 80 division directions that occur by the combination of the angle parameter and the distance parameter, 10 overlapping division directions and 6 overlapping division directions between the binary tree division and the ternary tree division are excluded. Thus, the geometric division mode uses a total of 64 division directions.

[0105] FIG. 11 is a diagram for explaining a lookup table of division direction information in the geometric division mode according to an embodiment of the present invention.

[0106] Combinations of angle parameters and distance parameters are defined in a look-up table. Split direction information is transmitted for each coding unit. In the geometric partition mode, a merge candidate list for the geometric partition mode including only one-directional motion information is constructed from the general merge candidate list. This simplifies the coding of motion information and reduces the number of possible combinations. A merge index to be used for each partition region is transmitted using the merge candidate list for the geometric partition mode.

[0107] Referring to FIG 11, split direction information (e.g., merge_gpm_partition_idx) is determined based on information on an angle parameter (e.g., angleIdx) and information on a distance parameter (e.g., distanceIdx). merge_gpm_partition_idx based on a combination of angleIdx and distanceIdx is defined in a lookup table. The value of merge_gpm_partition_idx ranges from 0 to 63. merge_gpm_partition_idx is transmitted for each coding unit.

[0108] FIG. 12 is a diagram for explaining directional information of a motion vector difference according to an embodiment of the present invention.

[0109] In the general merge mode, the motion information of the merge candidate list is used as the motion information of the current block, and no additional motion information is transmitted. The general merge mode has the advantage of minimizing the amount of bits, but has the limitation of not being able to express optimal motion information. The merge mode with motion vector difference (Merge with Motion Vector Difference: MMVD) corrects the motion information by adding the motion vector difference to the motion vector induced in the general merge mode.

[0110] To reduce complexity, the merge mode using motion vector difference uses only the first and second candidates in the candidate list of the general merge mode. One candidate is selected from the two candidates, and the motion vector of the selected candidate is set as the initial motion vector. The final motion vector is determined by adding the additionally transmitted motion vector difference information to the initial motion vector. The motion vector difference information includes direction information and distance information.

[0111] Referring to FIG. 12, the direction information of the motion vector difference can only be corrected in the vertical or horizontal direction. The direction information of the motion vector difference is composed of four directions. Index 0 is assigned to the direction information of the motion vector difference that corrects by +1 in the horizontal direction and 0 in the vertical direction. Index 1 is assigned to the direction information of the motion vector difference that corrects by -1 in the horizontal direction and 0 in the vertical direction. Index 2 is assigned to the direction information of the motion vector difference that corrects by 0 in the horizontal direction and +1 in the vertical direction. Index 3 is assigned to the direction information of the motion vector difference that corrects by 0 in the horizontal direction and -1 in the vertical direction.

[0112] FIG. 13 is a diagram for explaining distance information of a motion vector difference according to an embodiment of the present invention.

[0113] 13, the distance information of the motion vector difference is composed of a luminance pixel distance, which is a motion distance. The direction information of the motion vector difference is composed of eight motion distances. An index of 0 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 1 / 4. An index of 1 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 1 / 2. An index of 2 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 1. An index of 3 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 2. An index of 4 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 4. An index of 5 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 8. An index of 6 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 16. An index of 7 is assigned to the distance information of the motion vector difference whose luminance pixel distance is 32.

[0114] FIG. 14 is a flow chart of applying inter prediction according to an embodiment of the present invention.

[0115] Referring to FIG. 14, it is determined whether or not the SKIP mode is applied (step S1410). If it is determined that the SKIP mode is applied (step S1410-YES), the SKIP mode is applied (step S1420). If it is determined that the SKIP mode is not applied (step S1410-NO), it is determined whether or not the merge mode is applied (step S1430). If it is determined that the merge mode is not applied (step S1430-NO), an Adaptive Motion Vector Prediction (AMVP) mode is applied (step S1440). If it is determined that the merge mode is applied (step S1430-YES), it is determined whether or not the merge mode is applied to the sub-block (step S1450). If it is determined that the merge mode is applied to the sub-block (step S1450-YES), the sub-block merge mode is applied (step S1460). Sub-block based temporal motion vector prediction (SbTMVP) and affine (AFFINE) merge modes are applied. If it is determined that no merge mode is applied to the sub-block (step S1450-NO), then it is determined whether a general merge mode is applied (step S1470).

[0116] If it is determined that the general merge mode is not applied (step S1470-no), it is determined whether or not the CIIP mode is applied (step S1480). If it is determined that the CIIP mode is not applied (step S1480-no), the geometric partition mode (GPM) is applied (step S1481). If it is determined that the CIIP mode is applied (step S1480-yes), the CIIP mode is applied (step S1482). If it is determined that the general merge mode is applied (step S1470-yes), it is determined whether or not the merge mode using the motion vector difference (MMVD) is applied (step S1490). If it is determined that the merge mode using the motion vector difference is applied (step S1490-yes), the merge mode using the motion vector difference (MMVD) is applied (step S1491). If it is determined that the merge mode using the motion vector difference is not applied (step S1490-no), the general merge mode is applied (step S1492).

[0117] FIG. 15 is a flow chart of applying a new mode in inter prediction according to an embodiment of the present invention.

[0118] When the general merge mode is applied to the current block instead of the sub-block merge mode, it is determined whether the current block is to be split. Then, a prediction method for the current block is determined. The prediction method for the current block includes an intra-inter prediction method and an inter prediction method. Finally, it is determined whether a motion vector difference is to be used. It is determined whether the motion vector difference is to be used and whether motion information is to be corrected.

[0119] Referring to FIG. 15, it is determined whether a merge mode is applied to a sub-block (step S1510). If it is determined that a merge mode is applied to a sub-block (step S1510-yes), the merge mode is applied to the sub-block (step S1520). Sub-block based temporal motion vector prediction (SbTMVP) and affine merge mode are applied. If it is determined that a merge mode is not applied to a sub-block (step S1510-no), it is determined whether a geometric partition mode (GPM) is applied (step S1530). If it is determined that a geometric partition mode is applied (step S1530-yes), it is determined whether a CIIP mode is applied (step S1540). If it is determined that a CIIP mode is applied (step S1540-yes), it is determined whether a merge mode using a motion vector difference (MMVD) is applied (step S1541). If it is determined that the merge mode using the motion vector difference is applied (step S1541--YES), a new mode that combines the geometric partitioning mode (GPM), the CIIP mode, and the merge mode using the motion vector difference (MMVD) is applied (step S1542).If it is determined that the merge mode using the motion vector difference is not applied (step S1541--NO), a new mode that combines the geometric partitioning mode (GPM) and the CIIP mode is applied (step S1543).

[0120] If it is determined that the CIIP mode is not applied (step S1540-no), it is determined whether or not the merge mode using the motion vector difference (MMVD) is applied (step S1544). If it is determined that the merge mode using the motion vector difference is applied (step S1544-yes), a new mode that combines the geometric partition mode (GPM) and the merge mode using the motion vector difference (MMVD) is applied (step S1545). If it is determined that the merge mode using the motion vector difference is not applied (step S1544-no), the geometric partition mode (GPM) is applied (step S1546).

[0121] If it is determined that the geometric partition mode is not applied (step S1530--NO), it is determined whether or not the CIIP mode is applied (step S1550). If it is determined that the CIIP mode is applied (step S1550--YES), it is determined whether or not the merge mode using the motion vector difference (MMVD) is applied (step S1551). If it is determined that the merge mode using the motion vector difference is applied (step S1551--YES), a new mode that combines the CIIP mode and the merge mode using the motion vector difference (MMVD) is applied (step S1552). If it is determined that the merge mode using the motion vector difference is not applied (step S1551--NO), the CIIP mode is applied (step S1553).

[0122] If it is determined that the CIIP mode is not applied (step S1550--NO), it is determined whether or not the merge mode using the motion vector difference (MMVD) is applied (step S1554). If it is determined that the merge mode using the motion vector difference is applied (step S1554--YES), the merge mode using the motion vector difference (MMVD) is applied (step S1555). If it is determined that the merge mode using the motion vector difference is not applied (step S1554--NO), the general merge mode is applied (step S1556).

[0123] FIG. 16 is a flowchart illustrating a method for applying a new mode in inter prediction according to another embodiment of the present invention.

[0124] Referring to FIG. 16, it is determined whether a merge mode is applied to a sub-block (step S1610). If it is determined that a merge mode is applied to a sub-block (step S1610-YES), the merge mode is applied to the sub-block (step S1620). Sub-block based temporal motion vector prediction (SbTMVP) and affine merge mode are applied. If it is determined that a merge mode is not applied to a sub-block (step S1610-NO), it is determined whether a geometric partition mode (GPM) is applied (step S1630). If it is determined that a geometric partition mode is applied (step S1630-YES), it is determined whether a CIIP mode is applied (step S1640). If it is determined that a CIIP mode is applied (step S1640-YES), a new mode in which the geometric partition mode (GPM) and the CIIP mode are combined is applied (step S1641).

[0125] If it is determined that the CIIP mode is not applied (step S1640--NO), it is determined whether or not the merge mode using the motion vector difference (MMVD) is applied (step S1642). If it is determined that the merge mode using the motion vector difference is applied (step S1642--YES), a new mode that combines the geometric partition mode (GPM) and the merge mode using the motion vector difference (MMVD) is applied (step S1643). If it is determined that the merge mode using the motion vector difference is not applied (step S1642--NO), the geometric partition mode (GPM) is applied (step S1644).

[0126] If it is determined that the geometric partition mode is not applied (step S1630--NO), it is determined whether or not the CIIP mode is applied (step S1650). If it is determined that the CIIP mode is applied (step S1650--YES), the CIIP mode is applied (step S1651). If it is determined that the CIIP mode is not applied (step S1650--NO), it is determined whether or not the merge mode using the motion vector difference (MMVD) is applied (step S1652). If it is determined that the merge mode using the motion vector difference is applied (step S1652--YES), the merge mode using the motion vector difference (MMVD) is applied (step S1653). If it is determined that the merge mode using the motion vector difference is not applied (step S1652--NO), the general merge mode is applied (S1654).

[0127] FIG. 17 is a flowchart illustrating a method for applying a new mode in inter prediction according to yet another embodiment of the present invention.

[0128] Referring to FIG. 17, it is determined whether a merge mode is applied to a sub-block (step S1710). If it is determined that a merge mode is applied to a sub-block (step S1710-YES), the merge mode is applied to the sub-block (step S1720). Sub-block based temporal motion vector prediction (SbTMVP) and affine merge mode are applied. If it is determined that a merge mode is not applied to a sub-block (step S1710-NO), it is determined whether a general merge mode is applied (step S1730). If it is determined that a general merge mode is not applied (step S1730-NO), it is determined whether a CIIP mode is applied (step S1740). If it is determined that a CIIP mode is not applied (step S1740-NO), it is determined whether a new mode that combines a geometric partition mode (GPM) and a merge mode using a motion vector difference (MMVD) is applied (step S1741). If it is determined that the new mode combining the geometric partitioning mode and the merge mode using the motion vector difference is not to be applied (step S1741-No), the geometric partitioning mode (GPM) is applied (step S1742).If it is determined that the new mode combining the geometric partitioning mode and the merge mode using the motion vector difference is to be applied (step S1741-Yes), the new mode combining the geometric partitioning mode (GPM) and the merge mode using the motion vector difference (MMVD) is applied (step S1743).

[0129] If it is determined that the CIIP mode is to be applied (step S1740--YES), it is determined whether a new mode combining the geometric partition mode (GPM) and the CIIP mode is to be applied (step S1744). If it is determined that the new mode combining the geometric partition mode and the CIIP mode is not to be applied (step S1744--NO), the CIIP mode is applied (step S1745). If it is determined that the new mode combining the geometric partition mode and the CIIP mode is to be applied (step S1744--YES), the new mode combining the geometric partition mode (GPM) and the CIIP mode is applied (step S1746).

[0130] If it is determined that the general merge mode is applied (step S1730--YES), it is determined whether or not the merge mode using the motion vector difference (MMVD) is applied (step S1750). If it is determined that the merge mode using the motion vector difference is applied (step S1750--YES), it is determined whether or not the new mode combining the CIIP mode and the merge mode using the motion vector difference (MMVD) is applied (step S1751). If it is determined that the new mode combining the CIIP mode and the merge mode using the motion vector difference is not applied (step S1751--NO), the merge mode using the motion vector difference (MMVD) is applied (step S1752). If it is determined that the new mode combining the CIIP mode and the merge mode using the motion vector difference is applied (step S1751--YES), the new mode combining the CIIP mode and the merge mode using the motion vector difference (MMVD) is applied (step S1753). If it is determined that the merge mode using the motion vector difference is not applied (step S1750-NO), the general merge mode is applied (step S1754).

[0131] FIG. 18 is a diagram illustrating a new mode that combines the geometric division mode and the CIIP mode according to an embodiment of the present invention.

[0132] Referring to FIG. 18, in this new mode that combines the geometric partition mode and the CIIP mode, the current block is partitioned into two regions by the geometric partition mode. The two regions correspond to a P1 region and a P2 region. The P1 region and the P2 region generate inter-predicted blocks independently using different motion information. The two independently generated inter-predicted blocks are weighted averaged or weighted summed to generate a final inter-predicted block P inter The intra prediction block P intra is generated using the planar mode. intra is generated using an intra prediction mode derived using various methods such as a template based intra mode derivation method (TIMD) or a decoder side intra mode derivation method (DIMD). Pred is the final inter-predicted block P inter and intra prediction block P intra Here, the final inter prediction block P inter The weighting value W applied to inter and intra prediction block P intra The weighting value W applied to intra is arbitrarily determined.

[0133] FIG. 19 is a diagram illustrating a new mode that combines the CIIP mode and the merge mode using a motion vector difference according to an embodiment of the present invention.

[0134] Referring to FIG. 19, a new mode that combines the CIIP mode and the merge mode using the motion vector difference is used to calculate the inter-prediction block P inter is generated using the compensated motion information generated by adding the motion vector differential (MVD) to the motion vector derived from the merge mode. intra is generated using the planar mode. intrais generated using an intra prediction mode derived using various methods such as a template-based intra prediction mode derivation method or an intra prediction mode derivation method in a decoder. pred is the inter prediction block P inter and intra prediction block P intra Here, the inter prediction block P inter The weighting value W applied to inter and intra prediction block P intra The weighting value W applied to intra is arbitrarily determined.

[0135] FIG. 20 is a diagram illustrating a new mode that combines a geometric partition mode and a merge mode using a motion vector difference according to an embodiment of the present invention.

[0136] Referring to FIG. 20, this is a new mode that combines the geometric partition mode and the merge mode using motion vector difference, in which the current block is divided into two regions by the geometric partition mode. The two regions correspond to the P1 region and the P2 region. The motion information of the P1 region and the motion information of the P2 region are corrected using the same motion vector difference information for the P1 region and the P2 region. The motion information of the P1 region and the motion information of the P2 region are corrected using different motion vector difference information (MVD1, MVD2) for the P1 region and the P2 region, respectively. The P1 region and the P2 region independently generate two inter-predicted blocks using the corrected motion information. The two independently generated inter-predicted blocks are weighted averaged or weighted summed to generate the final inter-predicted block P inter The final inter-predicted block P inter and the final predicted block P pred is the same as.

[0137] FIG. 21 is a diagram illustrating a new mode that combines a geometric partition mode, a CIIP mode, and a merge mode using a motion vector difference according to an embodiment of the present invention.

[0138] Referring to FIG. 21, this is a new mode that combines the geometric partition mode, the CIIP mode, and the merge mode using motion vector difference, in which the current block is divided into two regions by the geometric partition mode. The two regions correspond to the P1 region and the P2 region. The motion information of the P1 region and the motion information of the P2 region are corrected using the same motion vector difference information for the P1 region and the P2 region. The motion information of the P1 region and the motion information of the P2 region are corrected using different motion vector difference information (MVD1, MVD2) for the P1 region and the P2 region, respectively. The P1 region and the P2 region independently generate two inter-predicted blocks using the corrected motion information. The two independently generated inter-predicted blocks are weighted averaged or summed to generate the final inter-predicted block P inter is generated.

[0139] Intra prediction block P intra is generated using the planar mode. intra is generated using an intra prediction mode derived using various methods such as a template-based intra prediction mode derivation method or an intra prediction mode derivation method in a decoder. pred is the final inter-predicted block P inter and intra prediction block P intra Here, the final inter prediction block P inter The weighting value W applied to inter and intra prediction block P intra The weighting value W applied to intra is arbitrarily determined.

[0140] <Method of refining motion vectors in a decoder (Decoder Side Motion Vector Refinement (DMVR)) and method of refining motion vectors based on bi-directional optical flow (BDOF)>

[0141] The DMVR method corresponds to a method of correcting bilateral motion vectors predicted in the general merge mode through a bilateral matching (BM) based motion vector search during the decoding process. The DMVR method is performed when the current block is decoded in the general merge mode. The BDOF based motion vector correction method corresponds to a method of searching for an optimal motion vector in a 4x4 sub-block unit using the bilateral motion vectors and performing prediction based on the optimal motion vector. The BDOF based motion vector correction method is performed when the current block is decoded in the general merge mode or the merge mode using motion vector difference.

[0142] In Fig. 15, when a merge mode using a motion vector difference (MMVD) is applied (step S1555), a method of compensating a motion vector based on BDOF is applied. In Fig. 15, when a general merge mode is applied (step S1556), a method of compensating a motion vector based on a DMVR method and BDOF is applied. However, the present invention is not limited to such an embodiment. The method of compensating a motion vector based on the DMVR method and BDOF is also applied when any merge mode is applied.

[0143] FIG. 22 is a flow chart illustrating a video decoding process according to an embodiment of the present invention.

[0144] 22, the decoding apparatus determines a first mode based on at least one of a CIIP mode, a geometric partition mode, and a merge mode using a motion vector difference (step S2210). The decoding apparatus determines a motion vector from a merge candidate list (step S2220). The decoding apparatus generates a prediction block for a current block based on the first mode and the motion vector (step S2230).

[0145] When the first mode is a mode combining the CIIP mode and the geometric partition mode, determining a motion vector from the merge candidate list includes determining a motion vector for each of the two divided regions from the merge candidate list. When the first mode is a mode combining the CIIP mode and the geometric partition mode, generating a predicted block for the current block includes generating a final inter predicted block for the current block based on the two divided regions and the motion vectors for each of the two divided regions, generating an intra predicted block for the current block based on at least one reference pixel adjacent to the current block, and generating a predicted block for the current block by performing a weighted average of the final inter predicted block and the intra predicted block.

[0146] The step of generating a final inter-predicted block for the current block includes a step of generating an inter-predicted block for each of the two divided regions using a motion vector for each of the two divided regions, and a step of combining or weighted averaging the inter-predicted blocks for each of the two divided regions to generate a final inter-predicted block for the current block.

[0147] When the first mode is a combined mode of the CIIP mode and the merge mode using the motion vector difference, the step of generating a predicted block for the current block includes the steps of generating an inter predicted block for the current block based on the compensated motion vector, generating an intra predicted block for the current block based on at least one reference pixel adjacent to the current block, and generating a predicted block for the current block by performing a weighted average of the inter predicted block and the intra predicted block. The compensated motion vector is generated based on the motion vector and the motion vector difference information.

[0148] When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, the step of determining a motion vector from the merge candidate list includes a step of determining a motion vector for each of the two divided regions from the merge candidate list. When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, the step of generating a predicted block for the current block includes a step of generating an inter predicted block for each of the two divided regions using the corrected motion vector, and a step of generating a predicted block for the current block by combining or weighted averaging the inter predicted blocks for each of the two divided regions. The corrected motion vector is generated based on the motion vector for each of the two divided regions and the motion vector difference information.

[0149] When the first mode is a mode combining the CIIP mode, the geometric partition mode, and the merge mode using the motion vector difference, the step of determining a motion vector from the merge candidate list includes a step of determining a motion vector for each of the two divided regions from the merge candidate list. When the first mode is a mode combining the CIIP mode, the geometric partition mode, and the merge mode using the motion vector difference, the step of generating a predicted block for the current block includes a step of generating a final inter predicted block for the current block based on the two divided regions and the corrected motion vectors for each of the two divided regions, a step of generating an intra predicted block for the current block based on at least one reference pixel adjacent to the current block, and a step of generating a predicted block for the current block by performing a weighted average of the final inter predicted block and the intra predicted block. The corrected motion vector is generated based on the motion vector and the motion vector difference information for each of the two divided regions.

[0150] The step of generating a final inter-predicted block for the current block includes the steps of generating an inter-predicted block for each of the two divided regions using the corrected motion vector for each of the two divided regions, and combining or weighted averaging the inter-predicted blocks for each of the two divided regions to generate a final inter-predicted block for the current block.

[0151] FIG. 23 is a diagram illustrating a video encoding process according to an embodiment of the present invention.

[0152] 23, the encoding apparatus determines a first mode based on at least one of a CIIP mode, a geometric partition mode, and a merge mode using a motion vector difference (step S2310). The encoding apparatus determines a motion vector from a merge candidate list (step S2320). The encoding apparatus generates a prediction block for a current block based on the first mode and the motion vector (step S2330).

[0153] When the first mode is a mode combining the CIIP mode and the geometric partition mode, determining a motion vector from the merge candidate list includes determining a motion vector for each of the two divided regions from the merge candidate list. When the first mode is a mode combining the CIIP mode and the geometric partition mode, generating a predicted block for the current block includes generating a final inter predicted block for the current block based on the two divided regions and the motion vectors for each of the two divided regions, generating an intra predicted block for the current block based on at least one reference pixel adjacent to the current block, and generating a predicted block for the current block by performing a weighted average of the final inter predicted block and the intra predicted block.

[0154] The step of generating a final inter-predicted block for the current block includes a step of generating an inter-predicted block for each of the two divided regions using a motion vector for each of the two divided regions, and a step of combining or weighted averaging the inter-predicted blocks for each of the two divided regions to generate a final inter-predicted block for the current block.

[0155] When the first mode is a combined mode of the CIIP mode and the merge mode using the motion vector difference, the step of generating a predicted block for the current block includes the steps of generating an inter predicted block for the current block based on the compensated motion vector, generating an intra predicted block for the current block based on at least one reference pixel adjacent to the current block, and generating a predicted block for the current block by performing a weighted average of the inter predicted block and the intra predicted block. The compensated motion vector is generated based on the motion vector and the motion vector difference information.

[0156] When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, the step of determining a motion vector from the merge candidate list includes a step of determining a motion vector for each of the two divided regions from the merge candidate list. When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, the step of generating a predicted block for the current block includes a step of generating an inter predicted block for each of the two divided regions using the corrected motion vector, and a step of generating a predicted block for the current block by combining or weighted averaging the inter predicted blocks for each of the two divided regions. The corrected motion vector is generated based on the motion vector for each of the two divided regions and the motion vector difference information.

[0157] When the first mode is a mode combining the CIIP mode, the geometric partition mode, and the merge mode using the motion vector difference, the step of determining a motion vector from the merge candidate list includes a step of determining a motion vector for each of the two divided regions from the merge candidate list. When the first mode is a mode combining the CIIP mode, the geometric partition mode, and the merge mode using the motion vector difference, the step of generating a predicted block for the current block includes a step of generating a final inter predicted block for the current block based on the two divided regions and the corrected motion vectors for each of the two divided regions, a step of generating an intra predicted block for the current block based on at least one reference pixel adjacent to the current block, and a step of generating a predicted block for the current block by performing a weighted average of the final inter predicted block and the intra predicted block. The corrected motion vector is generated based on the motion vector and the motion vector difference information for each of the two divided regions.

[0158] The step of generating a final inter-predicted block for the current block includes the steps of generating an inter-predicted block for each of the two divided regions using the corrected motion vector for each of the two divided regions, and combining or weighted averaging the inter-predicted blocks for each of the two divided regions to generate a final inter-predicted block for the current block.

[0159] In the flow charts / timing diagrams of this specification, each step is described to be executed in sequence, but this is merely an illustrative description of the technical idea of ​​one embodiment of the present invention. In other words, a person skilled in the art to which the present invention belongs can apply various modifications and variations by changing the order described in the flow charts / timing diagrams or executing one or more steps in parallel without departing from the essential characteristics of the present invention, so the flow charts / timing diagrams are not limited to a chronological order.

[0160] It should be understood from the above description that the exemplary embodiments may be embodied in many other ways. The functions or methods described in one or more of the examples may be embodied in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to particularly emphasize the independence of their implementation.

[0161] Meanwhile, various functions or methods described in the present embodiment may be implemented as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. Non-transitory recording media include, for example, any type of recording device in which data is stored in a form readable by a computer system. For example, non-transitory recording media include recording media such as erasable programmable read only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0162] The above description is merely an illustrative example of the technical idea of ​​the present invention, and a person having ordinary knowledge in the technical field to which the present invention belongs may make various modifications and variations without departing from the essential characteristics of the present invention. Therefore, the present embodiment is for the purpose of explanation, not for the purpose of limiting the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by such an embodiment. The scope of protection of the present invention should be interpreted according to the claims, and all technical ideas within the scope of the equivalents thereof should be interpreted as being included in the scope of the present invention.

[0163] <CROSS-REFERENCE TO RELATED APPLICATION>

[0164] This patent application claims priority to Patent Application No. 10-2022-0000677 filed in Korea on January 4, 2022, and Patent Application No. 10-2022-0190744 filed in Korea on December 30, 2022, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0165] 110 Picture division section 120 Prediction Department 122, 542 Intra prediction section 124, 544 Inter Prediction Section 130 Subtractor 140 Conversion unit 145 Quantization section 150, 515 Sorting section 155 Entropy coding unit 160, 520 Inverse quantization section 165, 530 Reverse conversion unit 170, 550 Adder 180 Loop filter section 182 Deblocking Filter 184, 564 SAO (Sample Adaptive Offset) filter 186, 566 ALF (Adaptive Loop Filter) 190 Memory 510 Entropy Decoding Unit 540 Prediction Department 560 Loop filter section 562 Deblocking Filter 570 Memory A Upper peripheral block A0 Left block A1 Lower left block B0 Upper Block B1 Upper right block B2 Upper left block BTTT Binary Tree Ternary Tree L Left peripheral block P CIIP Predicted Blocks P inter Reference Blocks P Planar Intra-predicted blocks P Pred Final predicted block QT Quad Tree W inter Weighting applied to inter-predicted blocks W intra Weighting applied to intra-predicted blocks

Claims

1. determining a first mode based on at least one of a combined inter intra prediction (CIIP) mode, a geometric partitioning mode, and a merge mode using a motion vector difference; determining a motion vector from the merging candidate list; generating a prediction block for a current block based on the first mode and the motion vector.

2. When the first mode is a mode combining the CIIP mode and the geometric partitioning mode, determining a motion vector from the merge candidate list includes determining a motion vector for each of two partitioned regions from the merge candidate list; When the first mode is a mode that combines the CIIP mode and the geometric partitioning mode, the step of generating a prediction block of the current block includes: generating a final inter-predicted block for the current block based on the two divided regions and a motion vector for each of the two divided regions; generating an intra-predicted block for the current block based on at least one reference pixel adjacent to the current block; and performing a weighted average of the final inter prediction block and the intra prediction block to generate a prediction block for the current block.

3. The step of generating a final inter-predicted block for the current block includes: generating an inter-predicted block for each of the two divided regions using a motion vector for each of the two divided regions; and generating a final inter-predicted block for the current block by combining or weighted averaging the inter-predicted blocks for each of the two divided regions.

4. When the first mode is a mode combining the CIIP mode and the merge mode using the motion vector difference, the step of generating a prediction block for the current block includes: generating an inter-prediction block for the current block based on the compensated motion vector; generating an intra-predicted block for the current block based on at least one reference pixel adjacent to the current block; generating a predicted block of the current block by performing a weighted average of the inter predicted block and the intra predicted block; 2. The method of claim 1, wherein the compensated motion vector is generated based on the motion vector and motion vector differential information.

5. When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, determining a motion vector from the merge candidate list includes determining a motion vector for each of two divided regions from the merge candidate list; When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, the step of generating a prediction block for the current block includes: generating an inter-prediction block for each of the two divided regions using the corrected motion vector; generating a prediction block of the current block by combining or weighted averaging the inter-prediction blocks for each of the two divided regions; 2. The method of claim 1, wherein the compensated motion vector is generated based on motion vectors and motion vector difference information for each of the two divided regions.

6. When the first mode is a mode combining the CIIP mode, the geometric partitioning mode, and the merge mode using the motion vector difference, determining a motion vector from the merge candidate list includes determining a motion vector for each of two divided regions from the merge candidate list; When the first mode is a mode that combines the CIIP mode, the geometric partitioning mode, and the merge mode using the motion vector difference, the step of generating a prediction block for the current block includes: generating a final inter-predicted block for the current block based on the two divided regions and the corrected motion vectors for each of the two divided regions; generating an intra-predicted block for the current block based on at least one reference pixel adjacent to the current block; generating a prediction block of the current block by performing a weighted average of the final inter prediction block and the intra prediction block; 2. The method of claim 1, wherein the compensated motion vector is generated based on motion vectors and motion vector difference information for each of the two divided regions.

7. The step of generating a final inter-predicted block for the current block includes: generating an inter-predicted block for each of the two divided regions using the corrected motion vector for each of the two divided regions; and combining or weighted averaging the inter-predicted blocks for each of the two divided regions to generate a final inter-predicted block for the current block.

8. determining a first mode based on at least one of a CIIP mode, a geometric partition mode, and a merge mode using a motion vector difference; determining a motion vector from the merging candidate list; generating a prediction block for a current block based on the first mode and the motion vector.

9. When the first mode is a mode combining the CIIP mode and the geometric partitioning mode, determining a motion vector from the merge candidate list includes determining a motion vector for each of two partitioned regions from the merge candidate list; When the first mode is a mode that combines the CIIP mode and the geometric partitioning mode, the step of generating a prediction block of the current block includes: generating a final inter-predicted block for the current block based on the two divided regions and a motion vector for each of the two divided regions; generating an intra-predicted block for the current block based on at least one reference pixel adjacent to the current block; 10. The method of claim 8, further comprising: generating a predicted block of the current block by performing a weighted average of the final inter predicted block and the intra predicted block.

10. The step of generating a final inter-predicted block for the current block includes: generating an inter-predicted block for each of the two divided regions using a motion vector for each of the two divided regions; and combining or weighted averaging the inter-predicted blocks for each of the two divided regions to generate a final inter-predicted block for the current block.

11. When the first mode is a mode combining the CIIP mode and the merge mode using the motion vector difference, the step of generating a prediction block for the current block includes: generating an inter-prediction block for the current block based on the compensated motion vector; generating an intra-predicted block for the current block based on at least one reference pixel adjacent to the current block; generating a predicted block of the current block by performing a weighted average of the inter predicted block and the intra predicted block; 9. The method of claim 8, wherein the compensated motion vector is generated based on the motion vector and motion vector differential information.

12. When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, determining a motion vector from the merge candidate list includes determining a motion vector for each of two divided regions from the merge candidate list; When the first mode is a mode combining the geometric partition mode and the merge mode using the motion vector difference, the step of generating a prediction block for the current block includes: generating an inter-prediction block for each of the two divided regions using the corrected motion vector; generating a prediction block of the current block by combining or weighted averaging the inter-prediction blocks for each of the two divided regions; 9. The method of claim 8, wherein the compensated motion vector is generated based on motion vectors and motion vector difference information for each of the two divided regions.

13. When the first mode is a mode combining the CIIP mode, the geometric partitioning mode, and the merge mode using the motion vector difference, determining a motion vector from the merge candidate list includes determining a motion vector for each of two divided regions from the merge candidate list; When the first mode is a mode that combines the CIIP mode, the geometric partitioning mode, and the merge mode using the motion vector difference, the step of generating a prediction block for the current block includes: generating a final inter-predicted block for the current block based on the two divided regions and the corrected motion vectors for each of the two divided regions; generating an intra-predicted block for the current block based on at least one reference pixel adjacent to the current block; generating a prediction block of the current block by performing a weighted average of the final inter prediction block and the intra prediction block; 9. The method of claim 8, wherein the compensated motion vector is generated based on motion vectors and motion vector difference information for each of the two divided regions.

14. The step of generating a final inter-predicted block for the current block includes: generating an inter-predicted block for each of the two divided regions using the corrected motion vector for each of the two divided regions; and combining or weighted averaging the inter-predicted blocks for each of the two divided regions to generate a final inter-predicted block for the current block.

15. A computer-readable recording medium storing a bitstream generated by a video encoding method, comprising: The video encoding method comprises: determining a first mode based on at least one of a CIIP mode, a geometric partition mode, and a merge mode using a motion vector difference; determining a motion vector from the merging candidate list; generating a prediction block for a current block based on the first mode and the motion vector.

Images