Image encoding / decoding method and apparatus, and recording medium having stored bit stream

Abstract

Provided in the present invention is an image decoding method comprising the steps of: partitioning a current block into a first partition and a second partition according to a partition boundary; determining a first basic motion vector and a second basic motion vector corresponding to the first partition and the second partition; determining a first corrected motion vector and a second corrected motion vector by correcting the first base motion vector and the second base motion vector; determining a first prediction block for the first partition and a second prediction block for the second partition according to the first corrected motion vector and the second corrected motion vector; a final prediction block is determined based on the first prediction block and the second prediction block.

Description

Technical Field

[0001] The present invention relates to an image encoding / decoding method and apparatus and a recording medium for storing a bit stream. More specifically, the present invention relates to an image encoding / decoding method and apparatus using an inter-frame prediction method and a recording medium for storing a bit stream. Background Art

[0002] Recently, in various application fields, the demand for high-resolution, high-quality images such as Ultra High Definition (UHD) images is increasing. As image data becomes higher in resolution and quality, the amount of data increases relatively compared to existing image data. Therefore, when image data is transmitted using a medium such as existing wired and wireless broadband lines or stored using existing storage media, transmission and storage costs increase. In order to solve these problems that arise as image data becomes higher in resolution and quality, efficient image encoding / decoding technology for images with higher resolution and quality is needed.

[0003] In inter prediction, a method of partitioning a coding unit block into partitions having various shapes and predicting each partition according to different motion information has been discussed. At this time, various methods for accurately predicting a mixed area of ​​a partition boundary of a partition have been discussed. Summary of the invention

[0004] Technical issues

[0005] An object of the present invention is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] Another object of the present invention is to provide a recording medium for storing a bit stream generated by the image decoding method or apparatus provided by the present invention.

[0007] Technical Solution

[0008] An image decoding method according to an embodiment of the present invention may include: partitioning a current block into a first partition and a second partition according to a partition boundary; determining a first basic motion vector and a second basic motion vector corresponding to the first partition and the second partition; determining a first refined motion vector and a second refined motion vector by refining the first basic motion vector and the second basic motion vector; determining a first prediction block for the first partition and a second prediction block for the second partition according to the first refined motion vector and the second refined motion vector; and determining a final prediction block based on the first prediction block and the second prediction block.

[0009] According to one embodiment, when determining the first refined motion vector and the second refined motion vector, the first refined motion vector and the second refined motion vector are determined according to the distance between the current picture including the current block and the first reference picture referenced by the first partition and the distance between the current picture and the second reference picture referenced by the second partition.

[0010] According to one embodiment, the size of the first differential motion vector representing the difference between the first basic motion vector and the first refined motion vector and the size of the second differential motion vector representing the difference between the second basic motion vector and the second refined motion vector may be proportional to the distance between the current picture and the first reference picture and the distance between the current picture and the second reference picture.

[0011] According to one embodiment, the magnitudes of the first differential motion vector and the second differential motion vector may be limited within a predetermined range.

[0012] According to one embodiment, when determining the first refined motion vector and the second refined motion vector, the first refined motion vector and the second refined motion vector may be determined so that distortion between a first prediction block indicated by the first refined motion vector and a second prediction block indicated by the second refined motion vector is minimized.

[0013] According to one embodiment, when determining the final prediction block, the final prediction block may be determined according to a weighted sum of the first prediction block and the second prediction block.

[0014] According to one embodiment, the weighted sum of the first prediction block and the second prediction block can be determined by a first weighting value and a second weighting value, wherein the first weighting value is determined based on the distance between a current picture including the current block and a first reference picture referenced by the first partition, and the second weighting value is determined based on the distance between the current picture and the second reference picture referenced by the second partition.

[0015] According to one embodiment, the first weighting value applied to the first prediction block may be proportional to the distance between the current picture and the second reference picture, and the second weighting value applied to the second prediction block may be proportional to the distance between the current picture and the first reference picture.

[0016] According to one embodiment, when determining the first basic motion vector and the second basic motion vector, a first L0 basic motion vector and a first L1 basic motion vector corresponding to the first partition may be determined, and a second L0 basic motion vector and a second L1 basic motion vector corresponding to the second partition may be determined.

[0017] According to one embodiment, when determining the first refined motion vector and the second refined motion vector, the first L0 refined motion vector and the first L1 refined motion vector may be determined by refining the first L0 basic motion vector and the first L1 basic motion vector, and the second L0 refined motion vector and the second L1 refined motion vector may be determined by refining the second L0 basic motion vector and the second L1 basic motion vector.

[0018] According to one embodiment, when determining the first prediction block and the second prediction block, the first prediction block may be determined according to the first L0 refined motion vector and the first L1 refined motion vector, and the second prediction block may be determined according to the second L0 refined motion vector and the second L1 refined motion vector.

[0019] According to one embodiment, when determining the first L0-refined motion vector, the first L1-refined motion vector, the second L0-refined motion vector, and the second L1-refined motion vector, the first L0-refined motion vector and the first L1-refined motion vector are determined according to a distance between a current picture including the current block and a first L0 reference picture referenced by a first partition, and a distance between the current picture and a first L1 reference picture referenced by the first partition.

[0020] According to one embodiment, the second L0 refined motion vector and the second L1 refined motion vector may be determined according to the distance between the current picture and the second L0 reference picture referenced by the second partition and the distance between the current picture and the second L1 reference picture referenced by the second partition.

[0021] According to one embodiment, the size of the first L0 differential motion vector representing the difference between the first L0 base motion vector and the first L0 refined motion vector and the size of the first L1 differential motion vector representing the difference between the first L1 base motion vector and the first L1 refined motion vector may be proportional to the distance between the current picture and the first L0 reference picture and the distance between the current picture and the first L1 reference picture.

[0022] According to one embodiment, the size of the second L0 differential motion vector representing the difference between the second L0 base motion vector and the second L0 refined motion vector and the size of the second L1 differential motion vector representing the difference between the second L1 base motion vector and the second L1 refined motion vector may be proportional to the distance between the current picture and the second L0 reference picture and the distance between the current picture and the second L1 reference picture.

[0023] According to one embodiment, the sizes of the first L0 differential motion vector, the first L1 differential motion vector, the second L0 differential motion vector, and the second L1 differential motion vector may be limited within a predetermined range.

[0024] According to one embodiment, determining the first prediction block and the second prediction block may include: determining the first L0 prediction block and the first L1 prediction block according to the first L0 refined motion vector and the first L1 refined motion vector, and determining the second L0 prediction block and the second L1 prediction block according to the second L0 refined motion vector and the second L1 refined motion vector; and determining the first prediction block according to a weighted sum of the first L0 prediction block and the first L1 prediction block, and determining the second prediction block according to a weighted sum of the second L0 prediction block and the second L1 prediction block.

[0025] According to one embodiment, when determining the first L0-refined motion vector and the first L1-refined motion vector, the first L0-refined motion vector and the first L1-refined motion vector may be determined so that distortion between the first L0 prediction block indicated by the first L0-refined motion vector and the first L1 prediction block indicated by the first L1-refined motion vector is minimized.

[0026] According to one embodiment, when determining the second L0 refined motion vector and the second L1 refined motion vector, the second L0 refined motion vector and the second L1 refined motion vector may be determined so that distortion between the second L0 prediction block indicated by the second L0 refined motion vector and the second L1 prediction block indicated by the second L1 refined motion vector is minimized.

[0027] An image encoding method according to an embodiment of the present invention may include: partitioning a current block into a first partition and a second partition according to a partition boundary; determining a first basic motion vector and a second basic motion vector corresponding to the first partition and the second partition; determining a first refined motion vector and a second refined motion vector by refining the first basic motion vector and the second basic motion vector; determining a first prediction block for the first partition and a second prediction block for the second partition according to the first refined motion vector and the second refined motion vector; and determining a final prediction block based on the first prediction block and the second prediction block.

[0028] A non-volatile computer-readable recording medium according to an embodiment of the present invention stores a bit stream generated by an image encoding method.

[0029] A transmission method according to an embodiment of the present invention transmits a bit stream generated by an image encoding method.

[0030] The features briefly summarized above with respect to the present invention are merely exemplary aspects of the following detailed description of the invention and do not limit the scope of the invention.

[0031] Beneficial Effects

[0032] The present invention proposes various embodiments of a method for applying decoder-side motion vector refinement in geometric partitioning mode to improve the prediction accuracy of inter-frame prediction.

[0033] In addition, the present invention proposes various implementations of applying bidirectional prediction to each partition in a geometric partition mode to improve the prediction accuracy of inter-frame prediction.

[0034] In the present invention, as the accuracy of inter-frame prediction is improved, the overall encoding efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a block diagram showing the configuration of an encoding device according to an embodiment of the present invention.

[0036] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment of the present invention.

[0037] Figure 3 FIG. 1 is a schematic diagram schematically showing a video coding system to which the present invention can be applied.

[0038] Figure 4 An embodiment of a method of determining one of various inter prediction methods in an inter prediction mode is shown.

[0039] Figure 5 An implementation of decoder-side motion vector refinement in geometric partition mode is shown.

[0040] Figure 6 An implementation of decoder-side motion vector refinement in geometric partition mode with bi-directional prediction applied to each partition is shown.

[0041] Figure 7 An implementation of decoder-side motion vector refinement in geometric partition mode is shown when the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are not the same.

[0042] Figure 8 An implementation of decoder-side motion vector refinement in geometric partition mode with bidirectional prediction applied to each partition is shown when the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are not the same.

[0043] Fig. 9 A method of generating a final prediction block by considering the characteristics of a geometric partitioning mode of bidirectional prediction is shown.

[0044] Fig.10 is a flowchart of an implementation of a decoder-side motion vector refinement method for a geometric partitioning mode according to the present invention.

[0045] Fig.11 A content streaming system to which an embodiment according to the present invention is applicable is exemplarily shown. DETAILED DESCRIPTION

[0046] Best Mode for Carrying Out the Invention

[0047] An image decoding method according to an embodiment of the present invention may include: partitioning a current block into a first partition and a second partition according to a partition boundary, determining a first basic motion vector and a second basic motion vector corresponding to the first partition and the second partition, determining a first refined motion vector and a second refined motion vector by refining the first basic motion vector and the second basic motion vector, determining a first prediction block for the first partition and a second prediction block for the second partition according to the first refined motion vector and the second refined motion vector, and determining a final prediction block based on the first prediction block and the second prediction block.

[0048] Mode for carrying out the invention

[0049] The present invention may have various modifications and embodiments, and specific embodiments are shown in the drawings and described in detail in the detailed specification. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents or alternatives contained in the spirit and technical scope of the present invention. The same figure marks in the drawings indicate the same or similar functions in various aspects. For a clearer description, the shapes and sizes of the elements in the drawings can be provided by way of example. The detailed description of the exemplary embodiments described below refers to the drawings, which show specific embodiments by way of example. These embodiments are described in full detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments are different from each other, but not necessarily mutually exclusive. For example, without departing from the spirit and scope of the present invention with reference to an embodiment, the specific shapes, structures and features described herein can be implemented in other embodiments. It should also be understood that without departing from the spirit and scope of the embodiment, the position or arrangement of each component in each disclosed embodiment can be changed. Accordingly, the detailed description set forth below is not intended to be restrictive, and the scope of the exemplary embodiments is limited only by the full range of equivalent forms assigned by the attached claims and these claims (if appropriately described).

[0050] In the present invention, the terms first, second, etc. can be used to describe various components, but the components should not be limited by the terms. The terms are only used for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, the first component can be called the second component, and similarly, the second component can also be called the first component. The term is and / or includes a combination of multiple related description items or any item in multiple related description items.

[0051] The components shown in the embodiments of the present invention are depicted independently to indicate different characteristic functions, and do not represent that each component is formed as a separate hardware or software configuration unit. That is, for ease of explanation, each component is listed and included as a separate component, and at least two of the components can be combined to form a single component, or one component can be divided into multiple components to perform functions, as long as it does not depart from the essence of the present invention, the embodiment in which the components are integrated and the embodiment in which each component is divided are also included in the scope of the present invention.

[0052] The terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, some components of the present invention are not necessary components for performing necessary functions in the present invention, and can be optional components that are only used to improve performance. The present invention can be realized by only including necessary components for realizing the gist of the present invention and not including components that are only used to improve performance, and the structure that only includes necessary components and does not include optional components that are only used to improve performance is also included in the scope of the present invention.

[0053] In an embodiment, the term "at least one" may refer to one of a number greater than or equal to 1, such as 1, 2, 3, and 4. In an embodiment, the term "plurality" may refer to one of a number greater than or equal to 2, such as 2, 3, and 4.

[0054] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing the embodiments of the present specification, if it is determined that the detailed description of the related known configuration or function will obscure the subject of the present specification, the detailed description will be omitted, the same figure numerals will be used for the same components in the drawings, and the repeated description of the same components will be omitted.

[0055] Description of terms

[0056] Hereinafter, "image" may refer to a picture constituting a video, and may also refer to the video itself. For example, "encoding and / or decoding of an image" may refer to "encoding and / or decoding of a video", and may also refer to "encoding and / or decoding of one of the images constituting the video".

[0057] Hereinafter, "moving image" and "video" may be used with the same meaning and may be used interchangeably. In addition, the target image may be an encoding target image as a target of encoding and / or a decoding target image as a target of decoding. In addition, the target image may be an input image input to an encoding device, and may be an input image input to a decoding device. Here, the target image may have the same meaning as the current image.

[0058] Hereinafter, “image”, “picture”, “frame” and “screen” may be used with the same meaning and may be used interchangeably.

[0059] Hereinafter, a "target block" may be an encoding target block as a target of encoding and / or a decoding target block as a target of decoding. In addition, a target block may be a current block as a target of current encoding and / or decoding. For example, "target block" and "current block" may be used with the same meaning and may be used interchangeably.

[0060] In the following, "block" and "unit" may be used with the same meaning and may be used interchangeably. In addition, a "unit" may be represented as including a luma component block and a chroma component block corresponding thereto, so as to distinguish it from a block. For example, a coding tree unit (CTU) may consist of a luma component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated therewith.

[0061] Hereinafter, "sample", "picture element", and "pixel" may be used with the same meaning and may be used interchangeably. Herein, a sample may mean a basic unit constituting a block.

[0062] Hereinafter, “inter-frame” and “inter-screen” may be used with the same meaning and may be used interchangeably.

[0063] Hereinafter, “intra-frame” and “intra-screen” may be used with the same meaning and may be used interchangeably.

[0064] Figure 1 is a block diagram showing the configuration of an encoding device according to an embodiment of the present invention.

[0065] The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device 100 may sequentially encode one or more images.

[0066] refer to Figure 1The encoding device 100 may include: an image partitioning unit 110, an intra-frame prediction unit 120, a motion prediction unit 121, a motion compensation unit 122, a switch 115, a subtractor 113, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 117, a filtering unit 180 and a reference picture buffer 190.

[0067] In addition, the encoding device 100 may generate a bitstream including information encoded by encoding the input image, and output the generated bitstream. The generated bitstream may be stored in a computer-readable recording medium, or may be streamed through a wired / wireless transmission medium.

[0068] The image partitioning unit 110 can partition the input image into various forms to improve the efficiency of video encoding / decoding. That is, the input video consists of multiple pictures, and for compression efficiency, parallel processing, etc., a picture can be hierarchically partitioned and processed. For example, a picture can be partitioned into one or more tiles or slices, and then partitioned into multiple codec tree units (CTUs) again. Alternatively, a picture can first be partitioned into multiple sub-pictures defined as rectangular slice groups, and each sub-picture can be partitioned into tiles / slices. Here, sub-pictures can be used to support the function of partially independently encoding / decoding and transmitting pictures. Since multiple sub-pictures can be reconstructed separately, this has the advantage of easy editing in applications where multi-channel inputs are configured as one picture. In addition, tiles can be divided horizontally to generate bricks. Here, bricks can be used as basic units for parallel processing within a picture. In addition, a CTU can be recursively partitioned into a quad tree (quad tree, QT), and the terminal node of the partition can be defined as a coding unit (Coding Unit, CU). The CU can be partitioned into a PU (Prediction Unit) as a prediction unit and a TU (Transform Unit) as a transform unit to perform prediction and partitioning. On the other hand, the CU can be used as a prediction unit and / or a transform unit itself. Here, for flexible partitioning, each CTU can be recursively partitioned into multi-type trees (MTT) and quadtrees (QT). Partitioning the CTU into multi-type trees can start from the terminal node of the QT, and the MTT can be composed of a binary tree (BT) and a triple tree (TT). For example, the MTT structure can be classified into a vertical binary split mode (SPLIT_BT_VER), a horizontal binary split mode (SPLIT_BT_HOR), a vertical ternary split mode (SPLIT_TT_VER), and a horizontal ternary split mode (SPLIT_TT_HOR). In addition, during partitioning, the minimum block size (MinQTSize) of the quadtree of the luma block can be set to 16×16, the maximum block size (MaxBtSize) of the binary tree can be set to 128×128, and the maximum block size (MaxTtSize) of the ternary tree can be set to 64×64. In addition, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the ternary tree can be specified as 4×4, and the maximum depth (MaxMttDepth) of the multi-type tree can be specified as 4. In addition, in order to improve the coding efficiency of the I slice, a dual tree that uses the CTU partition structure of the luma component and the chroma component differently can be applied.On the other hand, in P slices and B slices, the luma and chroma coding tree blocks (CTBs) within a CTU can be partitioned into a single tree that shares the coding tree structure.

[0069] The encoding device 100 may perform encoding on the input image in intra mode and / or inter mode. Alternatively, the encoding device 100 may perform encoding on the input image in a third mode other than intra mode and inter mode (e.g., IBC mode, palette mode, etc.). However, if the third mode has functional characteristics similar to those of the intra mode or inter mode, it may be classified as the intra mode or inter mode for ease of explanation. In the present invention, the third mode is classified and described separately only when a specific description of the third mode is required.

[0070] When the intra mode is used as the prediction mode, the switch 115 can switch to the intra mode, and when the inter mode is used as the prediction mode, the switch 115 can switch to the inter mode. Here, the intra mode can represent the intra prediction mode, and the inter mode can represent the inter prediction mode. The encoding device 100 can generate a prediction block for an input block of an input image. In addition, the encoding device 100 can encode a residual block using the residual of the input block and the prediction block after generating the prediction block. The input image can be referred to as the current image as the current encoding target. The input block can be referred to as the current block or the encoding target block as the current encoding target.

[0071] When the prediction mode is intra mode, the intra prediction unit 120 may use samples of the already encoded / decoded blocks around the current block as reference samples. The intra prediction unit 120 may perform spatial prediction of the current block by utilizing the reference samples, or generate prediction samples of the input block by spatial prediction. In this article, intra prediction may refer to intra-picture prediction.

[0072] As the intra prediction method, non-directional prediction modes such as DC mode and planar mode and directional prediction modes (for example, 65 directions) may be applied. Here, the intra prediction method may be expressed as an intra prediction mode or an intra-screen prediction mode.

[0073] When the prediction mode is the inter mode, the motion prediction unit 121 can retrieve the area that best matches the input block from the reference image in the motion prediction process, and derive the motion vector by using the retrieved area. In this case, the search area can be used as the area. The reference image can be stored in the reference picture buffer 190. Here, when encoding / decoding of the reference image is performed, it can be stored in the reference picture buffer 190.

[0074] The motion compensation unit 122 may generate a prediction block of the current block by performing motion compensation using a motion vector. Herein, inter prediction may mean inter-picture prediction or motion compensation.

[0075] When the value of the motion vector is not an integer, the motion prediction unit 121 and the motion compensation unit 122 may generate a prediction block by applying an interpolation filter to a partial area of ​​the reference picture. In order to perform inter-frame prediction or motion compensation, it may be determined based on the codec unit whether the motion prediction and motion compensation mode of the prediction unit included in the codec unit is one of the skip mode, merge mode, advanced motion vector prediction (AMVP) mode, and intra block copy (IBC) mode, and inter-frame prediction or motion compensation may be performed according to each mode.

[0076] In addition, based on the above-mentioned inter-frame prediction method, the affine (AFFINE) mode based on sub-PU prediction, the sub-block-based temporal motion vector prediction (SbTMVP) mode, the PU-based prediction and MVD merge (Merge with MVD, MMVD) mode and the geometric partitioning mode (Geometric Partitioning Mode, GPM) mode can be applied. In addition, in order to improve the performance of each mode, history based MVP (HMVP), paired average MVP (PAMVP), combined intra / inter prediction (CIIP), adaptive motion vector resolution (AMVR), bi-directional optical flow (BDOF), bi-predictive with CU Weights (BCW), local illumination compensation (LIC), template matching (TM), overlapped block motion compensation (OBMC), etc. can be applied.

[0077] The subtractor 113 may generate a residual block by utilizing the difference between the input block and the prediction block. The residual block may be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming or quantizing, or transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal of a block unit.

[0078] The transform unit 130 may generate a transform coefficient by performing a transform on the residual block, and output the generated transform coefficient. Herein, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When the transform skip mode is applied, the transform unit 130 may skip the transform of the residual block.

[0079] The quantization level may be generated by applying quantization to a transform coefficient or a residual signal. Hereinafter, the quantization level may also be referred to as a transform coefficient in an embodiment.

[0080] For example, a 4×4 luminance residual block generated by intra prediction is transformed using a base vector based on a discrete sine transform (DST), and the remaining residual block can be transformed using a base vector based on a discrete cosine transform (DCT). In addition, the transform block is partitioned into a quadtree shape of a block using a residual quad tree (RQT) technique, and after transforming and quantizing each transform block partitioned by the RQT, a coded block flag (cbf) can be transmitted when all coefficients become 0 to improve coding efficiency.

[0081] As another alternative, a Multiple Transform Selection (MTS) technique that selectively uses multiple transform bases to perform transforms can be applied. That is, instead of partitioning the CU into TUs through RQT, a function similar to TU partitioning can be performed through a sub-block transform (SBT) technique. Specifically, SBT is only applied to inter-prediction blocks, and unlike RQT, the current block can be partitioned into a size of 1 / 2 or 1 / 4 in the vertical or horizontal direction, and then the transform can be performed only on one of the blocks. For example, if the current block is partitioned vertically, the transform can be performed on the leftmost or rightmost block, and if the current block is partitioned horizontally, the transform can be performed on the topmost or bottommost block.

[0082] In addition, a low frequency non-separable transform (LFNST) can be applied, which is a secondary transform technique that additionally transforms the transformed residual signal into the frequency domain by DCT or DST. LFNST additionally performs a transform on a 4×4 or 8×8 low frequency region on the upper left side so that the residual coefficients can be concentrated on the upper left side.

[0083] The quantization unit 140 may generate a quantization level by quantizing the transform coefficient or the residual signal according to a quantization parameter (QP), and output the generated quantization level. Herein, the quantization unit 140 may quantize the transform coefficient by using a quantization matrix.

[0084] For example, a quantizer using a QP value of 0 to 51 may be used. Alternatively, if the image size is large and high coding efficiency is required, a QP of 0 to 63 may be used. In addition, a dependent quantization (DQ) method using two quantizers instead of one quantizer may be applied. DQ performs quantization using two quantizers (e.g., Q0 and Q1), but even if there is no signaling information about the use of a specific quantizer, the quantizer used for the next transform coefficient may be selected based on the current state through a state transition model.

[0085] The entropy encoding unit 150 may generate a bit stream by performing entropy encoding on the value calculated by the quantization unit 140 or the encoding parameter value calculated when encoding is performed according to the probability distribution, and output the bit stream. The entropy encoding unit 150 may perform entropy encoding on information about samples of an image and information for decoding an image. For example, the information for decoding an image may include a syntax element.

[0086] When entropy coding is applied, the symbol is represented so that a smaller number of bits are assigned to symbols with a high probability of occurrence and a larger number of bits are assigned to symbols with a low probability of occurrence, so the size of the bit stream for the symbol to be encoded can be reduced. The entropy coding unit 150 can use coding methods such as exponential Golomb, context-adaptive variable length coding (context-adaptive variable length coding, CAVLC), context-adaptive binary arithmetic coding (context-adaptive binary arithmetic coding, CABAC) for entropy coding. For example, the entropy coding unit 150 can perform entropy coding by using a variable length coding / code (variable length coding / code, VLC) table. In addition, the entropy coding unit 150 can derive a binarization method of the target symbol and a probability model of the target symbol / binary, and perform arithmetic coding and decoding by using the derived binarization method and context model.

[0087] In this regard, when CABAC is applied, in order to reduce the size of the probability table stored in the decoding device, the table probability update method can be changed to a table update method using a simple equation and applied. In addition, two different probability models can be used to obtain more accurate symbol probability values.

[0088] In order to encode the transformation coefficient level (quantization level), the entropy encoding unit 150 may change the coefficients in a two-dimensional block form into a one-dimensional vector form through a transformation coefficient scanning method.

[0089] The coding and decoding parameters may include information (flags, indexes, etc.) encoded in the encoding device 100 and signaled to the decoding device 200, such as syntax elements, and information derived during the encoding or decoding process, and may represent information required when encoding or decoding an image.

[0090] Herein, signaling a flag or an index may mean that in an encoder, the corresponding flag or index is entropy encoded and included in a bitstream, and may mean that in a decoder, the corresponding flag or index is entropy decoded from the bitstream.

[0091] The encoded current image can be used as a reference image for another image to be processed later. Therefore, the encoding device 100 can reconstruct or decode the encoded current image again and store the reconstructed or decoded image in the reference picture buffer 190 as a reference image.

[0092] The quantization level may be dequantized in the dequantization unit 160 or may be inversely transformed in the inverse transform unit 170. The dequantized and / or inversely transformed coefficients may be added to the prediction block by the adder 117. Herein, the dequantized and / or inversely transformed coefficients may represent coefficients on which at least one of dequantization and inverse transformation is performed, and may represent a reconstructed residual block. The dequantization unit 160 and the inverse transform unit 170 may be performed as inverse processes of the quantization unit 140 and the transform unit 130.

[0093] The reconstructed block may pass through the filtering unit 180. The filtering unit 180 may apply a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a bilateral filter (BIF), a luma mapping with chroma scaling (LMCS) filter, etc. to the reconstructed sample, the reconstructed block, or the reconstructed image using all or some filtering techniques. The filtering unit 180 may be referred to as an in-loop filter. In this case, the in-loop filter may also be used as a name that does not include LMCS.

[0094] The deblocking filter can remove block distortion generated in the boundary between blocks. In order to determine whether to apply the deblocking filter, it can be determined whether to apply the deblocking filter to the current block based on samples included in several rows or columns included in the block. When the deblocking filter is applied to the block, different filters can be applied according to the required deblocking filter strength.

[0095] In order to compensate for the coding error using sample adaptive offset, an appropriate offset value may be added to the sample value. Sample adaptive offset can correct the offset from the original image in sample units for the deblocked image. A method of partitioning the samples included in the image into a predetermined number of regions, determining the region to which the offset is applied, and applying the offset to the determined region, or a method of applying the offset in consideration of edge information about each sample may be used.

[0096] The bilateral filter (BIF) can also correct the deviation from the original image on a sample-by-sample basis for the image on which deblocking has been performed.

[0097] The adaptive loop filter may perform filtering based on the comparison result of the reconstructed image with the original image. The samples included in the image may be partitioned into predetermined groups, the filter to be applied to each group may be determined, and differential filtering may be performed on each group. Information on whether the ALF is applied may be signaled by a codec unit (CU), and the form and coefficients of the adaptive loop filter to be applied to each block may vary.

[0098] In luma mapping with chroma scaling (LMCS), luma mapping (LM) means remapping luma values ​​by a piecewise linear model, and chroma scaling (CS) means a technique for scaling the residual values ​​of chroma components according to the average luma value of a prediction signal. In particular, LMCS can be used as an HDR correction technique that reflects the characteristics of a High Dynamic Range (HDR) image.

[0099] The reconstructed block or the reconstructed image that has passed through the filtering unit 180 may be stored in the reference picture buffer 190. The reconstructed block that has passed through the filtering unit 180 may be part of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed block that has passed through the filtering unit 180. The stored reference image may be used for inter-frame prediction or motion compensation later.

[0100] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment of the present invention.

[0101] The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.

[0102] refer to Figure 2 The decoding device 200 may include: an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, a motion compensation unit 250, an adder 201, a switch 203, a filtering unit 260 and a reference picture buffer 270.

[0103] The decoding device 200 may receive a bit stream output from the encoding device 100. The decoding device 200 may receive a bit stream stored in a computer-readable recording medium, or may receive a bit stream streamed through a wired / wireless transmission medium. The decoding device 200 may decode the bit stream in an intra-frame mode or an inter-frame mode. In addition, the decoding device 200 may generate a reconstructed image or a decoded image generated by decoding, and output the reconstructed image or the decoded image.

[0104] When the prediction mode for decoding is the intra mode, the switch 203 may switch to the intra mode. Alternatively, when the prediction mode for decoding is the inter mode, the switch 203 may switch to the inter mode.

[0105] The decoding device 200 can obtain a reconstructed residual block by decoding the input bit stream and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding device 200 can generate a reconstructed block that becomes a decoding target by adding the reconstructed residual block and the prediction block. The decoding target block can be called a current block.

[0106] The entropy decoding unit 210 may generate symbols by entropy decoding the bit stream according to the probability distribution. The generated symbols may include symbols in the form of quantization levels. In this article, the entropy decoding method may be an inverse process of the above entropy encoding method.

[0107] The entropy decoding unit 210 may change the coefficients of the one-dimensional vector shape into coefficients of the two-dimensional block shape through a transform coefficient scanning method to decode the transform coefficient level (quantization level).

[0108] The quantization level may be inversely quantized in the inverse quantization unit 220 or inversely transformed in the inverse transform unit 230. The quantization level may be a result of inverse quantization and / or inverse transformation and may be generated as a reconstructed residual block. Herein, the inverse quantization unit 220 may apply a quantization matrix to the quantization level. The inverse quantization unit 220 and the inverse transform unit 230 applied to the decoding device may apply the same technology as the inverse quantization unit 160 and the inverse transform unit 170 applied to the aforementioned encoding device.

[0109] When the intra mode is used, the intra prediction unit 240 can generate a prediction block by performing spatial prediction on the current block, which uses sample values ​​of blocks that have been decoded around the decoding target block. The intra prediction unit 240 applied to the decoding device can apply the same technology as the intra prediction unit 120 applied to the aforementioned encoding device.

[0110] When the inter-frame mode is used, the motion compensation unit 250 can generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer 270 on the current block. When the value of the motion vector is not an integer value, the motion compensation unit 250 can generate a prediction block by applying an interpolation filter to a partial area within the reference image. In order to perform motion compensation, it can be determined based on the codec unit whether the motion compensation mode of the prediction unit included in the corresponding codec unit is a skip mode, a merge mode, an AMVP mode, or a current picture reference mode, and motion compensation can be performed according to each mode. The motion compensation unit 250 applied to the decoding device can apply the same technology as the motion compensation unit 122 applied to the above-mentioned encoding device.

[0111] The adder 201 may generate a reconstructed block by adding the reconstructed residual block and the prediction block. The filtering unit 260 may apply at least one of an inverse LMCS, a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filtering unit 260 applied to the decoding device may apply the same filtering technique as the filtering unit 180 applied to the aforementioned encoding device.

[0112] The filtering unit 260 may output a reconstructed image. The reconstructed block or the reconstructed image may be stored in the reference picture buffer 270 and used for inter-frame prediction. The reconstructed block that has passed through the filtering unit 260 may be part of the reference image. That is, the reference image may be a reconstructed image composed of the reconstructed blocks that have passed through the filtering unit 260. The stored reference image may be used for inter-frame prediction or motion compensation later.

[0113] Figure 3 FIG. 1 is a schematic diagram schematically showing a video coding system to which the present invention can be applied.

[0114] The video coding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may transmit the encoded video and / or image information or data to the decoding device 20 in the form of a file or streaming through a digital storage medium or a network.

[0115] The encoding device 10 according to the embodiment may include a video source generating unit 11, an encoding unit 12, and a transmitting unit 13. The decoding device 20 according to the embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmitting unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may include a display unit, and the display unit may be configured as a separate device or an external component.

[0116] The video source generation unit 11 can obtain the video / image by capturing, synthesizing or generating the video / image process. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generation device may include, for example, a computer, a tablet computer, a smart phone, etc., and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc., in which case the video / image capture process may be replaced by a process for generating relevant data.

[0117] The encoding unit 12 may encode the input video / image. For compression and encoding efficiency, the encoding unit 12 may perform a series of processes such as prediction, transformation, and quantization. The encoding unit 12 may output the encoded data (encoded video / image information) in the form of a bit stream. The detailed configuration of the encoding unit 12 may also be the same as described above. Figure 1 The encoding device 100 is configured in the same manner.

[0118] The sending unit 13 may send the encoded video / image information or data output in the form of a bit stream to the receiving unit 21 of the decoding device 20 through a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sending unit 13 may include an element for generating a media file in a predetermined file format, and may include an element for transmission through a broadcast / communication network. The receiving unit 21 may extract / receive a bit stream from a storage medium or a network and send it to the decoding unit 22.

[0119] The decoding unit 22 can decode the video / image by performing a series of processes such as inverse quantization, inverse transformation, and prediction corresponding to the operations of the encoding unit 12. The detailed configuration of the decoding unit 22 can also be similar to Figure 2 The above-mentioned decoding device 200 is configured in the same manner.

[0120] The rendering unit 23 may render the decoded video / image. The rendered video / image may be displayed by the display unit.

[0121] In the present invention, a method for applying decoder-side motion vector refinement to geometric partitioning mode is provided.

[0122] Geometric partitioning mode (GPM) is a technique that partitions a codec unit (CU) into two partitions by a partition boundary, independently determines prediction signals corresponding to the two partitions, and then generates a final prediction block by a weighted sum of the generated prediction signals. Decoder-side motion vector refinement (DMVR) is a technique that refines motion vectors in the decoder without separately encoding the information.

[0123] In the geometric partitioning mode, the current block is partitioned into two partitions by a straight partition boundary. Then, prediction blocks for the areas of the two partitions are generated according to inter-inter prediction or intra-inter prediction. Then, the prediction block of the current block is generated by weighted summing of the prediction signals of the two prediction blocks.

[0124] In inter-frame prediction in geometric partition mode, two partitions have independent motion information. In addition, an inter-frame prediction signal for each region is generated based on each independent motion information. At this time, unidirectional motion compensation or bidirectional motion compensation is performed on each partition.

[0125] In the intra-inter prediction of the geometric partition mode, one of the two partitions is predicted by intra prediction and the other partition is predicted by inter prediction. At this time, the partition using inter prediction is predicted according to unidirectional motion compensation or bidirectional motion compensation.

[0126] When only unidirectional motion compensation is used for a partition, a merge candidate list of a geometric partition pattern including only unidirectional motion information is constructed. That is, in a conventional merge candidate list, a merge candidate list having only unidirectional motion information can be generated by utilizing the motion information of the L0 list for candidates with even merge indexes and the motion information of the L1 list for candidates with odd merge indexes. Conversely, the motion information of the L1 list can be used for candidates with even merge indexes, and the motion information of the L0 list can be used for candidates with odd merge indexes. In the generated merge candidate list, inter-frame prediction can be performed on the partition based on the merge candidate for the partition using inter-frame prediction.

[0127] According to an embodiment, only bidirectional motion compensation may be used for each partition. At this time, the merge candidate of the partition may include both motion information for the L0 direction and motion information for the L1 direction. The merge candidate list of each partition may be determined independently. Alternatively, two partitions may share a merge candidate list, but the partitions may be set to involve different merge candidates.

[0128] Decoder-side motion vector refinement is a method in which, during decoding of a bidirectional motion vector, a bidirectional motion vector is refined by searching for motion based on bilateral matching (BM). To refine the motion vector, the surroundings of the reference picture are searched from the initial motion vector. At this time, a motion vector is searched that minimizes the degree of distortion of the two reference blocks based on bilateral matching.

[0129] Hereinafter, various intra prediction modes applied in intra prediction will be described. In addition, conditions for applying decoder-side motion vector refinement to normal merge mode and geometric partition mode will be described.

[0130] Figure 4 An implementation 400 of a method for determining one of various inter prediction methods in inter prediction mode is shown.

[0131] According to the implementation scheme 400, when a current block is predicted and encoded or decoded by inter prediction, in step 410, it is determined whether the inter prediction mode of the current block is a merge mode or an Advanced Motion Vector Prediction (AMVP) mode.

[0132] The merge mode is an inter-prediction mode that obtains motion information such as the motion vector of the current block and the motion information of the reference picture from the neighboring blocks of the current block. On the other hand, according to the AMVP mode, the predicted motion vector is obtained from the neighboring blocks of the current block, and other motion information such as the differential motion vector and the reference picture information in addition to the predicted motion vector is obtained by parsing the bitstream. Therefore, the AMVP mode is different from the merge mode, in which the motion information of the neighboring blocks is all used for the prediction of the current block.

[0133] If the inter prediction mode of the current block is the merge mode, it is determined whether the inter prediction mode of the current block is the subblock merge mode in step 420. If the inter prediction mode of the current block is the subblock merge mode, then in step 422, the current block is partitioned into a plurality of subblocks according to the subblock merge mode, and each subblock may be predicted based on a motion vector according to an affine transformation.

[0134] If the inter prediction mode of the current block is not the sub-block merge mode, then at step 430, it is determined whether the inter prediction mode of the current block is the regular merge mode. If the inter prediction mode of the current block is not the regular merge mode, then at step 432, it is determined whether the inter prediction mode of the current block is the combined intra inter prediction (CIIP) mode.

[0135] If the inter prediction mode of the current block is not the intra-inter prediction mode, the inter prediction mode of the current block is determined as the geometric partition mode in step 434. The geometric partition mode is an inter prediction mode that partitions the current block into two partitions based on a predetermined boundary and determines a final prediction block of the current block by combining two prediction blocks for the two partitions. In step 436, the inter prediction mode of the current block is determined as a combined intra-inter prediction mode. According to the combined intra-inter prediction mode, a final prediction block of the current block can be determined by combining a prediction block according to intra prediction of the current block and a prediction block according to inter prediction of the current block.

[0136] If the geometric partitioning mode is applied to the current block, then when certain conditions are met, at step 438, the motion vector of the current block may be refined according to the decoder-side motion vector refinement mode.

[0137] If the inter prediction mode of the current block is the normal merge mode, it is determined whether the inter prediction mode of the current block is a merge mode with motion vector difference (MMVD) at step 440. According to the merge mode with motion vector difference, a refined motion vector of the current block is determined by adding a differential motion vector to a motion vector obtained from an adjacent block. In the merge mode with motion vector difference, the direction of the differential motion vector may be limited to one of +x, -x, +y, and -y. In addition, in the merge mode with motion vector difference, the size of the differential motion vector may be limited to be selected from a limited number of predetermined size candidates.

[0138] If the inter prediction mode of the current block is a merge mode with a motion vector difference, the current block is predicted according to the merge mode with a motion vector difference. In addition, if the current block is bidirectionally predicted, then in step 446, the prediction block of the current block can be adjusted by a bidirectional optical flow (BDOF) mode.

[0139] If the inter prediction mode of the current block is not a merge mode with motion vector difference, then at step 444, a regular merge mode is applied to the current block. Additionally, if the current block is bi-directionally predicted, then at step 450, the motion vector of the current block may be refined according to a decoder-side motion vector refinement mode. Additionally, at step 452, similar to step 446, the prediction block of the current block may be adjusted by a bi-directional optical flow mode.

[0140] Decoder-side motion vector refinement is a method in which, when decoding bidirectional motion vectors derived in conventional merge mode, motion vectors are refined by a motion vector search process based on bilateral matching without parsing additional information. According to decoder-side motion vector refinement, the accuracy of motion vectors in conventional merge mode can be improved. In addition, the coding efficiency of conventional merge mode can be improved accordingly.

[0141] Despite Figure 4 In the description, the decoder side motion vector refinement mode is described as being applied only to the normal merge mode, but according to an implementation scheme, the decoder side motion vector refinement mode can also be applied to the sub-block merge mode, the geometric partition mode, the combined intra-frame inter-frame prediction mode, and the merge mode with motion vector difference.

[0142] Based on the flag information parsed from the bitstream, it can be Figure 4 The inter prediction mode of the current block is determined in steps 410, 420, 430, 432 and 440. For example, in steps 410, 420, 430 and 440, a merge flag, a sub-block merge flag, a regular merge flag, a combined intra-frame inter-frame prediction flag, and a merge mode with a motion vector difference flag can be parsed from the bitstream, respectively.

[0143] In the following, the application conditions of the decoder-side motion vector refinement of steps 438 and 450 will be described.

[0144] According to one embodiment, decoder-side motion vector refinement may be applied when the codec unit (CU) block is in merge mode. Additionally, decoder-side motion vector refinement may be applied when the codec unit block is not in sub-block merge mode with inter prediction according to affine transform applied. Additionally, decoder-side motion vector refinement may be applied when merge mode with motion vector difference (MMVD) is not applied to the codec unit block. Additionally, decoder-side motion vector refinement may be applied when the codec unit block is in geometric partition mode with bi-directional prediction applied.

[0145] According to one embodiment, decoder-side motion vector refinement may be applied when the codec unit block is in bidirectional prediction mode. In addition, according to one embodiment, decoder-side motion vector refinement may be applied when two reference pictures referenced by the codec unit block are respectively located in a temporally opposite direction to the current picture. For example, decoder-side motion vector refinement may be applied when a first reference picture of the two reference pictures is temporally before the current picture and a second reference picture is temporally after the current picture.

[0146] According to one embodiment, decoder-side motion vector refinement may be applied when the temporal distance between two reference pictures and the current picture is the same. Temporal distance may refer to the size of the Picture Order Count (POC) difference between the reference picture and the current picture. In the present invention, the distance between pictures represents the size of the temporal distance and the POC difference.

[0147] According to an embodiment, decoder-side motion vector refinement may be applied even when the temporal distances between two reference pictures and the current picture are different in normal merge mode. By not applying the condition of the same temporal distance, decoder-side motion vector refinement may be applied more in normal merge mode. Accordingly, since decoder-side motion vector refinement is performed more by relaxing the performance condition, coding efficiency may be improved. However, in the case of geometric partition mode, decoder-side motion vector refinement may be set to be applied only when the temporal distances between two reference pictures and the current picture are the same.

[0148] According to an embodiment, decoder-side motion vector refinement may be applied even when the temporal distances between two reference pictures and the current picture are different in geometric partition mode. By not applying the condition of the same temporal distance, decoder-side motion vector refinement may be applied more in geometric partition mode. Accordingly, since decoder-side motion vector refinement is performed more by relaxing the performance condition, coding efficiency may be improved. However, in the case of conventional merge mode, decoder-side motion vector refinement may be set to be applied only when the temporal distances between two reference pictures and the current picture are the same.

[0149] According to the implementation scheme, for the normal merge mode and the geometric partition mode, decoder-side motion vector refinement can be applied even when the temporal distance between the two reference pictures and the current picture is different. Accordingly, since decoder-side motion vector refinement is applied to the two prediction modes without being restricted by the temporal distance condition, the coding efficiency can be improved.

[0150] According to one embodiment, decoder-side motion vector refinement may be applied when both reference pictures are short-term reference pictures.

[0151] According to one embodiment, whether to apply decoder-side motion vector refinement may be determined based on the size of the codec unit block. For example, when the size of the codec unit block is greater than a predetermined size, decoder-side motion vector refinement may be applied. The predetermined size may be expressed as the number of luminance samples included in the codec unit block. Additionally, the predetermined size may be a power of 2, such as 64, 128, 256, 512, or 1024.

[0152] According to one embodiment, whether to apply decoder-side motion vector refinement may be determined based on the width and height of the codec unit block. For example, when the height and / or width of the codec unit block is greater than a predetermined value, decoder-side motion vector refinement may be applied. The predetermined value may be a power of 2, such as 4, 8, 16, or 32.

[0153] According to one embodiment, decoder-side motion vector refinement may be applied when the bidirectional codec unit weighting values ​​applied to the two prediction blocks of the codec unit block are the same. The final prediction block of the codec unit block is determined as the weighted average of the two prediction blocks obtained from the bidirectional prediction. In this case, the bidirectional codec unit weighting values ​​are used to determine the weighted average of the two prediction blocks. However, according to an embodiment, even when the bidirectional codec unit weighting values ​​applied to the two prediction blocks are different, decoder-side motion vector refinement may be configured to be applied.

[0154] According to one embodiment, decoder-side motion vector refinement may be applied when the codec block is in a bidirectionally predicted normal merge mode and the bidirectional codec weights of the codec block are the same. If the codec block is not in normal merge mode, decoder-side motion vector refinement may be applied even if the bidirectional codec weights are not the same. For example, when a geometric partitioning mode is applied to a codec block, decoder-side motion vector refinement may be applied regardless of whether the bidirectional codec weights are the same.

[0155] According to one embodiment, decoder-side motion vector refinement may be applied when combined intra-inter prediction (CIIP) is not applied. Combined intra-inter prediction is a prediction method that determines a final prediction block by weighted averaging of a first prediction block derived from intra prediction and a second prediction block derived from inter prediction for a block.

[0156] According to one embodiment, at least one of the above-mentioned multiple conditions may be included in the conditions for performing decoder-side motion vector refinement.

[0157] As the conditions for performing decoder-side motion vector refinement increase, the frequency of decoder-side motion vector refinement may decrease. Conversely, as the conditions for performing decoder-side motion vector refinement decrease, the frequency of decoder-side motion vector refinement may increase. Accordingly, the frequency of motion vector refinement and the encoding efficiency of merge mode may be determined according to the conditions for decoder-side motion vector refinement.

[0158] In the following, a method for performing decoder-side motion vector refinement in geometric partitioning mode will be described.

[0159] Figure 5 An implementation of decoder-side motion vector refinement in geometric partition mode is shown.

[0160] The current picture 500 includes a current block 502. In addition, the current block 502 includes a first partition 504 and a second partition 506. According to one embodiment, the first partition 504 and the second partition 506 may be unidirectionally predicted, respectively.

[0161] The current picture 500 and the L0 reference picture 520 have a POC distance of N. In addition, the current picture 500 and the L1 reference picture 540 also have a POC distance of N. Therefore, the distance between the current picture 500 and the L0 reference picture 520 and the distance between the current picture 500 and the L1 reference picture 540 are the same.

[0162] Based on the motion information of the neighboring blocks, basic motion vectors MV0 522 and MV1 542 in the L0 and L1 directions of the current block 502 applied to the geometric partition mode may be obtained. MV0 522 represents an initial reference block 524 corresponding to the first partition 504. In addition, MV1 542 represents initial reference blocks 544 respectively corresponding to the second partitions 506. In some cases, there may be reference blocks more suitable for prediction of the current block around the initial reference blocks 524 and 544, and a motion vector indicating a more suitable reference block is searched based on bilateral matching.

[0163] MV0 522 and MV1 542 are basic motion vectors in the L0 and L1 directions respectively derived according to the geometric partitioning mode. MV0′ 528 is obtained by refining the basic motion vector MV0 522 in the L0 direction into MV diff 526. In addition, MV1'548 is obtained by refining the basic motion vector MV1 542 in the L1 direction into -MV diff 546 and obtain the motion vector. MV diff 526 and -MV diff 546 are L0 differential motion vector and L1 differential motion vector, respectively. The L0 differential motion vector and the L1 differential motion vector have the same magnitude, but their directions are set to be opposite.

[0164] If the distortion between the reference block indicated by MV0' 528 in the L0 reference picture 520 and the reference block indicated by MV1' 548 in the L1 reference picture 540 is minimized, MV0' 528 and MV1' 548 may be determined as the refined motion vector of the current block 502. At this time, various distortion measurement methods between the two reference blocks may be used, such as the sum of absolute difference (SAD) or the sum of square error (SSE).

[0165] Finally, when MV0′528 and MV1′548 are the refined motion vectors with the minimum distortion, the two prediction blocks P are determined using the refined motion vectors MV0′528 and MV1′548. L0 530 and P L1 550.

[0166] Although in Figure 5 , the unidirectional motion vector of the first partition 504 is described as being in the L0 direction, and the unidirectional motion vector of the second partition 506 is described as being in the L1 direction, but according to an embodiment, the unidirectional motion vector of the first partition 504 may be determined to be in the L1 direction, and the unidirectional motion vector of the second partition 506 may be determined to be in the L0 direction.

[0167] Figure 6 An implementation of decoder-side motion vector refinement in geometric partition mode with bi-directional prediction applied to each partition is shown.

[0168] The current picture 600 includes a current block 602. In addition, the current block 602 includes a first partition 604 and a second partition 606.

[0169] The current picture 600 and the L0 reference picture 620 have a POC distance of N. In addition, the current picture 600 and the L1 reference picture 650 also have a POC distance of N. Therefore, the distance between the current picture 600 and the L0 reference picture 620 and the distance between the current picture 600 and the L1 reference picture 650 are the same.

[0170] According to one embodiment, bidirectional motion prediction may be performed on each of the first partition 604 and the second partition 606. Figure 5 In an embodiment of the present invention, decoder-side motion vector refinement may be performed on the first partition 504 and the second partition 506 by considering the entire area of ​​the current block 502. On the other hand, in the case where the motion vectors of each partition are predicted according to bidirectional motion prediction, Figure 6 In an embodiment of the present invention, decoder-side motion vector refinement can be performed by considering only the area of ​​each partition.

[0171] For the bidirectional motion prediction of the first partition 604, the motion vector MV0 in the L0 direction _R0 622 and the motion vector MV1 in the L1 direction _R0 652 is determined as the basic motion vector. In addition, for the bidirectional motion prediction of the second partition 606, the motion vector MV0 in the L0 direction _R1 632 and the motion vector MV1 in the L1 direction _R1 662 is determined as the basic motion vector.

[0172] For the motion vector refinement of the first partition 604, MV0 is searched symmetrically. _R0 622 and MV1 _R0 In addition, among the plurality of adjacent motion vectors, the motion vectors in the L0 and L1 directions that minimize the distortion between the prediction signal in the L0 direction and the prediction signal in the L1 direction are derived as the refined motion vector MV0′ _R0 628 and MV1' _R0 658.

[0173] By transforming the motion vector MV0 in the L0 direction _R0 622 is refined into MV diff_R0626 to derive MV0' _R0 628. In addition, by converting the motion vector MV1 in the L1 direction _R0 652 is refined into -MV diff_R0 656 derives MV1′ symmetrically with the refined motion vector in the L0 direction _R0 658. At this time, the L0 reference picture 620 is composed of MV0' _R0 Block P indicated by 628 L0_R0 630 and MV1' in L1 reference picture 650 _R0 Block P indicated by 658 L1_R0 660 Distortion Minimized MV0' _R0 628 and MV1' _R0 658 is determined as the refined motion vector. At this point, various distortion measurement methods such as the sum of absolute difference (SAD) or the sum of squared errors (SSE) may be used for the distortion.

[0174] The same method as the refinement of the motion vector of the first partition 604 described above may be applied to the refinement of the motion vector of the second partition 606 .

[0175] For the bidirectional motion prediction of the second partition 606, the motion vector MV0 in the L0 direction _R1 632 and the motion vector MV1 in the L1 direction _R1 662 is determined as the basic motion vector.

[0176] For motion vector refinement of the second partition 606, MV0 is searched symmetrically. _R1 632 and MV1 _R1 In addition, among the plurality of adjacent motion vectors, the motion vectors in the L0 and L1 directions that minimize the distortion between the prediction signal in the L0 direction and the prediction signal in the L1 direction are derived as the refined motion vector MV0′ _R1 638 and MV1' _R1 668.

[0177] By transforming the motion vector MV0 in the L0 direction _R1 632 is refined into MV diff_R1 636 to derive MV0' _R1 638. In addition, by converting the motion vector MV1 in the L1 direction _R1 662 is refined into -MV diff_R1 666 derives MV1′ symmetrically with the refined motion vector in the L0 direction _R1 668. At this time, the L0 reference picture 620 is composed of MV0' _R1 Block P indicated by 638L0_R1 The distortion between 640 and the block P indicated by MV1' in the L1 reference picture 650 _R1 is minimized for MV0' L1_R1 between 670 _R1 638 and MV1' _R1 668 is determined as the refined motion vector. At this time, various distortion measurement methods such as sum of absolute differences (SAD) or sum of squared errors (SSE) can be used for distortion.

[0178] In Figure 6 it is described that the reference pictures of the first partition 604 and the second partition 606 are the same as the L0 reference picture 620 and the L1 reference picture 640. However, the reference pictures of the first partition 604 and the second partition 606 can be different from each other.

[0179] Figure 7 An embodiment of decoder-side motion vector refinement in a geometric partitioning mode is shown when the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are not the same.

[0180] The current picture 700 includes the current block 702. Additionally, the current block 702 includes a first partition 704 and a second partition 706. According to one embodiment, the first partition 704 and the second partition 706 can be unidirectionally predicted respectively.

[0181] In Figure 7 the current picture 700 is a picture at time t, the L0 reference picture 720 and the L1 reference picture 740 are pictures at time t - M (t - M > 0) and time t + N (t + N > 0) respectively (t - M < t + N). Here, M and N are different (M < N) arbitrary positive integer values. Thus, the temporal distance between the L0 reference picture 720 and the current picture 700 and the temporal distance between the L1 reference picture 740 and the current picture 700 are not the same.

[0182] According to Figure 7 , the L0-direction motion vector MV0 722 and the L1-direction motion vector MV1 742 derived in the geometric partitioning mode are basic motion vectors. MV0 722 and MV1 742 indicate the initial reference blocks 724 and 744 respectively. Additionally, the motion vector MV0'728 is determined by refining MV0 722 to MV diff_L0 726. Then, the motion vector MV1'748 is determined by refining MV1 742 to MV diff_L1 746. MV diff_L0 726 is determined from MVdiff_L1 746. When block P L0 730 and block P L1 When the distortion between the current picture and the L0 reference picture is minimized, the motion vectors MV0' 728 and MV1' 748 can be derived as the refined motion vectors in the L0 direction and the L1 direction, respectively. Equation 1 shows that the time distance between the current picture and the L0 reference picture and the time distance between the current picture and the L1 reference picture are calculated by considering the ratio of the time distance between the current picture and the L0 reference picture. diff_L0 MV diff_L1 method.

[0183] [Equation 1]

[0184]

[0185] In Equation 1, M and N represent the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, respectively. Here, M and N are different arbitrary positive integer values. In addition, in Equation 1, MV diff_L1_x , MV diff_L1_y , MV diff_L0_x and MV diff_L0_y Respectively represent MV diff_L1 The x-direction motion information, MV diff_L1 Y-direction motion information, MV diff_L0 The x-direction motion information and MV diff_L0 As shown in Equation 1, by considering the ratio (M:N) of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, the y-direction motion information corresponding to MV is calculated symmetrically. diff_L0 MV diff_L1 The motion vector MV1′ (which is the MV calculated by refining the final L1 direction motion vector MV1) can be diff_L1 (MV diff_L1_x and MV diff_L1_y ) is derived as the refined motion vector in the L1 direction. In equation 1, by diff_L0 The MV is calculated by reflecting the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture in the motion information of diff_L1 Conversely, you can also use the MV diff_L1 The MV is calculated by reflecting the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture in the motion information of diff_L0 sports information.

[0186] By diff_L0_x and MV diff_L0_yMultiply by N / M to determine the MV determined in Equation 1 diff_L1_x and MV diff_L1_y , thus MV diff_L1_x and MV diff_L1_y The value of can be a non-integer value. Therefore, according to one embodiment, MV diff_L1_x and MV diff_L1_y can be adjusted to integer units. In this case, the MV can be rounded or truncated. diff_L1_x and MV diff_L1_y Alternatively, the MV can be rounded or truncated according to the process of rounding or truncation. diff_L1_x and MV diff_L1_y Adjust to a predetermined precision other than integer units. The predetermined precision may be 1 / 2, 1 / 4, etc.

[0187] According to one embodiment, unlike Equation 1, MV may be determined without considering the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture. diff_L1_x , MV diff_L1_y , MV diff_L0_x and MV diff_L0_y For example, MV diff_L0_x and MV diff_L1_x can be set to have the same magnitude and opposite sign. In addition, MV diff_L0_y and MV diff_L1_y Can also be set to have the same magnitude and opposite sign.

[0188] As described above, by refining the motion vector in consideration of the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, the motion vector accuracy may be improved.

[0189] Figure 8 An implementation of decoder-side motion vector refinement in geometric partition mode with bidirectional prediction applied to each partition is shown when the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are not the same.

[0190] The current picture 800 includes a current block 802. In addition, the current block 802 includes a first partition 804 and a second partition 806.

[0191] exist Figure 8Among them, the current picture 800 is the picture at time t, and the L0 reference picture 820 and the L1 reference picture 850 are the pictures at time t - M (t - M > 0) and time t + N (t + N > 0) respectively (t - M < t + N). Here, M and N are different (M < N) arbitrary positive integer values. Therefore, the time distance between the L0 reference picture 820 and the current picture 800 is different from the time distance between the L1 reference picture 850 and the current picture 800.

[0192] According to one embodiment, bidirectional motion prediction can be performed on each of the first partition 804 and the second partition 806. In the embodiment of predicting each partition according to unidirectional motion prediction Figure 7 decoder-side motion vector refinement can be performed on the first partition 704 and the second partition 706 by considering the entire region of the current block 702. On the other hand, in the embodiment of predicting each partition according to bidirectional motion prediction Figure 8 decoder-side motion vector refinement can be performed by considering only the regions of each partition.

[0193] For the bidirectional motion prediction of the first partition 804, the motion vector MV0 _R0 822 in the L0 direction and the motion vector MV1 _R0 852 in the L1 direction are determined as the basic motion vectors. Additionally, for the bidirectional motion prediction of the second partition 806, the motion vector MV0 _R1 832 in the L0 direction and the motion vector MV1 _R1 862 in the L1 direction are determined as the basic motion vectors.

[0194] For the motion vector refinement of the first partition 804, the adjacent motion vectors of MV0 _R0 822 and MV1 _R0 852 are symmetrically searched. Additionally, among the multiple adjacent motion vectors, the motion vectors in the L0 and L1 directions that minimize the distortion between the prediction signal in the L0 direction and the prediction signal in the L1 direction are derived as the refined motion vectors MV0' _R0 828 and MV1' _R0 858.

[0195] By refining the motion vector MV0 _R0 822 in the L0 direction to MV diff_L0_R0 826, MV0' _R0 828 is derived. Additionally, by refining the motion vector MV1 _R0 852 in the L1 direction to MV diff_L1_R0 856 symmetrically to the refined motion vector in the L0 direction, MV1' _R0858. The ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture is considered to obtain the temporal distance between the current picture and the L0 reference picture. diff_L0_R0 826 Confirm MV diff_L1_R0 856. In this case, in determining MV diff_L1_R0 856, the implementation scheme described above in conjunction with equation 1 may be applied. Alternatively, MV diff_L1_R0 856 can be identified as in MV diff_L0_R0 826 have vectors of the same magnitude in opposite directions, regardless of the time-to-distance ratio.

[0196] At this time, the L0 reference picture 820 is composed of MV0′ _R0 Block P indicated by 828 L0_R0 830 and L1 reference picture 850 by MV1' _R0 Block P indicated by 858 L1_R0 860 Distortion Minimized MV0' _R0 828 and MV1' _R0 858 is determined as the refined motion vector. At this point, various distortion measurement methods such as the sum of absolute difference (SAD) or the sum of squared errors (SSE) may be used for the distortion.

[0197] The same method as the refinement of the motion vector of the first partition 804 described above may be applied to the refinement of the motion vector of the second partition 806 .

[0198] For the bidirectional motion prediction of the second partition 806, the motion vector MV0 in the L0 direction _R1 832 and the motion vector MV1 in the L1 direction_ R1 862 is determined as the basic motion vector.

[0199] For motion vector refinement of the second partition 806, MV0_ R1 832 and MV1_ R1 In addition, among the plurality of adjacent motion vectors, the motion vectors in the L0 and L1 directions that minimize the distortion between the prediction signal in the L0 direction and the prediction signal in the L1 direction are derived as the refined motion vector MV0′_ R1 838 and MV1'_ R1 868.

[0200] By moving the motion vector MV0 in the L0 direction R1 832 thinning to MVdiff_ L0 _ R1 836 to derive MV0'_ R1838. In addition, by converting the motion vector MV1 in the L1 direction R1 862 refinement to MVdiff_ L1 _ R1 MV1′ is derived symmetrically with the refined motion vector in the L0 direction. R1 868. The temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are considered from MVdiff_ L0 _ R1 836 Confirm MVdiff_ L1 _ R1 866. In this case, when determining MVdiff_ L1 _ R1 866, the implementation scheme described above in conjunction with equation 1 may be applied. Alternatively, MVdiff_ L1 _ R1 866 can be identified as in MVdiff_ L0 _ R1 836 have vectors of the same magnitude in opposite directions, regardless of the time-to-distance ratio.

[0201] At this time, the L0 reference picture 820 is composed of MV0'_ R1 Block P indicated by 838 L0 _ R1 840 and L1 reference picture 850 by MV1'_ R1 Block P indicated by 868 L1 _ R1 870 Distortion Minimized MV0'_ R1 838 and MV1'_ R1 The refined motion vector is determined at 868. At this point, various distortion measurement methods such as the sum of absolute differences (SAD) or the sum of squared errors (SSE) may be used for distortion.

[0202] exist Figure 8 , the reference pictures of the first partition 804 and the second partition 806 are described as being the same as the L0 reference picture 820 and the L1 reference picture 840. However, the reference pictures of the first partition 804 and the second partition 806 may be different from each other.

[0203] According to the implementation scheme, different from Figure 6 and Figure 8 , one of the two partitions of the current block can be unidirectionally predicted, and the other partition can be bidirectionally predicted. In this case, only the bidirectionally predicted partition can be performed Figure 6 and Figure 8 Decoder-side motion vector refinement presented in .

[0204] Hereinafter, an implementation for determining a weight value required for a final prediction block will be discussed.

[0205] exist Figure 5 , the current picture 500 is a picture at time t, and the L0 reference picture 520 and the L1 reference picture 540 are pictures at time tM and time t+N, respectively. The distance between the L0 reference picture 520 and the current picture 500 and the distance between the L1 reference picture 540 and the current picture 500 are equal to N. Therefore, when the final prediction block is generated using the motion vector refined in the geometric partitioning mode, Equation 2 may be applied.

[0206] [Equation 2]

[0207] Pred Final =(P L0 +P L1 +1)>>1

[0208] In Equation 2, P L0 , P L1 and Pred Final They respectively represent the first prediction block 530 indicated by MV0'528 in the L0 reference picture 520, the second prediction block 550 indicated by MV1'548 in the L1 reference picture 540, and the final prediction block generated using the two prediction blocks. Figure 5 As shown, since the distances between the two reference pictures and the current picture are the same, the same weight value can be assigned to the prediction block P generated from each reference picture. L0 and P L1 , to generate the final prediction block.

[0209] Bidirectional prediction may be applied to each of the first partition 604 and the second partition 606, such as Figure 6 At this time, when determining the first prediction block according to the prediction blocks 630 and 660 of the first partition 604 and the second prediction block according to the prediction blocks 640 and 670 of the second partition 606, the method according to Equation 2 may be used.

[0210] exist Figure 7, the current image 700 is an image at time t, and the L0 reference image 720 and the L1 reference image 740 are images at time tM and time t+N, respectively, and the distance between the L0 reference image 720 and the current image 700 and the distance between the L1 reference image 740 and the current image 700 are not equal to each other, which are M and N, respectively. Although the distance between the L0 reference image 720 and the current image 700 and the distance between the L1 reference image 740 and the current image 700 are not equal to each other, a final prediction block may be generated from the prediction blocks 730 and 750 of the current block 702 according to Equation 2 without considering the distance information. Alternatively, by considering the difference between the distance between the L0 reference image 720 and the current image 700 and the distance between the L1 reference image 740 and the current image 700, different weighting values ​​may be applied to the prediction blocks 730 and 750 of the current block 702.

[0211] According to one embodiment, when a bi-directionally predicted geometric partitioning mode is applied to the current block, a decoder-side motion vector refinement method may be used regardless of the bi-directional codec unit weighting values ​​of the codec unit block. Thus, in the case of a bi-directionally predicted geometric partitioning mode, different weighting values ​​may be used when synthesizing the prediction block.

[0212] According to one embodiment, a bidirectional prediction (BCW) method with CU-level weights using codec unit weights may be applied to generate a final prediction block by adaptively assigning adaptive weights to two prediction blocks in a geometric partitioning mode of bidirectional prediction. A prediction block may be generated by performing bidirectional prediction using various codec unit weights. According to an embodiment, five weighting values ​​(w∈{-2,3,4,5,10}) used in low-latency images may be utilized. Alternatively, three weighting values ​​(w∈{3,4,5}) used in non-low-latency images may be utilized. Alternatively, a predetermined arbitrary weighting value may be utilized. In this case, the number of weighting values ​​utilized may be arbitrarily determined.

[0213] In BCW, since information about the determined weighting value is transmitted / parsed, coding efficiency may be reduced. Therefore, when the geometric partition mode of bidirectional prediction is applied to the current block and the distance between the L0 reference picture and the current picture and the distance between the L1 reference picture and the current picture are not the same, the weighting value may be derived by considering the distance ratio between each reference picture and the current picture. Equation 3 shows that Figure 7 A method for determining a weighted value for a case where the distance between the L0 reference picture and the current picture is different from the distance between the L1 reference picture and the current picture.

[0214] [Equation 3]

[0215] PredFinal =W L0 ×P L0 +W L1 ×P L1

[0216]

[0217] In Equation 3, M and N represent the distance between the current picture and the L0 reference picture and the distance between the current picture and the L1 reference picture, respectively. L0 and W L1 They represent the prediction signal P in the L0 direction respectively. L0 The weighted value of and the prediction signal P in the L1 direction L1 In addition, Pred Final Represents the final prediction block generated by applying weighted values ​​to two prediction blocks.

[0218] Hereinafter, a method of generating a final prediction block by generating a prediction block of a partition area shape of a current block in each reference picture by considering the characteristics of a geometric partition mode of bidirectional prediction, and then combining them when generating a final prediction block of a geometric partition mode of bidirectional prediction will be described.

[0219] Fig. 9 A method of generating a final prediction block by considering the characteristics of a geometric partitioning mode of bidirectional prediction is shown.

[0220] The decoder side motion vector refinement method can be applied to the current block 902 together with the geometric partitioning mode of bidirectional prediction. Fig. 9 As shown, the motion vector MV0'928 in the L0 direction is used to generate a prediction block 930 corresponding to the first partition 904 of the current block 902. In addition, the motion vector MV1'948 in the L1 direction is used to generate a prediction block 950 of the second partition 906 of the current block, and as shown in box 960, the prediction block 930 of the first partition 904 and the prediction block 950 of the second partition 906 can be used to generate a final prediction block of the current block 902.

[0221] Fig. 9 A method for generating a final prediction block of a geometric partitioning mode for bidirectional prediction is shown when the distances between two reference pictures and the current picture are not the same. However, as an implementation, this can also be applied to the case where the distances between two reference pictures and the current picture are the same.

[0222] Fig.10 is a flowchart of an implementation of a decoder-side motion vector refinement method for a geometric partitioning mode according to the present invention.

[0223] In step 1002, the current block is partitioned into a first partition and a second partition according to a partition boundary.

[0224] At step 1004, a first basic motion vector and a second basic motion vector corresponding to the first partition and the second partition are determined.

[0225] At step 1006, a first refined motion vector and a second refined motion vector are determined by refining the first base motion vector and the second base motion vector.

[0226] According to one embodiment, in step 1006, the first refined motion vector and the second refined motion vector may be determined according to the distance between the current picture and the first reference picture referenced by the first partition and the distance between the current picture and the second reference picture referenced by the second partition.

[0227] According to one embodiment, the magnitude of the first differential motion vector representing the difference between the first basic motion vector and the first refined motion vector and the magnitude of the second differential motion vector representing the difference between the second basic motion vector and the second refined motion vector may be proportional to the distance between the current picture and the first reference picture and the distance between the current picture and the second reference picture. In addition, the first differential motion vector and the second differential motion vector may be in opposite directions.

[0228] According to one embodiment, the sizes of the first differential motion vector and the second differential motion vector may be limited to a predetermined range. Here, the predetermined range may be defined in pixels. For example, the predetermined range may be 1 or 2 pixel units.

[0229] According to one embodiment, the first refined motion vector and the second refined motion vector may be determined such that distortion between a first prediction block indicated by the first refined motion vector and a second prediction block indicated by the second refined motion vector is minimized.

[0230] At step 1008, a first prediction block for the first partition and a second prediction block for the second partition are determined based on the first refined motion vector and the second refined motion vector.

[0231] In step 1010, a final prediction block is determined based on the first prediction block and the second prediction block.

[0232] According to one embodiment, in step 1010, a final prediction block may be determined based on a weighted sum of the first prediction block and the second prediction block. Specifically, the weighted sum of the first prediction block and the second prediction block may be determined by a first weighted value and a second weighted value, the first weighted value being determined based on a distance between the current picture and a first reference picture referenced by the first partition, and the second weighted value being determined based on a distance between the current picture and a second reference picture referenced by the second partition. Here, the first weighted value may be proportional to the distance between the current picture and the second reference picture, and the second weighted value may be proportional to the distance between the current picture and the first reference picture.

[0233] According to one embodiment, bidirectional prediction may be performed on each partition of the geometric partition mode. At this time, in step 1004, when bidirectional prediction is performed on the first partition, a first L0 basic motion vector and a first L1 basic motion vector corresponding to the first partition may be determined. In addition, when bidirectional prediction is performed on the second partition, a second L0 basic motion vector and a second L1 basic motion vector corresponding to the second partition may be determined.

[0234] In step 1006, when bidirectional prediction is performed on the first partition, a first L0 refined motion vector and a first L1 refined motion vector may be determined by refining the first L0 basic motion vector and the first L1 basic motion vector. In addition, the first L0 refined motion vector and the first L1 refined motion vector may be determined according to a distance between the current picture and the first L0 reference picture referenced by the first partition and a distance between the current picture and the first L1 reference picture referenced by the first partition. Here, a size of the first L0 differential motion vector indicating a difference between the first L0 basic motion vector and the first L0 refined motion vector and a size of the first L1 differential motion vector indicating a difference between the first L1 basic motion vector and the first L1 refined motion vector may be proportional to a distance between the current picture and the first L0 reference picture and a distance between the current picture and the first L1 reference picture.

[0235] In addition, when bidirectional prediction is performed on the second partition, the second L0 refined motion vector and the second L1 refined motion vector may be determined by refining the second L0 basic motion vector and the second L1 basic motion vector. In addition, the second L0 refined motion vector and the second L1 refined motion vector may be determined according to the distance between the current picture and the second L0 reference picture referenced by the second partition and the distance between the current picture and the second L1 reference picture referenced by the second partition. Here, the size of the second L0 differential motion vector indicating the difference between the second L0 basic motion vector and the second L0 refined motion vector and the size of the second L1 differential motion vector indicating the difference between the second L1 basic motion vector and the second L1 refined motion vector may be proportional to the distance between the current picture and the second L0 reference picture and the distance between the current picture and the second L1 reference picture.

[0236] According to one embodiment, the size of the first L0 differential motion vector, the first L1 differential motion vector, the second L0 differential motion vector, and the second L1 differential motion vector may be limited to a predetermined range. Here, the predetermined range may be defined in pixels. For example, the predetermined range may be 1 or 2 pixel units.

[0237] In step 1008, when bidirectional prediction is performed on the first partition, a first prediction block may be determined based on the first L0 refined motion vector and the first L1 refined motion vector. In addition, when bidirectional prediction is performed on the second partition, a second prediction block may be determined based on the second L0 refined motion vector and the second L1 refined motion vector.

[0238] According to one embodiment, at step 1008, the first L0 prediction block and the first L1 prediction block may be determined based on the first L0 refined motion vector and the first L1 refined motion vector. In addition, the first prediction block may be determined based on a weighted sum of the first L0 prediction block and the first L1 prediction block. When determining the first L0 refined motion vector and the first L1 refined motion vector, the first L0 refined motion vector and the first L1 refined motion vector may be determined so that distortion between the first L0 prediction block and the first L1 prediction block is minimized.

[0239] In addition, the second L0 prediction block and the second L1 prediction block may be determined according to the second L0 refined motion vector and the second L1 refined motion vector. In addition, the second prediction block may be determined according to a weighted sum of the second L0 prediction block and the second L1 prediction block. When determining the second L0 refined motion vector and the second L1 refined motion vector, the second L0 refined motion vector and the second L1 refined motion vector may be determined so that distortion between the second L0 prediction block and the second L1 prediction block is minimized.

[0240] Fig.10The decoder-side motion vector refinement method for the geometric partition mode described in the above can be applied to the image decoding method and apparatus. Therefore, the motion vector can be refined in the geometric partition mode without separate explicit information.

[0241] Fig.10 The decoder-side motion vector refinement of the geometric partitioning mode described in the above can also be applied to the image encoding method. Therefore, in the image encoding method and the image decoding method, since the same decoder-side motion vector refinement method of the geometric partitioning mode is applied, the bitstream generated in the image encoding method can be decoded according to the image decoding method.

[0242] When applied Fig.10 When an image encoding device with decoder-side motion vector refinement of the geometric partitioning mode described in the above embodiment encodes an image, a bit stream may be generated. In addition, the generated bit stream may be stored in a recording medium or sent to an image decoding device. When the bit stream generated by the image decoding device is decoded, the encoded image may be reconstructed.

[0243] Fig.11 A content streaming system to which an embodiment according to the present invention can be applied is exemplarily shown.

[0244] like Fig.11 As shown, the content streaming system to which the embodiment of the present invention is applied may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device and a multimedia input device.

[0245] The encoding server compresses the content received from the multimedia input device such as a smartphone, a camera, a CCTV, etc. into digital data to generate a bit stream and transmits it to the streaming server. As another example, if the multimedia input device such as a smartphone, a camera, a CCTV, etc. directly generates a bit stream, the encoding server can be omitted.

[0246] A bitstream may be generated by the image encoding method and / or the image encoding device to which the embodiments of the present invention are applied, and a streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0247] The streaming server sends multimedia data to the user device based on a user request via the network server, and the network server can act as an intermediary to notify the user of any available services. When the user requests a desired service from the network server, the network server sends it to the streaming server, and the streaming server can send the multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can control the command / response between the devices within the content streaming system.

[0248] The streaming server can receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a stable streaming service, the streaming server can store the bitstream for a period of time.

[0249] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, HMDs), digital televisions, desktop computers, digital signage, etc.

[0250] Each server in the above-described content streaming system may operate as a distributed server, in which case data received from each server may be distributed and processed.

[0251] The above-mentioned embodiments may be performed in the same or corresponding manner in the encoding device and the decoding device. In addition, an image may be encoded / decoded using at least one or a combination of at least one of the above-mentioned embodiments.

[0252] The order of applying the above-described embodiments may be different in the encoding device and the decoding device. Alternatively, the order of applying the above-described embodiments may be the same in the encoding device and the decoding device.

[0253] The above-described embodiments may be performed on each of the luminance signal and the chrominance signal. Alternatively, the above-described embodiments of the luminance signal and the chrominance signal may be performed identically.

[0254] In the above-mentioned embodiment, the method is described based on a flow chart with a series of steps or units, but the present invention is not limited to the order of the steps, on the contrary, some steps can be performed simultaneously with other steps or in different orders. In addition, it should be understood by those of ordinary skill in the art that the steps in the flow chart are not mutually exclusive, and other steps can be added to the flow chart or some steps can be deleted from the flow chart without affecting the scope of the present invention.

[0255] The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include independent program instructions, data files, data structures, etc., or a combination thereof. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, or are well known to those of ordinary skill in the field of computer software technology.

[0256] The bit stream generated by the encoding method according to the above-mentioned embodiment can be stored in a non-volatile computer-readable recording medium. In addition, the bit stream stored in the non-volatile computer-readable recording medium can be decoded by the decoding method according to the above-mentioned embodiment.

[0257] Examples of computer-readable recording media include: magnetic recording media such as hard disks, floppy disks, and magnetic tapes; optical data storage media such as CD-ROMs or DVD-ROMs; magneto-optical media such as floppy disks; and hardware devices such as read-only memory (ROM), random access memory (RAM), flash memory, etc., which are specially configured to store and execute program instructions. Examples of program instructions include not only machine language codes formatted by a compiler, but also high-level language codes that can be implemented by a computer using an interpreter. The hardware device can be configured to be operated by one or more software modules or vice versa to perform the process according to the present invention.

[0258] Although the present invention has been described in terms of specific items such as detailed elements and limited embodiments and drawings, they are provided only to help a more comprehensive understanding of the present invention, and the present invention is not limited to the above embodiments. It will be appreciated by those skilled in the art that various modifications and changes can be made based on the above description.

[0259] Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the full scope of the appended claims and their equivalents will fall within the scope and spirit of the present invention.

[0260] Industrial Applicability

[0261] The present invention can be used in an apparatus for encoding / decoding an image and a recording medium for storing a bit stream.

Claims

1. An image decoding method, comprising: Partition the current block into a first partition and a second partition according to a partition boundary; determining a first base motion vector and a second base motion vector corresponding to the first partition and the second partition; determining a first refined motion vector and a second refined motion vector by refining the first base motion vector and the second base motion vector; determining a first prediction block for a first partition and a second prediction block for a second partition based on the first refined motion vector and the second refined motion vector; as well as A final prediction block is determined based on the first prediction block and the second prediction block.

2. The image decoding method according to claim 1, wherein: In determining the first refined motion vector and the second refined motion vector, The first refined motion vector and the second refined motion vector are determined according to a distance between a current picture including the current block and a first reference picture referenced by the first partition and a distance between the current picture and a second reference picture referenced by the second partition.

3. The image decoding method according to claim 2, wherein: A size of a first differential motion vector indicating a difference between the first basic motion vector and the first refined motion vector and a size of a second differential motion vector indicating a difference between the second basic motion vector and the second refined motion vector are proportional to a distance between the current picture and the first reference picture and a distance between the current picture and the second reference picture.

4. The image decoding method according to claim 3, wherein: The magnitudes of the first differential motion vector and the second differential motion vector are limited within a predetermined range.

5. The image decoding method according to claim 1, wherein: In determining the first refined motion vector and the second refined motion vector, The first refined motion vector and the second refined motion vector are determined such that distortion between a first prediction block indicated by the first refined motion vector and a second prediction block indicated by the second refined motion vector is minimized.

6. The image decoding method according to claim 1, wherein: When determining the final prediction block, the final prediction block is determined according to a weighted sum of the first prediction block and the second prediction block.

7. The image decoding method according to claim 6, wherein: The weighted sum of the first prediction block and the second prediction block is determined by a first weight value and a second weight value, wherein the first weight value is determined based on a distance between a current picture including the current block and a first reference picture referenced by the first partition, and the second weight value is determined based on a distance between the current picture and a second reference picture referenced by the second partition.

8. The image decoding method according to claim 7, wherein: A first weighting value applied to the first prediction block is proportional to a distance between the current picture and the second reference picture, and a second weighting value applied to the second prediction block is proportional to a distance between the current picture and the first reference picture.

9. The image decoding method according to claim 1, wherein: In determining the first base motion vector and the second base motion vector, a first L0 base motion vector and a first L1 base motion vector corresponding to the first partition are determined, and a second L0 base motion vector and a second L1 base motion vector corresponding to the second partition are determined, In determining the first refined motion vector and the second refined motion vector, the first L0 refined motion vector and the first L1 refined motion vector are determined by refining the first L0 base motion vector and the first L1 base motion vector, and the second L0 refined motion vector and the second L1 refined motion vector are determined by refining the second L0 base motion vector and the second L1 base motion vector, and In determining the first prediction block and the second prediction block, the first prediction block is determined according to the first L0 refined motion vector and the first L1 refined motion vector, and the second prediction block is determined according to the second L0 refined motion vector and the second L1 refined motion vector.

10. The image decoding method according to claim 9, wherein: In determining the first L0-refined motion vector, the first L1-refined motion vector, the second L0-refined motion vector, and the second L1-refined motion vector, determining a first L0 refined motion vector and a first L1 refined motion vector according to a distance between a current picture including the current block and a first L0 reference picture referenced by the first partition and a distance between the current picture and a first L1 reference picture referenced by the first partition, and The second L0 refined motion vector and the second L1 refined motion vector are determined according to a distance between the current picture and a second L0 reference picture referenced by the second partition and a distance between the current picture and a second L1 reference picture referenced by the second partition.

11. The image decoding method according to claim 10, wherein: a magnitude of a first L0 differential motion vector representing a difference between the first L0 base motion vector and the first L0 refined motion vector, and a magnitude of a first L1 differential motion vector representing a difference between the first L1 base motion vector and the first L1 refined motion vector, are proportional to a distance between the current picture and the first L0 reference picture, and a distance between the current picture and the first L1 reference picture, and The size of the second L0 differential motion vector representing the difference between the second L0 base motion vector and the second L0 refined motion vector and the size of the second L1 differential motion vector representing the difference between the second L1 base motion vector and the second L1 refined motion vector are proportional to the distance between the current picture and the second L0 reference picture and the distance between the current picture and the second L1 reference picture.

12. The image decoding method according to claim 11, wherein: The sizes of the first L0 differential motion vector, the first L1 differential motion vector, the second L0 differential motion vector, and the second L1 differential motion vector are limited within a predetermined range.

13. The image decoding method according to claim 11, wherein: Determining the first prediction block and the second prediction block includes: determining a first L0 prediction block and a first L1 prediction block according to the first L0 refined motion vector and the first L1 refined motion vector, and determining a second L0 prediction block and a second L1 prediction block according to the second L0 refined motion vector and the second L1 refined motion vector; and The first prediction block is determined according to a weighted sum of the first L0 prediction block and the first L1 prediction block, and the second prediction block is determined according to a weighted sum of the second L0 prediction block and the second L1 prediction block.

14. The image decoding method according to claim 13, in, When determining the motion vector for the first L0 refinement and the motion vector for the first L1 refinement, determining a first L0 refined motion vector and a first L1 refined motion vector such that distortion between a first L0 prediction block indicated by the first L0 refined motion vector and a first L1 prediction block indicated by the first L1 refined motion vector is minimized, and When determining the second L0 refined motion vector and the second L1 refined motion vector, The second L0 refined motion vector and the second L1 refined motion vector are determined such that distortion between the second L0 prediction block indicated by the second L0 refined motion vector and the second L1 prediction block indicated by the second L1 refined motion vector is minimized.

15. An image encoding method, comprising: Partition the current block into a first partition and a second partition according to a partition boundary; determining a first base motion vector and a second base motion vector corresponding to the first partition and the second partition; determining a first refined motion vector and a second refined motion vector by refining the first base motion vector and the second base motion vector; determining a first prediction block for a first partition and a second prediction block for a second partition based on the first refined motion vector and the second refined motion vector; as well as A final prediction block is determined based on the first prediction block and the second prediction block.

16. A computer-readable recording medium storing a bit stream generated by an image encoding method, the image encoding method comprising: Partition the current block into a first partition and a second partition according to a partition boundary; determining a first base motion vector and a second base motion vector corresponding to the first partition and the second partition; determining a first refined motion vector and a second refined motion vector by refining the first base motion vector and the second base motion vector; determining a first prediction block for a first partition and a second prediction block for a second partition based on the first refined motion vector and the second refined motion vector; as well as A final prediction block is determined based on the first prediction block and the second prediction block.

17. A method for transmitting a bit stream generated by an image encoding method, the method comprising: Encoding an image based on an image coding method; as well as Transmits a bitstream consisting of an encoded image, Wherein, the image encoding method comprises: Partition the current block into a first partition and a second partition according to a partition boundary; determining a first base motion vector and a second base motion vector corresponding to the first partition and the second partition; determining a first refined motion vector and a second refined motion vector by refining the first base motion vector and the second base motion vector; determining a first prediction block for the first partition and a second prediction block for the second partition based on the first refined motion vector and the second refined motion vector; and A final prediction block is determined based on the first prediction block and the second prediction block.

Images