Image encoding / decoding method and apparatus, and recording medium storing bit stream
By using the decoder-side motion vector refinement (DMVR) method, which refines motion vectors and employs different weights and template matching, the problem of high data transmission and storage costs for high-resolution images is solved, and a more efficient encoding/decoding process is achieved.
Patent Information
- Application Number
- CN202380026795.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-11
- Filing Date
- 2023-03-06
- Publication Date
- 2024-11-12
AI Technical Summary
With the increasing demand for high-resolution, high-quality images, the cost of existing image data transmission and storage is rising, necessitating highly efficient image encoding/decoding technologies to address this issue.
The motion vectors are refined using the decoder-side motion vector refinement (DMVR) method to improve the prediction accuracy of the current block. Different weights and template matching methods are used to optimize the motion vector refinement process.
It improves encoding/decoding efficiency, reduces data volume, and lowers transmission and storage costs.
Smart Images

Figure CN118947122A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and apparatus for image encoding / decoding and a recording medium for storing a bitstream. More specifically, the present invention relates to a method and apparatus for image encoding / decoding using a decoder-side motion vector refinement (DMVR) method and a recording medium for storing a bitstream. Background Art
[0002] Recently, in various application fields, the demand for high-resolution, high-quality images (such as ultra-high-definition (UHD) images) has been increasing. As the resolution and quality of image data become higher, the amount of data increases relatively compared to existing image data. Therefore, when image data is transmitted using a medium such as existing wired and wireless broadband lines or stored using existing storage media, the transmission and storage costs increase. In order to solve these problems that arise as the resolution and quality of image data become higher, efficient image encoding / decoding technology with images of higher resolution and quality is required.
[0003] In the merge mode in the inter prediction mode, motion information such as motion vectors and reference picture information from neighboring blocks of the current block can be used to predict the current block. If bidirectional inter prediction with reference to two reference pictures is applied in the merge mode, decoder-side motion vector refinement (DMVR) can be applied. Through the decoder-side motion vector refinement, the motion vector obtained from the neighboring blocks is refined even in the merge mode, thereby improving the prediction accuracy of the current block. Summary of the invention
[0004] Technical issues
[0005] An object of the present invention is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.
[0006] Another object of the present invention is to provide a recording medium for storing a bit stream generated by an image decoding method or apparatus according to an embodiment of the present invention.
[0007] Technical Solution
[0008] A video decoding method according to an embodiment of the present invention may include determining a first basic motion vector of a current block of a first reference picture and a second basic motion vector of a current block of a second reference picture, determining a first refined motion vector by refining the first basic motion vector with a first motion vector difference, and determining a second refined motion vector by refining the second basic motion vector with a second motion vector difference, determining a first prediction block and a second prediction block of the current block based on the first refined motion vector and the second refined motion vector, and determining a final prediction block of the current block based on a weighted sum of the first prediction block and the second prediction block.
[0009] According to an embodiment, determining the first refined motion vector and the second refined motion vector comprises: determining the first refined motion vector and the second refined motion vector when a distance between a current picture including the current block and the first reference picture and a distance between the current picture and the second reference picture are different.
[0010] According to an embodiment, the weighted sum of the first prediction block and the second prediction block may be determined by a weight determined according to a distance between the current picture and the first reference picture and a distance between the current picture and the second reference picture.
[0011] According to an embodiment, a first weight applied to the first prediction block may be proportional to the distance between the current picture and the second reference picture, and a second weight applied to the second prediction block may be proportional to the distance between the current picture and the first reference picture.
[0012] According to an embodiment, when determining the first refined motion vector and the second refined motion vector, the ratio of the magnitude of the first motion vector difference to the magnitude of the second motion vector difference may be proportional to the ratio of the distance between the current picture and the first reference picture to the distance between the current picture and the second reference picture.
[0013] According to an embodiment, the magnitudes of the first motion vector difference and the second motion vector difference may be limited within a predetermined range.
[0014] According to an embodiment, determining the first refined motion vector and the second refined motion vector may be determined such that distortion between a first prediction block indicated by the first refined motion vector and a second prediction block indicated by the second refined motion vector is minimized.
[0015] According to an embodiment, the first refined motion vector and the second refined motion vector may be determined so that distortion between a first template of a first prediction block indicated by the first refined motion vector, a second template of a second prediction block indicated by the second refined motion vector, and a current template of a current block is minimized.
[0016] According to an embodiment, the distortion may be calculated based on a final template determined by a weighted average of the first template, the second template, and the current template.
[0017] According to an embodiment, in the weighted average of the first template and the second template, the first weight applied to the first template may be proportional to the distance between the current picture and the second reference picture, and the second weight applied to the second template may be proportional to the distance between the current picture and the first reference picture.
[0018] According to an embodiment, a first weight applied to the first template may be proportional to distortions of the second template and the current template, and a second weight applied to the second template may be proportional to distortions of the first template and the current template.
[0019] According to an embodiment, the weighted sum of the first prediction block and the second prediction block may be determined by a weight determined based on a picture type of a current picture including the current block.
[0020] According to an embodiment, the weighted sum of the first prediction block and the second prediction block may be determined by a weight determined based on weight information generated by parsing a bitstream.
[0021] The video encoding method according to an embodiment of the present invention may include determining a first basic motion vector of a current block of a first reference picture and a second basic motion vector of a current block of a second reference picture, determining a first refined motion vector by refining the first basic motion vector with a first motion vector difference, and determining a second refined motion vector by refining the second basic motion vector with a second motion vector difference, determining a first prediction block and a second prediction block of the current block based on the first refined motion vector and the second refined motion vector, and determining a final prediction block of the current block based on a weighted sum of the first prediction block and the second prediction block.
[0022] The non-transitory computer-readable recording medium according to an embodiment of the present invention may store a bitstream generated by a video encoding method.
[0023] The transmission method according to an embodiment of the present invention transmits a bit stream generated by a video encoding method.
[0024] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.
[0025] Beneficial Effects
[0026] Embodiments of the present disclosure provide a method for improving decoder-side motion vector refinement (DMVR).
[0027] According to an embodiment of the present disclosure, the application frequency of the decoder-side motion vector refinement method may be increased by changing the application conditions of the decoder-side motion vector refinement method.
[0028] In addition, in the decoder-side motion vector refinement method, a method of applying different weights to bidirectional motion vectors instead of applying the same weight is disclosed.
[0029] Furthermore, in the decoder-side motion vector refinement method, a method of refining a motion vector based on a template matching method instead of bilateral matching is disclosed.
[0030] By using the disclosed method, the accuracy of the decoder-side motion vector refinement method can be improved, and the accuracy of the final prediction block can be increased, thereby improving the encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a block diagram showing the configuration of an encoding device according to an embodiment of the present invention.
[0032] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment of the present invention.
[0033] Figure 3 FIG. 2 is a schematic diagram schematically showing a video decoding system to which the present invention can be applied.
[0034] Figure 4 is a diagram illustrating an embodiment of a method of determining one of various inter prediction methods in an inter prediction mode.
[0035] Figure 5 is a diagram illustrating an implementation of decoder-side motion vector refinement when the temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture.
[0036] Figure 6 is a diagram illustrating decoder-side motion vector refinement when the temporal distance between the current picture and the L0 reference picture is not the same as the temporal distance between the current picture and the L1 reference picture.
[0037] Figure 7 Decoder-side motion vector refinement based on a template matching approach is shown.
[0038] Figure 8 is a flowchart of a decoder-side motion vector refinement method according to an embodiment.
[0039] Fig. 9 The content streaming system to which the embodiments of the present invention can be applied is exemplarily shown.
[0040] Best Mode
[0041] A video decoding method according to an embodiment of the present invention may include determining a first basic motion vector of a current block of a first reference picture and a second basic motion vector of a current block of a second reference picture, determining a first refined motion vector by refining the first basic motion vector with a first motion vector difference, and determining a second refined motion vector by refining the second basic motion vector with a second motion vector difference, determining a first prediction block and a second prediction block of the current block based on the first refined motion vector and the second refined motion vector, and determining a final prediction block of the current block based on a weighted sum of the first prediction block and the second prediction block. DETAILED DESCRIPTION
[0042] The present invention may have various modifications and embodiments, and specific embodiments are shown in the drawings and described in detail in the specific embodiments. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents or substitutes included in the spirit and scope of the present invention. In various aspects, similar reference numerals in the drawings indicate the same or similar functions. The shapes and sizes of the elements in the drawings can be provided by way of example for a clearer description. The detailed description of the exemplary embodiments described below refers to the drawings, which show specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice these embodiments. It should be understood that various embodiments are different from each other, but not necessarily mutually exclusive. For example, the specific shapes, structures and characteristics described herein can be implemented in other embodiments without departing from the spirit and scope of the present invention with respect to an embodiment. It should also be understood that the position or arrangement of each component in each disclosed embodiment can be changed without departing from the spirit and scope of the embodiment. Therefore, the specific embodiments set forth below are not intended to be restrictive, and the scope of the exemplary embodiments is limited only by the full range of equivalents to which the attached claims and these claims are entitled (if properly described).
[0043] In the present invention, the terms first, second, etc. may be used to describe various components, but these components should not be limited by these terms. These terms are only used for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The terms and / or include a combination of multiple related descriptive items or any item among multiple related descriptive items.
[0044] Each component in the embodiment of the present invention is described independently to indicate different characteristic functions, and does not mean that each component is formed as a separate hardware or software configuration unit. That is, for ease of explanation, each component is listed and included as a separate component, and at least two components can be combined to form a single component, or a component can be divided into multiple components to perform functions, and an embodiment in which the components are integrated and an embodiment in which each component is divided are also included in the scope of the present invention, as long as they do not deviate from the essence of the present invention.
[0045] The terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention. Unless the context clearly indicates otherwise, singular expressions include plural expressions. In addition, some components of the present invention are not necessary components for performing the basic functions of the present invention, and may be optional components that are only used to improve performance. The present invention can be implemented by only including necessary components for realizing the essence of the present invention and not including components that are only used to improve performance, and the structure that only includes necessary components except optional components that are only used to improve performance is also included in the scope of the present invention.
[0046] In an embodiment, the term "at least one" may mean one of a number greater than or equal to 1, such as 1, 2, 3, and 4. In an embodiment, the term "plurality" may mean one of a number greater than or equal to 2, such as 2, 3, and 4.
[0047] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the subject of the present specification, the detailed description will be omitted, and the same reference numerals will be used for the same components in the accompanying drawings, and repeated descriptions of the same components will be omitted.
[0048] Terminology Description
[0049] Hereinafter, "image" may refer to one picture constituting a video, and may also refer to the video itself. For example, "encoding and / or decoding of an image" may refer to "encoding and / or decoding of a video", and may also refer to "encoding and / or decoding of one of the images constituting a video".
[0050] Hereinafter, "motion picture" and "video" may be used with the same meaning and may be used interchangeably. In addition, the target image may be an encoding target image as an encoding target and / or a decoding target image as a decoding target. In addition, the target image may be an input image input to an encoding device, and may be an input image input to a decoding device. Here, the target image may have the same meaning as the current image.
[0051] Hereinafter, “image”, “picture”, “frame” and “screen” may be used with the same meaning and may be used interchangeably.
[0052] Hereinafter, a "target block" may be an encoding target block as an encoding target and / or a decoding target block as a decoding target. In addition, a target block may be a current block as a target of current encoding and / or decoding. For example, "target block" and "current block" may be used together with the same meaning and may be used interchangeably.
[0053] Hereinafter, "block" and "unit" may be used with the same meaning and may be used interchangeably. In addition, "unit" may mean a block including a luma component block and a chroma component block corresponding thereto so as to distinguish it from a block. For example, a coding tree unit (CTU) may be composed of one luma component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated therewith.
[0054] Hereinafter, "sample", "picture element" and "pixel" may be used together with the same meaning and may be used interchangeably. Herein, a sample may mean a basic unit constituting a block.
[0055] Hereinafter, “inter-frame” and “inter-screen” may be used with the same meaning and may be used interchangeably.
[0056] Hereinafter, “intra-frame” and “intra-screen” may be used with the same meaning and may be used interchangeably.
[0057] Figure 1 is a block diagram showing the configuration of an encoding device according to an embodiment of the present invention.
[0058] The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device 100 may sequentially encode one or more images.
[0059] refer to Figure 1 , the encoding device 100 may include an image division unit 110, an intra-frame prediction unit 120, a motion prediction unit 121, a motion compensation unit 122, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, a dequantization unit 160, an inverse transform unit 170, an adder 117, a filter unit 180 and a reference picture buffer 190.
[0060] In addition, the encoding apparatus 100 may generate a bitstream including information encoded by encoding an input image, and output the generated bitstream. The generated bitstream may be stored in a computer-readable recording medium, or may be streamed through a wired / wireless transmission medium.
[0061] The image division unit 110 can divide the input image into various forms to improve the efficiency of video encoding / decoding. That is, the input video is composed of multiple pictures, and one picture can be hierarchically divided and processed for compression efficiency, parallel processing, etc. For example, one picture can be divided into one or more tiles or slices, and then divided into multiple CTUs (coding tree units) again. Alternatively, one picture can be first divided into multiple sub-pictures defined as a group of rectangular slices, and each sub-picture can be divided into tiles / slices. Here, sub-pictures can be used to support the functions of partially independent encoding / decoding and transmitting images. Since multiple sub-pictures can be reconstructed separately, it has the advantage of being easy to edit in applications that configure multi-channel inputs as one picture. In addition, tiles can be divided horizontally to generate blocks. Here, blocks can be used as basic units for parallel processing within a picture. In addition, one CTU can be recursively divided into a quadtree (QT), and the terminal node of the division can be defined as a CU (coding unit). CU can be divided into PU (prediction unit) as a prediction unit and TU (transform unit) as a transform unit to perform prediction and division. At the same time, the CU can be used as a prediction unit and / or a transform unit itself. Here, for flexible partitioning, each CTU can be recursively divided into a multi-type tree (MTT) and a quadtree (QT). The division of the CTU into a multi-type tree can start from the terminal node of the QT, and the MTT can be composed of a binary tree (BT) and a ternary tree (TT). For example, the MTT structure can be classified into a vertical binary split mode (SPLIT_BT_VER), a horizontal binary split mode (SPLIT_BT_HOR), a vertical ternary split mode (SPLIT_TT_VER), and a horizontal ternary split mode (SPLIT_TT_HOR). In addition, the minimum block size (MinQTSize) of the quadtree of the luminance block during partitioning can be set to 16×16, the maximum block size (MaxBtSize) of the binary tree can be set to 128×128, and the maximum block size (MaxTtSize) of the ternary tree can be set to 64×64. In addition, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the ternary tree may be specified as 4×4, and the maximum depth (MaxMttDepth) of the multi-type tree may be specified as 4. In addition, in order to improve the coding efficiency of the I slice, a dual tree that uses the CTU partition structure of the luminance component and the chrominance component differently may be applied. On the other hand, in P and B slices, the luminance and chrominance CTBs (coding tree blocks) within the CTU may be divided into a single tree that shares the coding tree structure.
[0062] The encoding device 100 may perform encoding on the input image in intra mode and / or inter mode. Optionally, the encoding device 100 may perform encoding on the input image in a third mode other than intra mode and inter mode (e.g., IBC mode, palette mode, etc.). However, if the third mode has functional characteristics similar to those of the intra mode or the inter mode, the third mode may be classified as the intra mode or the inter mode for ease of explanation. In the present invention, the third mode is separately classified and described only when its specific description is required.
[0063] When the intra mode is used as the prediction mode, the switch 115 may be switched to intra, and when the inter mode is used as the prediction mode, the switch 115 may be switched to inter. Here, the intra mode may represent the intra prediction mode, and the inter mode may represent the inter prediction mode. The encoding device 100 may generate a prediction block for an input block of an input image. In addition, the encoding device 100 may encode the residual block using the residual of the input block and the prediction block after generating the prediction block. The input image may be referred to as the current image of the current encoding target. The input block may be referred to as the current block or the encoding target block as the current encoding target.
[0064] When the prediction mode is intra mode, the intra prediction unit 120 may use samples of blocks that have been encoded / decoded around the current block as reference samples. The intra prediction unit 120 may generate prediction samples of the input block by performing spatial prediction on the current block using the reference samples, or by spatial prediction. Here, intra prediction may refer to intra-picture prediction.
[0065] As the intra prediction method, a non-directional prediction mode and a directional prediction mode (eg, 65 directions) such as a DC mode and a planar mode may be applied. Here, the intra prediction method may be expressed as an intra prediction mode or an intra-screen prediction mode.
[0066] When the prediction mode is the inter mode, the motion prediction unit 111 may retrieve the area that best matches the input block from the reference image in the motion prediction process, and derive the motion vector by using the retrieved area. In this case, the search area may be used as the area. The reference image may be stored in the reference picture buffer 190. Here, when encoding / decoding for the reference image is performed, it may be stored in the reference picture buffer 190.
[0067] The motion compensation unit 112 may generate a prediction block of the current block by performing motion compensation using a motion vector. Here, inter prediction may refer to inter-picture prediction or motion compensation.
[0068] When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a partial area of the reference picture. In order to perform inter prediction or motion compensation, it may be determined whether the motion prediction and motion compensation mode of the prediction unit included in the coding unit is one of a skip mode, a merge mode, an advanced motion vector prediction (AMVP) mode, and a coding unit-based intra block copy (IBC) mode, and inter prediction or motion compensation may be performed according to each mode.
[0069] In addition, based on the above-mentioned inter-frame prediction method, the AFFINE mode based on sub-PU prediction, the SbTMVP (sub-block-based temporal motion vector prediction) mode, the MMVD (merged with MVD) mode based on PU prediction, and the GPM (geometric partitioning mode) mode can be applied. In addition, in order to improve the performance of each mode, HMVP (history-based MVP), PAMVP (paired average MVP), CIIP (combined intra / inter prediction), AMVR (adaptive motion vector resolution), BDOF (bidirectional optical flow), BCW (bidirectional prediction with CU weights), LIC (local illumination compensation), TM (template matching), OBMC (overlapping block motion compensation), etc. can be applied.
[0070] Among them, the AFFINE mode is a technology used in both AMVP and MERGE modes, and has high coding efficiency. In the existing video decoding standard, because MC (motion compensation) is performed by considering only the parallel motion of the block, its disadvantage is that it cannot properly compensate for the actual motion, such as enlargement / reduction and rotation. To supplement this, a four-parameter affine motion model using two control point motion vectors (CPMV) and a six-parameter affine motion model using three control point motion vectors can be used and applied to inter-frame prediction. Here, CPMV is a vector representing an affine motion model of one of the upper left, upper right, and lower left of the current block. The AFFINE mode is divided into AMVP or MERGE mode for CPMV encoding. At the same time, considering the computational complexity of video decoding, affine motion compensation can be performed in units of 4×4 blocks without performing pixel-by-pixel affine motion compensation. That is, when viewed in units of 4×4 blocks, it is the same as the existing motion compensation, but from the perspective of the entire PU, it can be regarded as affine motion compensation.
[0071] The subtractor 113 may generate a residual block by using the difference between the input block and the prediction block. The residual block may be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming or quantizing, or transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal of a block unit.
[0072] The transform unit 130 may generate a transform coefficient by performing a transform on the residual block, and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When the transform skip mode is applied, the transform unit 130 may skip the transform of the residual block.
[0073] The quantized levels may be generated by applying quantization to the transform coefficients or the residual signal. Hereinafter, the quantized levels may also be referred to as transform coefficients in embodiments.
[0074] For example, a 4×4 luminance residual block generated by intra prediction is transformed using a basic vector based on DST (discrete sine transform), and a basic vector based on DCT (discrete cosine transform) can be used to transform the remaining residual block. In addition, using RQT (Residual Quad Tree) technology, the transform block is divided into a quadtree shape for one block, and after transforming and quantizing each transform block divided by RQT, a decoded block flag (cbf) can be transmitted when all coefficients become 0 to increase coding efficiency.
[0075] As another alternative, a multiple transform selection (MTS) technique that selectively uses multiple transform bases to perform transforms may be applied. In this case, instead of dividing the CU into TUs by RQT, a function similar to TU partitioning may be performed by a sub-block transform (SBT) technique. For example, SBT is only applied to inter-prediction blocks, and unlike RQT, the current block can be divided into 1 / 2 or 1 / 4 sizes in the vertical or horizontal direction, and then the transform may be performed on only one of the blocks. For example, if it is divided vertically, the transform may be performed on the leftmost or rightmost block, and if it is divided horizontally, the transform may be performed on the topmost or bottommost block.
[0076] In addition, LFNST (Low Frequency Non-Separable Transform) can be applied. LFNST is a secondary transform technique that further transforms the residual signal transformed into the frequency domain by DCT or DST. LFNST further transforms the 4×4 or 8×8 low-frequency region in the upper left so that the residual coefficients can be concentrated in the upper left.
[0077] The quantization unit 140 may generate a quantization level by quantizing the transform coefficient or the residual signal according to a quantization parameter (QP) and output the generated quantization level. Here, the quantization unit 140 may quantize the transform coefficient by using a quantization matrix.
[0078] For example, a quantizer with a QP value of 0 to 51 may be used. Alternatively, if the image size is large and high encoding efficiency is required, a QP of 0 to 63 may be used. In addition, a DQ (Dependent Quantization) method using two quantizers instead of one quantizer may be applied. DQ performs quantization using two quantizers (e.g., Q0 and Q1), but even without using signaling information about a specific quantizer, a quantizer to be used for the next transform coefficient may be selected based on the current state through a state transition model.
[0079] The entropy encoding unit 150 may generate a bit stream by performing entropy encoding on the value calculated by the quantization unit 140 or on the encoding parameter value calculated when performing encoding according to the probability distribution, and output the bit stream. The entropy encoding unit 150 may perform entropy encoding on information about samples of an image and information for decoding the image. For example, the information for decoding the image may include a syntax element.
[0080] When entropy coding is applied, symbols are represented so that a smaller number of bits are allocated to symbols with a high probability of occurrence and a larger number of bits are allocated to symbols with a low probability of occurrence, and therefore, the size of the bitstream for the symbol to be encoded can be reduced. The entropy coding unit 150 may use a coding method such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. for entropy coding. For example, the entropy coding unit 150 may perform entropy coding by using a variable length coding / coding (VLC) table. In addition, the entropy coding unit 150 may derive a binary method of a target symbol and a probability model of a target symbol / bin, and perform arithmetic decoding by using the derived binary method and context model.
[0081] In this regard, when CABAC is applied, in order to reduce the size of the probability table stored in the decoding device, the table probability update method can be changed to a table update method using a simple equation and applied. In addition, two different probability models can be used to obtain more accurate symbol probability values.
[0082] In order to encode the transformation coefficient level (quantization level), the entropy encoding unit 150 may change the two-dimensional block form coefficient into a one-dimensional vector form through a transformation coefficient scanning method.
[0083] The encoding parameters may include information (flags, indexes, etc.) encoded in the encoding device 100 and notified to the decoding device 200 by signals (such as syntax elements) and information derived in the encoding or decoding process, and may represent information required when encoding or decoding an image.
[0084] Here, signaling a flag or an index may mean that the corresponding flag or index is entropy encoded in an encoder and included in a bitstream, and may mean that the corresponding flag or index is entropy decoded from the bitstream in a decoder.
[0085] The encoded current image may be used as a reference image of another image to be processed later. Therefore, the encoding apparatus 100 may reconstruct or decode the encoded current image again and store the reconstructed or decoded image in the reference picture buffer 190 as a reference image.
[0086] The quantized level may be dequantized in the dequantization unit 160, or may be inversely transformed in the inverse transform unit 170. The dequantized and / or inversely transformed coefficient may be added to the prediction block by the adder 117. Here, the dequantized and / or inversely transformed coefficient may mean a coefficient on which at least one of dequantization and inverse transformation is performed, and may mean a reconstructed residual block. The dequantization unit 160 and the inverse transform unit 170 may be performed as inverse processes of the quantization unit 140 and the transform unit 130.
[0087] The reconstructed block may pass through the filter unit 180. The filter unit 180 may apply a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), a bilateral filter (BIF), a luminance mapping with chroma scaling (LMCS), etc. to the reconstructed sample, the reconstructed block, or the reconstructed image using all or some filtering techniques. The filter unit 180 may be referred to as an in-loop filter. In this case, the in-loop filter is also used as a name excluding LMCS.
[0088] The deblocking filter can remove block distortion generated in the boundary between blocks. In order to determine whether to apply the deblocking filter, it can be determined whether to apply the deblocking filter to the current block based on samples included in several rows or columns included in the block. When the deblocking filter is applied to the block, different filters can be applied according to the required deblocking filter strength.
[0089] In order to compensate for the encoding error using the sample adaptive offset, an appropriate offset value may be added to the sample value. The sample adaptive offset may correct the offset of the deblocked image from the original image by sample units. A method of dividing the samples included in the image into a predetermined number of regions, determining the regions to which the offset is applied, and applying the offset to the determined regions, or a method of applying the offset in consideration of edge information about each sample may be used.
[0090] The bilateral filter (BIF) can also correct the deviation from the original image on a sample-by-sample basis for the deblocked image.
[0091] The adaptive loop filter may perform filtering based on a comparison result of the reconstructed image and the original image. The samples included in the image may be divided into predetermined groups, the filter to be applied to each group may be determined, and differential filtering may be performed for each group. Information on whether the ALF is applied may be signaled by a coding unit (CU), and the form and coefficient of the adaptive loop filter applied to each block may be changed.
[0092] In LMCS (luminance mapping with chrominance scaling), luminance mapping (LM) means remapping luminance values by a piecewise linear model, and chrominance scaling (CS) means a technique for scaling the residual values of chrominance components according to the average luminance value of a prediction signal. Specifically, LMCS can be used as an HDR correction technique that reflects the characteristics of an HDR (High Dynamic Range) image.
[0093] The reconstructed block or the reconstructed image that has passed through the filter unit 180 may be stored in the reference picture buffer 190. The reconstructed block that has passed through the filter unit 180 may be a part of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed block that has passed through the filter unit 180. The stored reference image may be used later in inter-frame prediction or motion compensation.
[0094] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment of the present invention.
[0095] The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.
[0096] refer to Figure 2 , the decoding device 200 may include an entropy decoding unit 210, a dequantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, a motion compensation unit 250, an adder 201, a switch 203, a filter unit 260 and a reference picture buffer 270.
[0097] The decoding device 200 may receive a bit stream output from the encoding device 100. The decoding device 200 may receive a bit stream stored in a computer-readable recording medium, or may receive a bit stream transmitted through a wired / wireless transmission medium stream. The decoding device 200 may decode the bit stream in an intra-frame mode or an inter-frame mode. In addition, the decoding device 200 may generate a reconstructed image or a decoded image generated by decoding, and output the reconstructed image or the decoded image.
[0098] When the prediction mode for decoding is the intra mode, the switch 203 may be switched to intra. Alternatively, when the prediction mode for decoding is the inter mode, the switch 203 may be switched to inter.
[0099] The decoding device 200 can obtain a reconstructed residual block by decoding the input bit stream and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding device 200 can generate a reconstructed block that becomes a decoding target by adding the reconstructed residual block and the prediction block. The decoding target block can be referred to as a current block.
[0100] The entropy decoding unit 210 may generate symbols by entropy decoding the bit stream according to the probability distribution. The generated symbols may include symbols in the form of quantization levels. Here, the entropy decoding method may be an inverse process of the above entropy encoding method.
[0101] The entropy decoding unit 210 may change the coefficients of the one-dimensional vector shape into coefficients of the two-dimensional block shape through a transform coefficient scanning method to decode the transform coefficient level (quantized level).
[0102] The quantized level may be dequantized in the dequantization unit 220 or inversely transformed in the inverse transform unit 230. The quantized level may be a result of dequantization and / or inverse transformation and may be generated as a reconstructed residual block. Here, the dequantization unit 220 may apply a quantization matrix to the quantized level. The dequantization unit 220 and the inverse transform unit 230 applied to the decoding device may apply the same technology as the dequantization unit 160 and the inverse transform unit 170 applied to the above-mentioned encoding device.
[0103] When the intra mode is used, the intra prediction unit 240 can generate a prediction block by performing spatial prediction on the current block using sample values of blocks decoded around the decoding target block. The intra prediction unit 240 applied to the decoding device can apply the same technology as the intra prediction unit 120 applied to the above-mentioned encoding device.
[0104] When the inter-frame mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation on the current block using a motion vector and a reference image stored in the reference picture buffer 270. When the value of the motion vector is not an integer value, the motion compensation unit 250 may generate a prediction block by applying an interpolation filter to a partial area within the reference image. In order to perform motion compensation, it may be determined based on the decoding unit whether the motion compensation method of the prediction unit included in the corresponding decoding unit is a skip mode, a merge mode, an AMVP mode, or a current picture reference mode, and motion compensation may be performed according to each mode. The motion compensation unit 250 applied to the decoding device may apply the same technology as the motion compensation unit 122 applied to the above-mentioned encoding device.
[0105] The adder 201 may generate a reconstructed block by adding the reconstructed residual block and the prediction block. The filter unit 260 may apply at least one of an inverse LMCS, a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filter unit 260 applied to the decoding device may apply the same filtering technology as the filtering technology of the filter unit 180 applied to the above-mentioned encoding device.
[0106] The filter unit 260 may output a reconstructed image. The reconstructed block or the reconstructed image may be stored in the reference picture buffer 270 and used for inter-frame prediction. The reconstructed block that has passed through the filter unit 260 may be a part of the reference image. In other words, the reference image may be a reconstructed image composed of the reconstructed blocks that have passed through the filter unit 260. The stored reference image may be used later in inter-frame prediction or motion compensation.
[0107] Figure 3 FIG. 1 is a diagram schematically showing a video encoding system to which the present invention can be applied.
[0108] The video encoding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may transmit the encoded video and / or image information or data to the decoding device 20 in the form of a file or a stream through a digital storage medium or a network.
[0109] The encoding device 10 according to the embodiment may include a video source generating unit 11, an encoding unit 12, and a transmission unit 13. The decoding device 20 according to the embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmission unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may include a display unit, and the display unit may be configured as a separate device or an external component.
[0110] The video source generation unit 11 can obtain the video / image by the process of capturing, synthesizing or generating the video / image. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include (for example) one or more cameras, a video / image archive containing previously captured videos / images, etc. The video / image generation device may include, for example, a computer, a tablet computer, a smart phone, etc., and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc., in which case the video / image capture process may be replaced by a process of generating relevant data.
[0111] The encoding unit 12 may encode the input video / image. The encoding unit 12 may perform a series of processes for compression and encoding efficiency, such as prediction, transformation, and quantization. The encoding unit 12 may output encoded data (encoded video / image information) in the form of a bit stream. The detailed configuration of the encoding unit 12 may also be similar to that described above. Figure 1 The encoding device 100 is configured in the same manner.
[0112] The transmission unit 13 may transmit the encoded video / image information or data output in the form of a bit stream to the receiving unit 21 of the decoding device 20 in the form of a file or stream transmission through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit 13 may include an element for generating a media file in a predetermined file format, and may include an element for sending through a broadcast / communication network. The receiving unit 21 may extract / receive a bit stream from a storage medium or a network and transmit it to the decoding unit 22.
[0113] The decoding unit 22 may decode the video / image by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transformation, and prediction. The detailed configuration of the decoding unit 22 may also be similar to that described above. Figure 2 The decoding device 200 is configured in the same manner.
[0114] The rendering unit 23 may render the decoded video / image. The rendered video / image may be displayed by the display unit.
[0115] Figure 4 is a diagram illustrating an embodiment (400) of a method for determining one of various inter prediction methods in an inter prediction mode.
[0116] According to the embodiment (400), when the current block is predicted and encoded or decoded by inter prediction, in step 410, it is determined whether the inter prediction mode of the current block is a merge mode or an advanced motion vector prediction (AMVP) mode.
[0117] The merge mode is an inter-frame prediction mode in which motion information (such as a motion vector) and a reference picture of the current block are obtained from neighboring blocks of the current block. On the other hand, according to the AMVP mode, a predicted motion vector is obtained from neighboring blocks of the current block, and other motion information (such as a motion vector difference) and reference picture information other than the predicted motion vector are obtained by parsing the bitstream. Therefore, the AMVP mode is different from the merge mode in which all motion information of neighboring blocks is used for prediction of the current block.
[0118] If the inter prediction mode of the current block is the merge mode, it is determined whether the inter prediction mode of the current block is the subblock merge mode in step 420. If the inter prediction mode of the current block is the subblock merge mode, in step 422, the current block is divided into a plurality of subblocks according to the subblock merge mode, and each subblock may be predicted based on a motion vector according to an affine transformation.
[0119] If the inter prediction mode of the current block is not the sub-block merge mode, then in step 430, it is determined whether the inter prediction mode of the current block is the normal merge mode. If the inter prediction mode of the current block is not the normal merge mode, then in step 432, it is determined whether the inter prediction mode of the current block is the combined intra inter prediction (CIIP) mode.
[0120] If the inter prediction mode of the current block is not the combined intra inter prediction mode, in step 434, it is determined that the inter prediction mode of the current block is a geometric partitioning mode. The geometric partitioning mode is an inter prediction mode in which the current block is divided into two partitions based on a predetermined boundary, and a final prediction block of the current block is determined by combining two prediction blocks of the two partitions. In step 436, the inter prediction mode of the current block is determined to be a combined intra inter prediction mode. According to the combined intra inter prediction mode, the final prediction block of the current block can be determined by combining a prediction block according to intra prediction of the current block and a prediction block according to inter prediction of the current block.
[0121] If the inter prediction mode of the current block is the normal merge mode, in step 440, it is determined whether the inter prediction mode of the current block is a merge mode with motion vector difference (MMVD). According to the merge mode with motion vector difference, a final motion vector of the current block is determined by adding the motion vector difference to the motion vector obtained from the neighboring block. In the merge mode with motion vector difference, the direction of the motion vector difference may be limited to one of +x, -x, +y, and -y. In addition, in the merge mode with motion vector difference, the magnitude of the motion vector difference may be limited to be selected from a limited number of predetermined size candidates.
[0122] If the inter prediction mode of the current block is a merge mode with motion vector difference, the current block is predicted according to the merge mode with motion vector difference. If the current block is bidirectionally predicted, the prediction block of the current block may be adjusted by a bidirectional optical flow (BDOF) mode at step 446.
[0123] If the inter prediction mode of the current block is not a merge mode with motion vector difference, a normal merge mode is applied to the current block in step 444. If the current block is bi-directionally predicted, the motion vector of the current block may be refined according to a decoder-side motion vector refinement mode in step 450. In step 452, similar to step 446, the prediction block of the current block may be adjusted by a bi-directional optical flow mode.
[0124] Decoder-side motion vector refinement is a method of refining motion vectors by a motion vector search process based on bilateral matching without parsing additional information in decoding the bidirectional motion vector derived in the regular merge mode. According to the decoder-side motion vector refinement, the accuracy of the motion vector in the regular merge mode can be improved. In addition, the coding efficiency of the regular merge mode can be improved accordingly.
[0125] exist Figure 4 In the description, the decoder side motion vector refinement mode is described as being applied only to the normal merge mode, but depending on the implementation, the decoder side motion vector refinement mode can also be applied to the sub-block merge mode, the geometric partitioning mode, the combined intra-frame inter-frame prediction mode, and the merge mode with motion vector difference.
[0126] Based on the tag information parsed from the bitstream, Figure 4 The inter prediction mode of the current block is determined in steps 410, 420, 430, 432 and 440. For example, a merge flag may be parsed in step 410, a sub-block merge flag may be parsed in step 420, a regular merge flag may be parsed in step 430, a combined intra-frame inter-frame prediction flag may be parsed in step 432, and a merge flag with motion vector difference may be parsed from the bitstream.
[0127] Hereinafter, embodiments of the decoder-side motion vector refinement described in step 450 in the present invention will be described in detail. In the present invention, various embodiments of the conditions for performing the decoder-side motion vector refinement are provided. In addition, embodiments are provided in which the motion vector is searched based on template matching (TM) instead of searching the motion vector based on bilateral matching in the decoder-side motion vector refinement. In addition, embodiments are provided in which different decoding unit weights are used in the decoder-side motion vector refinement.
[0128] In accordance with Figure 4In one embodiment of decoder-side motion vector refinement (DMVR) of the present invention, a bidirectional motion vector predicted in a conventional merge mode is refined by motion search based on bilateral matching (BM). According to an embodiment, neighboring blocks of a reference block may be searched from an initial motion vector to find an optimal reference block. For example, a motion vector that minimizes the degree of distortion between two reference blocks located in two reference pictures based on bilateral matching (BM) may be searched from the initial motion vector.
[0129] Decoder-side motion vector refinement may not be targeted at all bidirectional motion vectors predicted in regular merge mode, and may only be applied when one or more predetermined conditions are met. The predetermined conditions are described below.
[0130] According to an embodiment, decoder-side motion vector refinement may be applied when a coding unit (CU) block is in merge mode. In addition, decoder-side motion vector refinement may be applied when the coding unit block is not in sub-block merge mode to which inter prediction according to an affine transform is applied. In addition, decoder-side motion vector refinement may be applied when merge mode with motion vector difference is not applied to the coding unit block.
[0131] According to an embodiment, decoder-side motion vector refinement may be applied when the coding unit block is in a bidirectional prediction mode. In addition, according to an embodiment, decoder-side motion vector refinement may be applied when two reference pictures referenced by the coding unit block are respectively located in a temporally opposite direction to the current picture. For example, decoder-side motion vector refinement may be applied when a first reference picture of the two reference pictures is temporally before the current picture and a second reference picture is temporally after the current picture.
[0132] According to an embodiment, decoder-side motion vector refinement may be applied when the temporal distance between two reference pictures and the current picture is the same. The temporal distance may mean the magnitude of the POC (Picture Order Count) difference between the reference picture and the current picture. In the present invention, the distance between pictures represents the temporal distance and the magnitude of the POC difference.
[0133] In some embodiments, decoder-side motion vector refinement may be applied even when the temporal distances between two reference pictures and the current picture are different. By not applying the same temporal distance condition, decoder-side motion vector refinement is more applicable in regular merge mode. Thus, coding efficiency may be improved as decoder-side motion vector refinement is performed more by relaxing the above performance conditions.
[0134] According to an embodiment, decoder-side motion vector refinement may be applied when both reference pictures are short-term reference pictures.
[0135] According to an embodiment, whether to apply decoder-side motion vector refinement may be determined based on the size of the decoding unit block. For example, when the size of the decoding unit block is greater than a predetermined size, decoder-side motion vector refinement may be applied. The predetermined size may be expressed as the number of luminance samples included in the decoding unit block. In addition, the predetermined size may be a power of 2, such as 64, 128, 256, 512, or 1024.
[0136] According to an embodiment, whether to apply decoder-side motion vector refinement may be determined based on the width and height of the decoding unit block. For example, when the height and / or width of the decoding unit block is greater than a predetermined value, decoder-side motion vector refinement may be applied. The predetermined value may be a power of 2, such as 4, 8, 16, or 32.
[0137] According to an embodiment, decoder-side motion vector refinement may be applied when the bidirectional decoding unit weights applied to the two prediction blocks of the decoding unit block are the same as each other. The final prediction block of the decoding unit block is determined as the weighted average of the two prediction blocks obtained from the bidirectional prediction. In an embodiment, the bidirectional decoding unit weights may be used to determine the weighted average of the two prediction blocks. However, in some embodiments, decoder-side motion vector refinement may be configured to be applied even when the bidirectional decoding unit weights applied to the two prediction blocks are different from each other.
[0138] According to an embodiment, when combined intra-frame inter-frame prediction (CIIP) is not applied, decoder-side motion vector refinement may be applied. According to an embodiment, combined intra-frame inter-frame prediction is a prediction method that determines a final prediction block by performing a weighted average of a first prediction block derived from intra-frame prediction and a second prediction block derived from inter-frame prediction on a block.
[0139] According to an embodiment, the conditions for performing decoder-side motion vector refinement may include at least one of the above-mentioned conditions.
[0140] As the conditions for performing decoder-side motion vector refinement increase, the frequency of decoder-side motion vector refinement may decrease. Conversely, as the conditions for performing decoder-side motion vector refinement decrease, the frequency of decoder-side motion vector refinement may increase. Therefore, the frequency of motion vector refinement and the encoding efficiency of the merge mode according thereto may be determined according to the conditions for decoder-side motion vector refinement.
[0141] Figure 5 An implementation of decoder-side motion vector refinement is shown when the temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture.
[0142] exist Figure 5, the current block 500 and the L0 reference picture 520 have a POC distance N. In addition, the current block 500 and the L1 reference picture 540 also have a POC distance N. Therefore, the distance between the current block 500 and the L0 reference picture 520 is the same as the distance between the current block 500 and the L1 reference picture 540.
[0143] The current block 500 may obtain initial motion vectors MV0 522 and MV1 542 in the L0 direction and the L1 direction based on motion information of the neighboring blocks. MV0 522 and MV1 542 respectively indicate initial reference blocks 524 and 544. In some cases, since there may be reference blocks more suitable for prediction of the current block around the initial reference blocks 524 and 544, a motion vector indicating a more suitable reference block is searched based on bidirectional matching.
[0144] As described above, MV0 522 and MV1 542 are the initial motion vectors in the L0 and L1 directions, respectively, derived in the normal merge mode. MV0′ 528 is obtained by MV diff 526 is a motion vector obtained by refining MV0 522 as the initial motion vector in the L0 direction. In addition, MV1′ 548 is obtained by -MV diff 546 refines the motion vector obtained by MV1 542 which is the initial motion vector in the L1 direction. MV diff 526 and -MV diff 546 are L0 motion vector difference and L1 motion vector difference, respectively. The L0 motion vector difference and L1 motion vector difference have the same magnitude, but their directions are set to be opposite.
[0145] If the distortion between the reference block 530 indicated by MV0' 528 in the L0 reference picture 520 and the reference block 550 indicated by MV1' 548 in the L1 reference picture 540 is minimized, MV0' 528 and MV1' 548 may be determined as the final motion vector of the current block 502. In various embodiments, different distortion measurement methods such as the sum of absolute differences (SAD) or the sum of squared errors (SSE) between two reference blocks may be used.
[0146] Finally, when MV0'528 and MV1'548 are the final motion vectors with the minimum distortion, the final motion vectors MV0'528 and MV1'548 are used to determine the two prediction blocks P L0 530 and P L1 550.
[0147] Hereinafter, bidirectional prediction with coding unit level weight (BCW) will be described.
[0148] In bidirectional prediction using coding unit level weight (BCW), a prediction block may be generated by applying adaptive weights to two prediction blocks derived from motion vectors in the L0 direction and the L1 direction. In bidirectional prediction using coding unit level weight, the prediction block may be calculated using Equation 1.
[0149] [Equation 1]
[0150] Pred bi-pred =((8-w)×P 0 +w×P 1 +4)>>3
[0151] Wherein, P0 and P1 represent the motion-refined prediction block from the L0 reference picture and the motion-refined prediction block from the L1 reference picture, respectively. w represents the weight applied to each reference block, and the value of w is determined differently depending on whether the current picture is a low-delay picture. If the current picture is a low-delay picture, five weights (w∈{-2, 3, 4, 5, 10}) can be used. If the current picture is not a low-delay picture, three weights (w∈{3, 4, 5}) can be used.
[0152] In bidirectional prediction with coding unit level weights, if the current coding unit (CU) block is in merge mode, weight information can be derived from the merge candidate block. If the current block is not in merge mode, weight information can be parsed after parsing motion vector difference information.
[0153] Equation 2 shows a method of generating a final prediction block using decoder-side motion vector refinement.
[0154] [Equation 2]
[0155] Pred Final =(P L0 +P L1 +1)>>1
[0156] In Equation 2, P L0 , P L1 and P redFinal They respectively represent the prediction block indicated by the MV0' motion vector in the L0 reference picture, the prediction block indicated by the MV1' motion vector in the L1 reference picture, and the final prediction block generated using the two prediction blocks. Figure 5 As shown in FIG, since the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are the same, the same weight can be assigned to the prediction block P generated from the corresponding reference picture. L0 and P L1 To generate the final prediction block.
[0157] The method of generating a prediction block according to the decoder-side motion vector refinement method according to an embodiment may be used only when the temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture. In this case, the prediction block may be generated by averaging two prediction blocks generated from two reference pictures. However, if the prediction block is generated using the average value in all cases, there may be a limitation on encoding efficiency. Therefore, when generating the final prediction block, the prediction block may be generated by averaging the two prediction blocks generated from the two reference pictures. L0 and L1 prediction block P L1 Different weights are assigned to generate a final prediction block. In this case, in order to generate a final prediction block by assigning adaptive weights to two prediction blocks, Equation 1 for generating a final prediction block in bidirectional prediction with CU-level weight (BCW) may be used.
[0158] In bidirectional prediction with CU-level weights, different weights may be used depending on whether the current picture is a low-delay picture. In bidirectional prediction with CU-level weights, if the current coding unit (CU) block is in merge mode, the weight information may be derived from the merge candidate block. Otherwise, if not in merge mode, the weight information may be parsed from the bitstream.
[0159] According to an embodiment, in bidirectional prediction with CU-level weights, a prediction block can be generated according to the determined weights. For example, five weights (w∈{-2, 3, 4, 5, 10}) used in a low-latency picture can be used to generate a prediction block. Alternatively, three weights (w∈{3, 4, 5}) used in cases other than low-latency pictures can be used to generate prediction blocks. Alternatively, a predetermined arbitrary weight can be used. In this case, the number of weights used can be arbitrarily determined.
[0160] According to an embodiment, the weight information determined by the encoder is transmitted and parsed. In various embodiments, as a method of transmitting / parsing the weight information, a fixed length code (FLC), a unary code, or a truncated binary code may be used in consideration of the weights used.
[0161] In contrast, the bidirectional coding unit weights may be determined based on the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture.
[0162] Hereinafter, a method of applying decoder-side motion vector refinement and a method of generating a final prediction block when a temporal distance between a current picture and an L0 reference picture and a temporal distance between a current picture and an L1 reference picture are not the same are described.
[0163] If the application condition of the decoder-side motion vector refinement method requires the condition that the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are the same, when the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are not the same, the decoder-side motion vector refinement is not applied to the current block. However, if the above condition is excluded from the application condition of the decoder-side motion vector refinement method, the decoder-side motion vector refinement may be applied to the current block even if the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are different.
[0164] Therefore, when the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are not the same, various embodiments for decoder-side motion vector refinement and derivation of the final prediction block can be discussed. In the present invention, embodiments of the relationship between the L0 motion vector difference and the L1 motion vector difference for decoder-side motion vector refinement will be discussed below. In addition, embodiments of the present invention for determining the required weights for the final prediction block are discussed below.
[0165] Figure 6 Decoder-side motion vector refinement is shown when the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture are not the same.
[0166] exist Figure 6 In , the current picture 600 is a picture at time t, and the L0 reference picture 620 and the L1 reference picture 640 are pictures at time tM (tM>0) and time t+N (t+N>0) (tM<t+N), respectively. Here, M and N are different (M<N) arbitrary positive integer values. Therefore, as Figure 6 As shown, the temporal distance between the L0 reference picture 620 and the current picture 600 is different from the temporal distance between the L1 reference picture 640 and the current picture 600 .
[0167] according to Figure 6 , the L0 direction motion vector MV0 622 and the L1 direction motion vector MV1 642 derived in the normal merge mode are initial motion vectors. MV0 622 and MV1 642 indicate initial reference blocks 624 and 644, respectively. diff_L0 626 refines MV0 622 to determine motion vector MV0′ 628. diff_L1 646 refines MV1 642 to determine motion vector MV1′ 648. Considering the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, the motion vector MV1′ 648 can be obtained from MVdiff_L0 626 Confirm MV diff_L1 646. When block P L0 630 and block P L1 When the distortion between the current picture and the L0 reference picture is minimized, the motion vectors MV0'628 and MV1'648 can be derived as the final motion vectors in the L0 direction and the L1 direction, respectively. Equation 3 shows the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture to calculate the MV diff_L0 Corresponding MV diff_L1 method.
[0168] [Equation 3]
[0169]
[0170] In Equation 3, M and N represent the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, respectively. Here, M and N are different arbitrary positive integer values. In addition, in Equation 3, MV diff_L1_x , MV diff_L1_y , MV diff_L0_x and MV diff_L0_y Respectively represent MV diff_L1 The x-direction motion information, MV diff_L1 Y-direction motion information, MV diff_L0 The x-direction motion information and MV diff_L0 As shown in Equation 3, by considering the ratio (M:N) of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, the y-direction motion information of MV is calculated symmetrically. diff_L0 Corresponding MV diff_L1 . The calculated MV diff_L1 (MV diff_L1_x and MV diff_L1_y ) The motion vector MV1′ obtained by refining the motion vector MV1 in the L1 direction can be derived as the final motion vector in the L1 direction. In Equation 3, by reflecting the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture to MV diff_L0 The motion information is used to calculate the MV diff_L1 Conversely, the ratio of the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture can also be reflected in the MV diff_L1 The motion information is used to calculate the MV diff_L0 sports information.
[0171] The MV determined in Equation 3 diff_L1_xand MV diff_L1_y By diff_L0_x and MV diff_L0_y Multiply by N / M to determine, and thus MV diff_L1_x and MV diff_L1_y The value of can be a non-integer value. Therefore, according to the implementation mode, MV diff_L1_x and MV diff_L1_y Can be adjusted to a value in integer units. In an embodiment, MV diff_L1_x and MV diff_L1_y may be integerized according to a rounding or truncation process. Alternatively, the MV may be adjusted with a predetermined precision other than integer units according to a rounding or truncation process. diff_L1_x and MV diff_L1_y The predetermined accuracy may be 1 / 2, 1 / 4, etc.
[0172] According to an embodiment, MV diff_L1_x , MV diff_L1_y , MV diff_L0_x and MV diff_L0_y The absolute value of can be limited to a predetermined range. The predetermined range can be determined as an integer sampling unit of 1, 2, 4, 8, etc. In addition, considering the ratio of M and N, it can be compared with MV diff_L0_x and MV diff_L0_y The predetermined range determines MV differently diff_L1_x and MV diff_L1_y For example, MV diff_L1_x , MV diff_L1_y , MV diff_L0_x and MV diff_L0_y The absolute value of can be set to be less than or equal to one of 1, 2, 4 and 8.
[0173] As described above, the motion vector accuracy may be improved by refining the motion vector in consideration of the ratio of the temporal distance between the current picture and the L0 reference picture to the temporal distance between the current picture and the L1 reference picture.
[0174] Hereinafter, an embodiment of determining the weights required for the final prediction block is described.
[0175] like Figure 6 As shown, if the temporal distance between the current picture and the L0 reference picture is different from the temporal distance between the current picture and the L1 reference picture, when applying the decoder-side motion vector refinement method, the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture may be considered to determine the bidirectional decoding unit weight. In addition, the final prediction block may be generated according to the determined weight.
[0176] According to the implementation mode, Figure 5As shown in Equation 2, when the temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture, the prediction block P indicated by the refined motion vectors MV0' and MV1' can be L0 and P L1 Using the same weights, a final prediction block is produced for the current block.
[0177] However, when Figure 6 As shown, when the temporal distance between the current picture and the L0 reference picture is different from the temporal distance between the current picture and the L1 reference picture, the weight to be applied to each prediction block can be determined considering the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture.
[0178] As described above, the L0 direction motion vector MV0 and the L1 direction motion vector MV1 may be determined as initial motion vectors. diff_L0 The motion vector MV0′ obtained by refining the L0 direction motion vector MV0 and the motion vector MV0′ obtained by using MV diff_L1 The motion vector MV1' obtained by refining the motion vector MV1 in the L1 direction can determine the prediction block P in the L0 direction L0 and the prediction block P in the L1 direction L1 .
[0179] According to the ratio of the temporal distance between the current picture and the L0 reference picture to the temporal distance between the current picture and the L1 reference picture, different weights are used to calculate the final prediction block. Hereinafter, Equation 4 shows a method of generating a final prediction block by considering the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, using a refined motion vector in a decoder to generate each prediction block, and assigning different weights to each prediction block.
[0180] [Equation 4]
[0181]
[0182] In Equation 4, M and N represent the temporal distance between the current picture and the L0 reference picture and the temporal distance between the current picture and the L1 reference picture, respectively. Here, M and N are different (M<N) arbitrary positive integer values. As shown in Equation 4, P is calculated by considering the ratio of the temporal distance between the current picture and the L0 reference picture to the temporal distance between the current picture and the L1 reference picture. L0 Weight and P L1 Weight According to Equation 4, the larger weight The prediction block P assigned to the reference picture closer to the current picture L0 On the contrary, a smaller weight The prediction block P assigned to the reference picture farther from the current picture L1 Since the weight used in Equation 4 is derived in consideration of the ratio of the temporal distance between the current picture and the L0 reference picture to the temporal distance between the current picture and the L1 reference picture, there is no need to explicitly obtain the weight information from the bitstream.
[0183] According to an embodiment, even if the temporal distance between the current picture and the L0 reference picture is different from the temporal distance between the current picture and the L1 reference picture, weight information may be parsed from the bitstream. According to an embodiment, even if the temporal distance between the current picture and the L0 reference picture is different from the temporal distance between the current picture and the L1 reference picture, the weight may be determined considering characteristics of the picture, or the weight may be determined by obtaining weight information from the bitstream.
[0184] In addition, predetermined arbitrary weights may be used to generate a prediction block. For example, weights used in a low-delay picture may be applied, or weights used in a case other than a low-delay picture may be applied. Alternatively, a predetermined arbitrary weight value may be applied. In this case, the number of weights used may also be arbitrarily determined.
[0185] The weight information determined by the encoder is parsed from the bitstream. In various embodiments, as a method of transmitting / parsing the weight information, any effective method such as fixed length code (FLC), unary code or truncated binary code may be used, taking into account the weights used.
[0186] According to an embodiment, decoder-side motion vector refinement and final prediction block generation may be performed using a template matching method.
[0187] In the decoder side motion vector refinement method, a template matching (TM) method may be applied instead of a bilateral matching (BM) method. The template matching method may be applied to the decoder side motion vector refinement method regardless of whether the temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture.
[0188] Figure 7 Decoder-side motion vector refinement based on a template matching approach is shown.
[0189] exist Figure 7In the example, the current picture 700 is a picture at time t, and the L0 reference picture 720 and the L1 reference picture 740 are pictures at time tM (tM>0) and time t+N (t+N>0) (tM<t+N), respectively. Here, M and N are different (M<N) arbitrary positive integer values. Therefore, as Figure 7 As shown, the temporal distance between the L0 reference picture 720 and the current picture 700 and the temporal distance between the L1 reference picture 740 and the current picture 700 are not equal to M and N, respectively.
[0190] exist Figure 7 In the example, the final motion vectors MV0′ 728 and MV1′ 748 may be determined based on a template matching method by comparing the current template 704, the L0 reference template 732, and the L1 reference template 752. Subsequently, the prediction block P may be derived based on the determined final motion vectors MV0′ 728 and MV1′ 748. L0 730 and P L1 750 to generate the final prediction block.
[0191] In the decoder-side motion vector refinement method based on template matching, the motion vector MV0 722 in the L0 direction and the motion vector MV1 742 in the L1 direction derived in the normal merge mode are determined as initial motion vectors. In addition, a motion vector that minimizes the degree of distortion of the current template 704, the L0 reference template 732, and the L1 reference template 752 may be searched based on the two initial motion vectors. In addition, the motion vector with the minimum distortion may be determined as the final motion vectors MV0' 728 and MV1' 748.
[0192] Considering the ratio of the temporal distance between the current picture and the L0 reference picture to the temporal distance between the current picture and the L1 reference picture, it can be obtained from MV diff L0 Calculate MV diff L1 MV can be calculated according to the above equation 3 diff_L1 In contrast to Equation 3, it is also possible to diff_L1 The MV is calculated by taking the ratio of the temporal distance between the current picture and the L0 reference picture to the temporal distance between the current picture and the L1 reference picture. diff_L0 .
[0193] According to the template matching-based decoder-side motion vector refinement method of the present invention, a motion vector that minimizes the distortion between templates based on the initial motion vectors MV0 and MV1 is searched in two reference pictures. In an embodiment, the refined motion vector MV0 may be based on the MV representing the initial motion vector. diff_L0 and MV diff_L1 to perform the search. MV diff_L0 and MVdiff_L1 The MV is determined based on the ratio of the temporal distance between the L0 reference picture and the current picture to the temporal distance between the L1 reference picture and the current picture. diff_L0 and MV diff_L1 By dividing MV diff_L0 and MV diff_L1 Added to MV0 and MV1 respectively, refined motion vectors MV0' and MV1' may be determined.
[0194] In order to obtain distortion by the refined motion vectors MV0' and MV1', a final template is generated using a reference template (L0 reference template) of a block indicated by MV0' and a reference template (L1 reference template) of a block indicated by MV1'. In addition, the degree of distortion between the generated final template and the current template is determined. diff_L0 and MV diff_L1 The distortion is calculated for each of a certain range of MV0' and MV1' candidates. In addition, the MV0' and MV1' with the smallest distortion are determined as the final motion vector. In various embodiments, different distortion measurement methods such as the sum of absolute differences (SAD) or the sum of squared errors (SSE) can be used. Equations 5 and 6 show a method for calculating the final template from the L0 reference template and the L1 reference template.
[0195] [Equation 5]
[0196] template final =(Template L0 + Templates L1 +1)>>1
[0197] [Equation 6]
[0198]
[0199] In Equation 5 and Equation 6, the template L0 ,template L1 and templates final Denote the L0 reference template, the L1 reference template, and the final template, respectively. According to Equation 5, the final template is calculated as the average of the templates of the two reference pictures without considering the ratio of the temporal distance between the L0 reference picture and the current picture to the temporal distance between the L1 reference picture and the current picture. On the other hand, according to Equation 6, the final template is calculated from the L0 reference template and the L1 reference template according to the ratio of the temporal distance between the L0 reference picture and the current picture to the temporal distance between the L1 reference picture and the current picture. The final template can be calculated using either of Equations 5 and 6.
[0200] The decoder-side motion vector refinement method based on template matching can be applied to refine the bidirectional motion vectors predicted in the regular merge mode.
[0201] Hereinafter, an embodiment of a method of determining a weight based on template matching is described.
[0202] according to Figure 7 , obtained from the initial motion vector MV0 722 in the L0 direction and the initial motion vector MV1742 in the L1 direction by MV diff_L0 726 systematizes the motion vector MV0' 728 obtained by MV0 722 and diff_L1 746 is a motion vector MV1' 748 obtained by understanding MV1 742. In addition, an L0 reference template 732 of a block indicated by a motion vector MV0' 728 in the L0 reference picture 720 and an L1 reference template 752 of a block indicated by a motion vector MV1' 748 in the L1 reference picture 740 are determined. If the distortion between the L0 reference template 732 and the current template 704 is defined as DL0 and the distortion between the L1 reference template 752 and the current template 704 is defined as DL1, the final prediction block may be calculated using Equation 7.
[0203] [Equation 7]
[0204]
[0205] According to Equation 7, the distortion D between the L1 reference template and the current template is used L1 To calculate the prediction block P L0 Weight in the L0 direction Instead, the distortion D between the L0 reference template and the current template is used L0 Calculate the weight of the prediction block PL1 in the L1 direction The difference in these weights is due to the fact that a smaller distortion value indicates a greater similarity between the current signal and the reference signal, whereas a larger distortion value indicates a smaller similarity between the current signal and the reference signal. The distortion between the current template and the reference template may be determined based on any distortion measurement method, such as the sum of absolute differences (SAD) or the sum of squared errors (SSE).
[0206] The above-mentioned embodiments of the decoder-side motion vector refinement method may be combined within a range not conflicting with each other.
[0207] In the decoder side motion vector refinement method, the ratio of the distance between the two reference pictures and the current picture may be considered. Alternatively, the decoder side motion vector refinement method may be performed without considering the ratio of the distance between the two reference pictures and the current picture. In addition, in the decoder side motion vector refinement method, a bilateral matching method or a template matching method may be used.
[0208] In the method of determining the weight of motion vector refinement, the distance information between two reference pictures and the current picture may be considered. Alternatively, the weight may be determined by transmitting / parsing weight information from a bitstream. Alternatively, the weight may be determined based on a distortion value of template matching.
[0209] Thus, any of the various methods mentioned above may be used for motion vector refinement, and any of the various methods mentioned above may be used for weight determination.Thus, the motion vector refinement method and the weight determination method may be independently selected and used.
[0210] Thus, any combination of a decoder-side motion vector refinement method for selecting a bidirectional prediction block in the decoder and a method for determining weights to produce a final prediction block may be implemented.
[0211] Figure 8 is a flowchart of a decoder-side motion vector refinement method according to an embodiment.
[0212] In step 802, a first base motion vector of a current block of a first reference picture and a second base motion vector of a current block of a second reference picture are determined. According to an embodiment, the first reference picture and the second reference picture are located in a different temporal direction than the current block.
[0213] In step 804, a first refined motion vector is determined by refining the first base motion vector with the first motion vector difference, and a second refined motion vector is determined by refining the second base motion vector with the second motion vector difference.
[0214] According to an embodiment, when the distance between the current picture including the current block and the first reference picture is different from the distance between the current picture and the second reference picture, the first refined motion vector and the second refined motion vector may be determined. Alternatively, the first refined motion vector and the second refined motion vector may be determined regardless of the relationship between the distance between the current picture including the current block and the first reference picture and the distance between the current picture and the second reference picture.
[0215] According to an embodiment, a ratio of the magnitude of the first motion vector difference and the magnitude of the second motion vector difference may be set to be proportional to a ratio of a distance between the current picture and the first reference picture to a distance between the current picture and the second reference picture.
[0216] According to an embodiment, the magnitudes of the first motion vector difference and the second motion vector difference are limited to within a predetermined range.
[0217] According to an embodiment, the first refined motion vector and the second refined motion vector may be determined so as to minimize distortion between a first prediction block indicated by the first refined motion vector and a second prediction block indicated by the second refined motion vector according to a motion vector search based on bidirectional matching.
[0218] According to an embodiment, the first refined motion vector and the second refined motion vector may be determined so that, according to a motion vector search based on template matching, the distortion between the first template of the first prediction block indicated by the first refined motion vector, the second template of the second prediction block indicated by the second refined motion vector, and the current template of the current block is minimized. The distortion may be calculated based on a final template determined by a weighted average of the first template, the second template, and the current template.
[0219] According to an embodiment, in the weighted average of the first template and the second template, the first weight applied to the first template may be set to be proportional to the distance between the current picture and the second reference picture, and the second weight applied to the second template may be set to be proportional to the distance between the current picture and the first reference picture. In addition, the weighted average of the first template and the second template may be determined based on the set first weight and second weight.
[0220] In step 806, a first prediction block and a second prediction block of the current block are determined based on the first refined motion vector and the second refined motion vector.
[0221] In step 808, a final prediction block of the current block is determined based on a weighted sum of the first prediction block and the second prediction block.
[0222] According to an embodiment, the weighted sum of the first prediction block and the second prediction block may be determined by a weight determined according to a distance between the current picture and the first reference picture and a distance between the current picture and the second reference picture.
[0223] According to an embodiment, a first weight applied to the first prediction block may be set in proportion to the distance between the current picture and the second reference picture, and a second weight applied to the second prediction block may be set in proportion to the distance between the current picture and the first reference picture.
[0224] According to an embodiment, a first weight applied to the first prediction block may be determined in proportion to distortions of the second template and the current template, and a second weight applied to the second prediction block may be determined in proportion to distortions of the first template and the current template.
[0225] According to an embodiment, the weighted sum of the first prediction block and the second prediction block may be determined by a weight determined based on a picture type of a current picture including the current block.
[0226] According to an embodiment, the weighted sum of the first prediction block and the second prediction block may be determined by a weight determined based on weight information generated by parsing a bitstream.
[0227] In the decoder, the Figure 8 The current block is reconstructed based on the final prediction block of the current block reconstructed by the decoder-side motion vector refinement method. In the encoder, the current block can be reconstructed based on the final prediction block of the current block reconstructed by the decoder-side motion vector refinement method. Figure 8 The final prediction block of the current block reconstructed by the decoder-side motion vector refinement method is used to encode the current block.
[0228] Can provide storage for applications Figure 8 A computer-readable recording medium for a bit stream generated by a video encoding method for a decoder-side motion vector refinement method. Figure 8 The bit stream generated by the video encoding method of the decoder side motion vector refinement method can be stored in a computer readable recording medium. Figure 8 A bitstream generated by a video encoding method of a decoder-side motion vector refinement method may be transmitted from a video encoding device to a video decoding device.
[0229] Can be applied to Figure 8 The video decoding method of the decoder side motion vector refinement method is used to decode the bit stream of video data stored in the computer readable recording medium. In addition, the bit stream sent from the video encoding device to the video decoding device can be applied to Figure 8 A video decoding method of a decoder-side motion vector refinement method for decoding.
[0230] Fig. 9 The content streaming system to which the embodiments of the present invention can be applied is exemplarily shown.
[0231] like Fig. 9 As shown, the content streaming transmission system to which the embodiment of the present invention is applied may mainly include an encoding server, a streaming transmission server, a network server, a media storage, a user device, and a multimedia input device.
[0232] The encoding server compresses the content received from the multimedia input device (such as a smart phone, camera, CCTV, etc.) into digital data to generate a bit stream and transmits it to the streaming server. As another example, if the multimedia input device (e.g., a smart phone, camera, CCTV, etc.) directly generates a bit stream, the encoding server can be omitted.
[0233] A bitstream may be generated by applying the image encoding method and / or the image encoding apparatus according to the embodiments of the present invention, and in the process of transmitting or receiving the bitstream, the streaming server may temporarily store the bitstream.
[0234] The streaming server transmits multimedia data to the user device via the web server based on the user request, and the web server can act as an intermediary to inform the user of any available services. When the user requests the desired service from the web server, the web server sends it to the streaming server, and the streaming server can send the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server can control the command / response between the devices within the content streaming system.
[0235] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a specific period of time.
[0236] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.
[0237] Each server in the above content streaming system may operate as a distributed server, in which case data received from each server may be distributed and processed.
[0238] The above embodiments may be performed in the encoding device and the decoding device in the same or corresponding manner. In addition, at least one embodiment of the above embodiments or a combination of at least one of the above embodiments may be used to encode / decode an image.
[0239] In the encoding device and the decoding device, the order of applying the above embodiments may be different. Alternatively, in the encoding device and the decoding device, the order of applying the above embodiments may be the same.
[0240] The above embodiments may be performed for each of the luminance and chrominance signals. Alternatively, the above embodiments for the luminance and chrominance signals may be performed identically.
[0241] In the above embodiments, the method is described based on a flow chart having a series of steps or units, but the present invention is not limited to the order of the steps, but some steps can be performed simultaneously with other steps or in a different order. In addition, it should be understood by those skilled in the art that the steps in the flow chart are not mutually exclusive, and other steps can be added to the flow chart or some steps can be deleted from the flow chart without affecting the scope of the present invention.
[0242] The embodiments may be implemented in the form of program instructions executable by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include independent program instructions, data files, data structures, etc. or a combination of program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, or may be known to those of ordinary skill in the field of computer software technology.
[0243] The bit stream generated by the encoding method according to the above-described embodiment may be stored in a non-transitory computer-readable recording medium. In addition, the bit stream stored in the non-transitory computer-readable recording medium may be decoded by the decoding method according to the above-described embodiment.
[0244] Examples of computer-readable recording media include magnetic recording media such as hard disks, floppy disks, and magnetic tapes; optical data storage media such as CD-ROMs or DVD-ROMs; magnetic optimal media such as floppy disks; and hardware devices such as read-only memory (ROM), random access memory (RAM), flash memory, etc., which are specially constructed to store and implement program instructions. Examples of program instructions include not only machine language codes formatted by a compiler, but also high-level language codes that can be implemented by a computer using an interpreter. Hardware devices can be configured to be operated by one or more software modules, and vice versa, to perform processing according to the present invention.
[0245] Although the present invention has been described with respect to specific items such as detailed elements and limited embodiments and drawings, these items are only provided to help a more comprehensive understanding of the present invention, and the present invention is not limited to the above embodiments. It will be appreciated by those skilled in the art that various modifications and changes may be made from the above description.
[0246] Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the full scope of the appended claims and their equivalents will fall within the scope and spirit of the present invention.
[0247] Industrial Applicability
[0248] The present invention can be used for an apparatus for encoding / decoding an image and a recording medium for storing a bit stream.
Claims
1. A video decoding method, include: determining a first base motion vector for a current block of a first reference picture and a second base motion vector for the current block of a second reference picture; determining a first refined motion vector by refining the first base motion vector with a first motion vector difference, and determining a second refined motion vector by refining the second base motion vector with a second motion vector difference; determining a first prediction block and a second prediction block of the current block based on the first refined motion vector and the second refined motion vector; as well as A final prediction block of the current block is determined based on a weighted sum of the first prediction block and the second prediction block.
2. The video decoding method according to claim 1, in, Determining the first refined motion vector and the second refined motion vector includes determining the first refined motion vector and the second refined motion vector when a distance between a current picture including the current block and the first reference picture is different from a distance between the current picture and the second reference picture.
3. The video decoding method according to claim 1, in, A weighted sum of the first prediction block and the second prediction block is determined by a weight determined according to a distance between a current picture and the first reference picture and a distance between the current picture and the second reference picture.
4. The video decoding method according to claim 3, in: A first weight applied to the first prediction block is proportional to a distance between the current picture and the second reference picture; as well as A second weight applied to the second prediction block is proportional to a distance between the current picture and the first reference picture.
5. The video decoding method according to claim 1, in, In determining the first refined motion vector and the second refined motion vector, a ratio of a magnitude of the first motion vector difference and a magnitude of the second motion vector difference is proportional to a ratio of a distance between a current picture and the first reference picture to a distance between the current picture and the second reference picture.
6. The video decoding method according to claim 1, in, The magnitude of the first motion vector difference and the magnitude of the second motion vector difference are limited within a predetermined range.
7. The video decoding method according to claim 1, in, The first refinement motion vector and the second refinement motion vector are determined to minimize distortion between the first prediction block indicated by the first refinement motion vector and the second prediction block indicated by the second refinement motion vector.
8. The video decoding method according to claim 1, in, The first and second refined motion vectors are determined to minimize distortion between a first template of the first prediction block indicated by the first refined motion vector, a second template of the second prediction block indicated by the second refined motion vector, and a current template of the current block.
9. The video decoding method according to claim 8, in, The distortion is calculated based on a final template determined by a weighted average of the first template, the second template, and the current template.
10. The video decoding method according to claim 9, in, In the weighted average of the first template and the second template, a first weight applied to the first template is proportional to the distance between the current picture and the second reference picture, and a second weight applied to the second template is proportional to the distance between the current picture and the first reference picture.
11. The video decoding method according to claim 8, in: A first weight applied to the first template is proportional to the distortion of the second template and the current template; as well as A second weight applied to the second template is proportional to the distortion of the first template and the current template.
12. The video decoding method according to claim 1, in, The weighted sum of the first prediction block and the second prediction block is determined by a weight determined based on a picture type of a current picture including the current block.
13. The video decoding method according to claim 1, in, The weighted sum of the first prediction block and the second prediction block is determined by a weight determined based on weight information generated by parsing a bitstream.
14. A video encoding method, include: determining a first base motion vector for a current block of a first reference picture and a second base motion vector for the current block of a second reference picture; determining a first refined motion vector by refining the first base motion vector with a first motion vector difference, and determining a second refined motion vector by refining the second base motion vector with a second motion vector difference; Determine a first prediction block and a second prediction block of the current block based on the first refined motion vector and the second refined motion vector; as well as A final prediction block of the current block is determined based on a weighted sum of the first prediction block and the second prediction block.
15. A computer-readable recording medium for storing a bit stream generated by a video encoding method, the video encoding method include: determining a first base motion vector for a current block of a first reference picture and a second base motion vector for the current block of a second reference picture; determining a first refined motion vector by refining the first base motion vector with a first motion vector difference, and determining a second refined motion vector by refining the second base motion vector with a second motion vector difference; Determine a first prediction block and a second prediction block of the current block based on the first refined motion vector and the second refined motion vector; as well as A final prediction block of the current block is determined based on a weighted sum of the first prediction block and the second prediction block.
16. A method for transmitting a bit stream generated by a video encoding method, the method include: Encoding the image based on the video encoding method; as well as The transmission includes a bit stream of coded images, The video encoding method includes: determining a first base motion vector for a current block of a first reference picture and a second base motion vector for the current block of a second reference picture; determining a first refined motion vector by refining the first base motion vector with a first motion vector difference, and determining a second refined motion vector by refining the second base motion vector with a second motion vector difference; determining a first prediction block and a second prediction block of the current block based on the first refined motion vector and the second refined motion vector; and A final prediction block of the current block is determined based on a weighted sum of the first prediction block and the second prediction block.
