Video encoding / decoding method and device, and recording medium on which bitstream is stored

Abstract

The video decoding / encoding method and apparatus according to the present disclosure may generate a base prediction block of a current block by performing bidirectional prediction based on a first inter prediction mode, generate an additional prediction block of the current block based on a second inter prediction mode, and generate a final prediction block of the current block by performing a weighted sum of the base prediction block and the additional prediction block.

Description

[Technical field]

[0001] The present invention relates to a video encoding / decoding method and apparatus, and a recording medium on which a bitstream is stored. [Background technology]

[0002] 2. Description of the Related Art In recent years, the demand for high-resolution, high-quality images such as high definition (HD) images and ultra high definition (UHD) images has been increasing in various application fields, and thus, highly efficient image compression techniques have been discussed.

[0003] There are various video compression techniques, such as inter-prediction techniques that predict pixel values ​​contained in a current picture from pictures before or after the current picture, intra-prediction techniques that predict pixel values ​​contained in the current picture using pixel information within the current picture, and entropy coding techniques that assign short codes to values ​​that occur frequently and long codes to values ​​that occur less frequently. Using such video compression techniques, video data can be effectively compressed and transmitted or stored. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure seeks to provide a method and apparatus for performing inter prediction based on multiple hypothesis prediction modes.

[0005] The present disclosure seeks to provide a method and apparatus for defining syntax parsing conditions and orders between various inter prediction modes, including multiple hypothesis prediction modes. [Means for solving the problem]

[0006] The image decoding method and apparatus according to the present disclosure may generate a base predictive block of a current block by performing bidirectional prediction based on a first inter prediction mode, generate an additional predictive block of the current block based on a second inter prediction mode, and generate a final predictive block of the current block by performing a weighted sum of the base predictive block and the additional predictive block.

[0007] In the video decoding method and apparatus according to the present disclosure, the first inter prediction mode may include at least one of a merge mode, an AMVP mode, or a combined merge-AMVP mode.

[0008] In the video decoding method and apparatus according to the present disclosure, the second inter prediction mode may include a multiple hypothesis prediction mode.

[0009] In the image decoding method and apparatus according to the present disclosure, when a plurality of additional predictive blocks are generated, the final predictive block may be generated by sequentially performing a weighted sum of the plurality of additional predictive blocks to the base predictive block.

[0010] The video decoding method and apparatus according to the present disclosure may further include obtaining information about the second inter-prediction mode from a bitstream.

[0011] In the video decoding method and apparatus according to the present disclosure, the information related to the second inter prediction mode may include at least one of weight information or prediction information.

[0012] In the video decoding method and apparatus according to the present disclosure, the weight information may represent information indicating a weight used in a weighted sum of the additional prediction block.

[0013] In the video decoding method and apparatus according to the present disclosure, the prediction information may represent information for deriving the additional prediction block.

[0014] In the video decoding method and apparatus according to the present disclosure, the prediction information may include a mode flag indicating whether a merge mode or an AMVP mode is used to derive the additional prediction block.

[0015] In the video decoding method and apparatus according to the present disclosure, when the mode flag indicates that a merge mode is used to derive the additional prediction block, the prediction information may further include a merge index.

[0016] In the video decoding method and apparatus of the present disclosure, when the mode flag indicates that AMVP mode is used to derive the additional prediction block, the prediction information may include at least one of a motion vector predictor flag, a reference index, or motion vector differential information.

[0017] In the video decoding method and apparatus according to the present disclosure, the information about the second inter prediction mode may be obtained from a bitstream if a predefined condition is satisfied.

[0018] In the video decoding method and apparatus according to the present disclosure, the predefined condition may be defined without taking into account adaptive motion vector resolution (AMVR) information.

[0019] The video decoding method and apparatus according to the present disclosure may obtain adaptive motion vector resolution (AMVR) information from a bitstream.

[0020] In the video decoding method and apparatus according to the present disclosure, the step of obtaining the AMVR information may be performed before the step of obtaining information about the second inter prediction mode.

[0021] The video decoding method and apparatus according to the present disclosure may obtain a Motion Vector Sign Derivation (MVSD) index from a bitstream, which identifies a candidate from among a plurality of candidate code combinations.

[0022] In the video decoding method and apparatus according to the present disclosure, the step of obtaining the MVSD index may be performed before the step of obtaining information about the second inter prediction mode.

[0023] The video encoding method and apparatus according to the present disclosure may generate a base predictive block of a current block by performing bidirectional prediction based on a first inter prediction mode, generate an additional predictive block of the current block based on a second inter prediction mode, and generate a final predictive block of the current block by performing a weighted sum of the base predictive block and the additional predictive block.

[0024] A computer-readable digital storage medium having encoded video / image information stored thereon is provided that enables a decoding apparatus according to the present disclosure to perform a video decoding method.

[0025] A computer-readable digital storage medium having stored thereon video / image information generated by the video encoding method according to the present disclosure is provided.

[0026] A method and apparatus are provided for transmitting video / image information generated by the video encoding method according to the present disclosure. Effect of the Invention

[0027] The present disclosure can improve prediction accuracy by performing inter prediction based on multiple hypothesis prediction modes.

[0028] The present disclosure improves the syntax structure by defining syntax parsing conditions and orders between various inter prediction modes, including multiple hypothesis prediction modes, thereby improving compression performance. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 illustrates a video / image coding system according to the present disclosure.

[0030] [Diagram 2] 1 is a schematic block diagram of an encoding device to which an embodiment of the present disclosure can be applied, in which video / image signals are encoded;

[0031] [Diagram 3] 1 is a schematic block diagram of a decoding device to which an embodiment of the present disclosure can be applied, for decoding a video / image signal;

[0032] [Figure 4] FIG. 1 illustrates an example of an inter-prediction based video / image encoding method to which embodiments of the present disclosure can be applied.

[0033] [Diagram 5] FIG. 2 illustrates an example of an inter-prediction based video / picture decoding method to which embodiments of the present disclosure can be applied.

[0034] [Figure 6] FIG. 1 illustrates an inter-prediction procedure to which the embodiments of the present disclosure can be applied.

[0035] [Figure 7] 1 is a diagram illustrating an inter-prediction method performed by a decoding device 300 according to an embodiment of the present disclosure.

[0036] [Figure 8]1 is a flowchart illustrating an example of a syntax parsing order according to an embodiment of the present disclosure.

[0037] [Figure 9] FIG. 1 is a diagram showing a schematic configuration of an inter prediction unit (332) that performs the inter prediction method according to the present disclosure.

[0038] [Figure 10] FIG. 2 is a diagram illustrating an inter-prediction method performed by an encoding device (200) according to an embodiment of the present disclosure.

[0039] [Figure 11] FIG. 2 is a diagram showing a schematic configuration of an inter prediction unit (221) that performs the inter prediction method according to the present disclosure.

[0040] [Figure 12] FIG. 1 illustrates an example of a content streaming system to which embodiments of the present disclosure can be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] The present disclosure may have various modifications and embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to a specific embodiment, and it should be understood that the present disclosure includes any modifications, equivalents, or alternatives within the spirit and technical scope of the present disclosure. In the description of each figure, similar reference numerals are used for similar components.

[0042] Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. These terms are only used to distinguish one component from another. For example, a first component may be named a second component, and similarly, a second component may be named a first component, without departing from the scope of the present disclosure. The term "and / or" includes any combination of multiple associated listed items or multiple associated listed items.

[0043] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that it may be directly coupled or connected to the other component, but there may be other components in between. On the other hand, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no other components in between.

[0044] The terms used in this application are merely for describing certain embodiments and are not intended to limit the present disclosure. The singular term includes the plural term unless otherwise specified in the context. In this application, the terms "comprise" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0045] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in the versatile video coding (VVC) standard. Also, the methods / embodiments disclosed herein may be applied to methods disclosed in the essential video coding (EVC) standard, the AOMedia Video1 (AV1) standard, the 2nd generation of audio video coding standard (AVS2) or next generation video / image coding standards (e.g., H.267 or H.268).

[0046] This specification presents various embodiments related to video / image coding, which may be performed in combination with each other, unless otherwise specified.

[0047] In this specification, video may mean a collection of a series of images over time. A picture generally means a unit showing one image in a specific time period, and a slice / tile is a unit constituting a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). One picture may be composed of one or more slices / tiles. A tile is a rectangular area composed of multiple CTUs in a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​CTUs having a height equal to the height of the picture and a width specified by a syntax requirement of a picture parameter set. A tile row is a rectangular area of ​​CTUs having a height specified by a picture parameter set and a width equal to the width of the picture. Within one tile, CTUs are arranged consecutively by CTU raster scanning, while tiles within one picture may be arranged consecutively by tile raster scanning. A slice may contain an integer number of complete tiles or an integer number of consecutive complete CTU rows within the tiles of a picture that may be exclusively contained in a single NAL unit, whereas a picture may be partitioned into two or more sub-pictures, which may be rectangular regions of one or more slices within a picture.

[0048] A picture element, pixel, or pel can mean the smallest unit that constitutes one picture (or image). A term corresponding to a pixel may be "sample." A sample can generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance (luma) component, or may represent only a pixel / pixel value of a chrominance (chroma) component.

[0049] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (ex. cb, cr) blocks. A unit may be used interchangeably with terms such as block or area in some cases. In general, an MxN block may include a set (or array) of samples or transform coefficients consisting of M columns and N rows.

[0050] As used herein, "A or B" can mean "A only," "B only," or "both A and B." In other words, as used herein, "A or B" can be interpreted as "A and / or B." For example, as used herein, "A, B or C" can mean "A only," "B only," "C only," or "any combination of A, B and C."

[0051] As used herein, a slash ( / ) or a comma can mean "and / or." For example, "A / B" can mean "A and / or B." Thus, "A / B" can mean "A only," "B only," or "both A and B." For example, "A, B, C" can mean "A, B, or C."

[0052] In this specification, "at least one of A and B" can mean "A only," "B only," or "both A and B." In addition, in this specification, the expressions "at least one of A or B" and "at least one of A and / or B" may be interpreted as the same as "at least one of A and B."

[0053] In addition, in this specification, "at least one of A, B and C" can mean "A only," "B only," "C only," or "any combination of A, B and C." In addition, "at least one of A, B or C" or "at least one of A, B and / or C" can mean "at least one of A, B and C."

[0054] In addition, parentheses used in this specification may mean "for example." Specifically, when "prediction (intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra prediction," and "intra prediction" may be proposed as an example of "prediction." In addition, when "prediction (i.e., intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction."

[0055] In this specification, technical features described separately in one drawing may be embodied separately or simultaneously.

[0056] FIG. 1 is a diagram illustrating a video / image coding system according to this disclosure.

[0057] Referring to FIG. 1, a video / image coding system may include a first device (a source device) and a second device (a receiving device).

[0058] A source device may transmit encoded video / image information or data to a receiving device via a digital storage medium or a network in the form of a file or streaming. The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The sending unit may be included in the encoding device. The receiving unit may be included in the decoding device. The renderer may also include a display unit, which may be configured as a separate device or an external component.

[0059] A video source may obtain video / video by a video / video capture, synthesis, or generation process, etc. A video source may include a video / video capture device and / or a video / video generation device. A video / video capture device may include one or more cameras, a video / video archive containing previously captured video / video, etc. A video / video generation device may include a computer, tablet, smartphone, etc., and may (electronically) generate video / video. For example, a virtual video / video may be generated by a computer, etc., in which case the process of generating the associated data may replace the video / video capture process.

[0060] An encoding device may encode input video / image. The encoding device may perform a series of steps such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) may be output in the form of a bitstream.

[0061] The transmitting unit may transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit may include elements for generating a media file according to a predetermined file format and may include elements for transmission via a broadcasting / communication network. The receiving unit may receive / extract the bitstream and transmit it to a decoding device.

[0062] The decoding device can decode the video / image by performing a series of steps such as inverse quantization, inverse transformation, and prediction, which correspond to the operations of the encoding device.

[0063] The renderer can render the decoded video / image. The rendered video / image can be displayed on a display unit.

[0064] FIG. 2 is a block diagram illustrating a schematic configuration of an encoding device to which the embodiments of the present disclosure can be applied, in which video / image signals are encoded.

[0065] Referring to FIG. 2, the encoding apparatus 200 may include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter predictor 221 and an intra predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. The image divider 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filtering unit 260 may be configured by one or more hardware components (e.g., an encoding device chipset or processor) depending on the embodiment. Also, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0066] The image division unit 210 may divide an input image (or picture, frame) input to the encoding device 200 into one or more processing units. As an example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree ternary-tree (QTBTTT) structure.

[0067] For example, one coding unit may be divided into a plurality of coding units having a deeper depth based on a quad tree structure, a binary tree structure, and / or a tannery structure. In this case, for example, the quad tree structure may be applied first, and the binary tree structure and / or the tannery structure may be applied later. Or, the binary tree structure may be applied before the quad tree structure. A coding procedure according to the present specification may be performed based on the final coding unit that is not further divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be immediately used as the final coding unit, or the coding unit may be recursively divided into coding units of a lower depth as necessary, and a coding unit having an optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later.

[0068] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the above-mentioned final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0069] The unit may be used interchangeably with terms such as block or area in some cases. In the general case, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, may represent only a pixel / pixel value of a luma component, or may represent only a pixel / pixel value of a chroma component. A sample may be used as a term corresponding to one picture (or image) for a pixel or pel.

[0070] The encoding apparatus 200 may generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (prediction block, prediction sample array) output from the inter prediction unit 221 or the intra prediction unit 222 from an input video signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, in the encoding apparatus 200, a unit that subtracts the prediction signal (prediction block, prediction sample array) from the input video signal (original block, original sample array) may be called a subtraction unit 231.

[0071] The prediction unit 220 may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including a prediction sample for the current block. The prediction unit 220 may determine whether intra prediction or inter prediction is applied to the current block or CU. The prediction unit 220 may generate various information related to prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit the information to the entropy encoding unit 240. The prediction information may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0072] The intra prediction unit 222 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located at a certain distance away from the current block depending on the prediction mode. In the intra prediction, the prediction mode may include one or more non-directional modes and a plurality of directional modes. The non-directional mode may include at least one of a DC mode or a planar mode. The directional mode may include 33 directional modes or 65 directional modes depending on the fineness of the prediction direction. However, this is merely an example, and more or less directional modes may be used depending on the setting. The intra prediction unit 222 may also determine the prediction mode applied to the current block using the prediction modes applied to the neighboring blocks.

[0073] The inter prediction unit 221 may derive a prediction block for a current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of a block, a sub-block, or a sample based on the correlation of motion information between a neighboring block and a current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In inter prediction, the neighboring block may include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, the inter prediction unit 221 may generate information indicating which candidate is used to derive a motion vector and / or a reference picture index of the current block by forming a motion information candidate list based on neighboring blocks. Inter prediction may be performed based on various prediction modes, for example, in skip mode and merge mode, the inter prediction unit 221 may use motion information of neighboring blocks as motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of a neighboring block is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0074] The prediction unit 220 may generate a prediction signal based on various prediction methods, which will be described later. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, and may also simultaneously apply intra prediction and inter prediction. This may be called CIIP (combined inter and intra prediction). The prediction unit may also use an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for coding of content images / moving images such as games, such as screen content coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in that a reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction methods described in this specification. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, a sample value within the picture may be signaled based on information regarding a palette table and a palette index. The predicted signal generated by the prediction unit 220 may be used to generate a reconstructed signal or a residual signal.

[0075] The transform unit 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. The CNT refers to a transform obtained based on a predicted signal generated using all previously restored pixels. In addition, the transform process may be applied to pixel blocks having the same square size, or may be applied to blocks of variable size other than a square.

[0076] The quantization unit 233 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoding unit 240, and the entropy encoding unit 240 may encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be called residual information. The quantization unit 233 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0077] The entropy encoding unit 240 can perform various encoding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. In addition to the quantized transform coefficients, the entropy encoding unit 240 can also encode information required for video / image restoration (e.g., values ​​of syntax elements, etc.) together with or separately from the quantized transform coefficients.

[0078] The encoded information (e.g., encoded video / video information) may be transmitted or stored in the form of a bitstream in units of network abstraction layer (NAL) units. The video / video information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / video information may further include general constraint information. In this specification, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / video information. The video / video information may be encoded by the above-mentioned encoding procedure and included in the bitstream. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media, such as a USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. A transmitting unit (not shown) for transmitting the signal output from the entropy encoding unit 240 and / or a memory unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0079] The quantized transform coefficients output from the quantizer 233 may be used to generate a prediction signal. For example, the inverse quantizer 234 and the inverse transformer 235 may apply inverse quantization and inverse transform to the quantized transform coefficients to reconstruct a residual signal (residual block or residual sample). The adder 250 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to a prediction signal output from the inter predictor 221 or the intra predictor 222. When there is no residual for the current block to be processed, such as when a skip mode is applied, the predicted block may be used as a reconstructed block. The adder 250 may be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next current block to be processed in the current picture, or may be used for inter prediction of the next picture after filtering as described below. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied in the picture encoding and / or reconstruction process.

[0080] The filtering unit 260 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 260 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may store the modified reconstructed picture in the memory 270, specifically, in the DPB of the memory 270. The various filtering methods may include deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 260 may generate various information related to filtering and transmit it to the entropy encoding unit 240. The information related to filtering may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0081] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter prediction unit 221. In this way, when inter prediction is applied, the encoding apparatus can avoid prediction mismatch between the encoding apparatus 200 and the decoding apparatus, and can also improve coding efficiency.

[0082] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter prediction unit 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of an already reconstructed block in the picture. The stored motion information may be transmitted to the inter prediction unit 221 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 270 may store reconstructed samples of blocks reconstructed in the current picture and transmit them to the intra prediction unit 222.

[0083] FIG. 3 is a block diagram illustrating a decoding device to which the embodiments of the present disclosure can be applied, for decoding a video / image signal.

[0084] 3, the decoding device 300 may include an entropy decoding unit (entropy decoder, 310), a residual processor (residual processor, 320), a prediction unit (predictor, 330), an adder (adder, 340), a filtering unit (filter, 350), and a memory (memory, 360). The prediction unit 330 may include an inter prediction unit 332 and an intra prediction unit 331. The residual processor 320 may include a dequantizer (dequantizer, 321) and an inverse transformer (inverse transformer, 321).

[0085] The entropy decoding unit 310, the residual processing unit 320, the prediction unit 330, the addition unit 340, and the filtering unit 350 may be configured as one hardware component (e.g., a decoding chipset or a processor) according to an embodiment. Also, the memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware components may further include the memory 360 as an internal / external component.

[0086] When a bitstream including video / image information is input, the decoding apparatus 300 can restore an image corresponding to a process in which the video / image information is processed by the encoding apparatus of FIG. 2. For example, the decoding apparatus 300 can derive a unit / block based on block division related information obtained from the bitstream. The decoding apparatus 300 can perform decoding using a processing unit applied by the encoding apparatus. Thus, the processing unit for decoding may be a coding unit, and the coding unit may be a quad tree structure, a binary tree structure, and / or a tannery tree structure, which is divided from a coding tree unit or a maximal coding unit. One or more transform units may be derived from the coding unit. Then, the restored image signal decoded and output by the decoding apparatus 300 may be reproduced by a reproduction device.

[0087] The decoding apparatus 300 may receive a signal output from the encoding apparatus of FIG. 2 in the form of a bitstream, and the received signal may be decoded by the entropy decoding unit 310. For example, the entropy decoding unit 310 may parse the bitstream to derive information (e.g., video / image information) required for image restoration (or picture restoration). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / image information may further include general constraint information. The decoding apparatus may decode a picture further based on information on the parameter set and / or the general constraint information. Signaling / received information and / or syntax elements described later in this specification may be decoded by the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 may decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients related to residuals. More specifically, the CABAC entropy decoding method may receive bins corresponding to each syntax element from the bitstream, determine a context model using information on the syntax element to be decoded, decoding information on neighboring and blocks to be decoded, or information on symbols / bins decoded in a previous step, predict the occurrence probability of the bins according to the determined context model, and perform arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. In this case, the CABAC entropy decoding method may update the context model using information on the decoded symbols / bins for the context model of the next symbol / bin after determining the context model.Among the information decoded by the entropy decoding unit 310, information related to prediction is provided to the prediction unit 330 (inter prediction unit 332 and intra prediction unit 331), and residual values ​​entropy decoded by the entropy decoding unit 310, i.e., quantized transform coefficients and related parameter information, may be input to the residual processing unit 320. The residual processing unit 320 may generate a residual signal (residual block, residual sample, residual sample array). Also, among the information decoded by the entropy decoding unit 310, information related to filtering may be provided to the filtering unit 350. Meanwhile, a receiving unit (not shown) for receiving a signal output from an encoding device may be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropy decoding unit 310.

[0088] Meanwhile, the decoding device according to the present specification may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoding device (video / image / picture sample decoding device). The information decoding device may include the entropy decoding unit 310, and the sample decoding device may include at least one of the inverse quantization unit 321, the inverse transform unit 322, the adder 340, the filtering unit 350, the memory 360, the inter prediction unit 332, and the intra prediction unit 331.

[0089] The inverse quantization unit 321 may inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit 321 may rearrange the quantized transform coefficients in the form of a two-dimensional block. In this case, the rearrangement may be performed based on a coefficient scan order performed in an encoding device. The inverse quantization unit 321 may perform inverse quantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0090] The inverse transform unit 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0091] The prediction unit 320 may perform prediction on a current block and generate a predicted block including a prediction sample for the current block. The prediction unit 320 may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoding unit 310, and may determine a specific intra / inter prediction mode.

[0092] The prediction unit 320 may generate a prediction signal based on various prediction methods, which will be described later. For example, the prediction unit 320 may apply intra prediction or inter prediction for prediction of one block, and may also simultaneously apply intra prediction and inter prediction. This may be referred to as a combined inter and intra prediction (CIIP) mode. In addition, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for coding of content images / moving images such as games, such as screen content coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in that a reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction methods described in this specification. The palette mode may be regarded as an example of intra coding or intra prediction. When the palette mode is applied, information regarding a palette table and a palette index may be included in the video / image information and signaled.

[0093] The intra prediction unit 331 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located a certain distance away from the current block depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and a plurality of directional modes. The intra prediction unit 331 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the neighboring blocks.

[0094] The inter prediction unit 332 may derive a prediction block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 332 may configure a motion information candidate list based on the neighboring blocks, and may derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating an inter prediction mode for the current block.

[0095] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to a prediction signal (prediction block, prediction sample array) output from a prediction unit (including the inter prediction unit 332 and / or the intra prediction unit 331). When there is no residual for the current block, such as when the skip mode is applied, the prediction block may be used as the reconstructed block.

[0096] The adder 340 may be referred to as a reconstruction unit or a reconstruction block generator. The generated reconstruction signal may be used for intra prediction of a next block to be processed in a current picture, may be output through filtering as described below, or may be used for inter prediction of a next picture. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during picture decoding.

[0097] The filtering unit 350 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 350 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods may include deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, and the like.

[0098] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter prediction unit 332. The memory 360 may store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 332 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 360 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 331.

[0099] In this specification, the embodiments described for the filtering unit 260, inter prediction unit 221 and intra prediction unit 222 of the encoding apparatus 200 may also be applied identically or correspondingly to the filtering unit 350, inter prediction unit 332 and intra prediction unit 331 of the decoding apparatus 300, respectively.

[0100] Meanwhile, when inter prediction is applied, a prediction unit of an encoding device / decoding device can derive a prediction sample by performing inter prediction on a block-by-block basis. Inter prediction refers to a prediction induced by a method that depends on data elements (e.g., sample values ​​or motion information) of a picture other than the current picture. When inter prediction is applied to a current block, a predicted block (prediction sample array) for the current block can be derived based on a reference block (reference sample array) identified by a motion vector on a reference picture indicated by a reference picture index.

[0101] In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and / or a reference picture index. The motion information may further include inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.) information. When inter prediction is applied, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture.

[0102] The reference picture including the reference block and the reference picture including the temporal surrounding block may be the same or different. The temporal surrounding block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal surrounding block may be called a collocated picture (colPic). For example, a motion information candidate list may be constructed based on the surrounding blocks of the current block, and flag or index information indicating which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block may be signaled.

[0103] Inter prediction may be performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected neighboring block. In skip mode, unlike merge mode, a residual signal may not be transmitted. In motion vector prediction (MVP) mode, the motion vector of the selected neighboring block may be used as a motion vector predictor, and a motion vector difference may be signaled. In this case, the motion vector of the current block may be derived using the sum of the motion vector predictor and the motion vector difference.

[0104] The motion information may include L0 motion information and / or L1 motion information depending on an inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction may be referred to as an L0 motion vector or MVL0, and a motion vector in the L1 direction may be referred to as an L1 motion vector or MVL1. Prediction based on an L0 motion vector may be referred to as an L0 prediction, prediction based on an L1 motion vector may be referred to as an L1 prediction, and prediction based on both the L0 motion vector and the L1 motion vector may be referred to as a bi-prediction. Here, the L0 motion vector may indicate a motion vector associated with a reference picture list L0 (L0), and the L1 motion vector may indicate a motion vector associated with a reference picture list L1 (L1). The reference picture list L0 may include previous pictures in output order compared to the current picture as reference pictures, and the reference picture list L1 may include subsequent pictures in output order compared to the current picture. The previous pictures may be referred to as forward (reference) pictures, and the subsequent pictures may be referred to as backward (reference) pictures.

[0105] The reference picture list L0 may further include a subsequent picture in output order as a reference picture compared to the current picture. In this case, the previous picture may be indexed first in the reference picture list L0, and the subsequent picture may be indexed next. The reference picture list L1 may further include a previous picture in output order as a reference picture compared to the current picture. In this case, the subsequent picture may be indexed first in the reference picture list L1, and the previous picture may be indexed next. Here, the output order may correspond to a picture order count (POC) order.

[0106] A video / picture encoding procedure based on inter prediction may generally include, for example:

[0107] FIG. 4 illustrates an example of an inter-prediction based video / image encoding method to which embodiments of the present disclosure can be applied.

[0108] The encoding apparatus may perform inter prediction on a current block (S400). The encoding apparatus may induce an inter prediction mode and motion information of the current block, and generate a prediction sample of the current block. Here, the inter prediction mode determination, motion information induction, and prediction sample generation procedures may be performed simultaneously, or any one procedure may be performed prior to the other procedures. For example, the inter prediction unit of the encoding apparatus may include a prediction mode determination unit, a motion information induction unit, and a prediction sample induction unit, and the prediction mode determination unit may determine a prediction mode for the current block, the motion information induction unit may induce motion information of the current block, and the prediction sample induction unit may induce a prediction sample of the current block.

[0109] For example, the inter prediction unit of the encoding apparatus may search for a block similar to the current block within a certain region (search region) of a reference picture by motion estimation, and derive a reference block whose difference with the current block is minimum or equal to or less than a certain criterion. Based on this, a reference picture index indicating a reference picture in which the reference block is located may be derived, and a motion vector may be derived based on a position difference between the reference block and the current block. The encoding apparatus may determine a mode to be applied to the current block from among various prediction modes. The encoding apparatus may compare RD costs for the various prediction modes to determine an optimal prediction mode for the current block.

[0110] For example, when a skip mode or a merge mode is applied to the current block, the encoding apparatus may construct a merge candidate list (to be described later) and derive a reference block having a minimum or a certain criterion or less difference between the current block and the reference block indicated by a merge candidate included in the merge candidate list. In this case, a merge candidate associated with the induced reference block may be selected, and merge index information indicating the selected merge candidate may be generated and signaled to a decoding apparatus. Motion information of the current block may be derived using motion information of the selected merge candidate.

[0111] As another example, when the (A)MVP mode is applied to the current block, the encoding apparatus may configure an (A)MVP candidate list described below, and may use a motion vector of a motion vector predictor candidate selected from the motion vector predictor (mvp) candidates included in the (A)MVP candidate list as a motion vector predictor of the current block. In this case, for example, a motion vector indicating a reference block derived by the above-mentioned motion estimation may be used as the motion vector of the current block, and a motion vector predictor candidate having a motion vector with the smallest difference from the motion vector of the current block among the motion vector predictor candidates may become the selected motion vector predictor candidate. A motion vector difference (MVD), which is a difference obtained by subtracting the motion vector predictor from the motion vector of the current block, may be derived. In this case, information regarding the MVD may be signaled to the decoding apparatus. Also, when the (A)MVP mode is applied, the value of the reference picture index may be configured as reference picture index information and separately signaled to the decoding apparatus.

[0112] The encoding apparatus may derive a residual sample based on the predicted sample (S410). The encoding apparatus may derive the residual sample by comparing an original sample of the current block with the predicted sample.

[0113] The encoding apparatus may encode video information including prediction information and residual information (S420). The encoding apparatus may output the encoded video information in the form of a bitstream. The prediction information is information related to the prediction procedure and may include prediction mode information (e.g., a skip flag, a merge flag, or a merge index) and / or motion information. The motion information may include candidate selection information (e.g., a merge index, an MVP flag, or an MVP index) that is information for guiding a motion vector. The motion information may also include information related to the above-mentioned MVD and / or reference picture index information. The motion information may also include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information related to the residual sample. The residual information may include information related to a quantized transform coefficient for the residual sample.

[0114] The output bitstream may be stored on a (digital) storage medium and transmitted to a decoding device, or may be transmitted to a decoding device over a network.

[0115] Meanwhile, as described above, the encoding apparatus can generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the reference sample and the residual sample. This is for the purpose of inducing the same prediction result as that performed by the decoding apparatus in the encoding apparatus, thereby improving coding efficiency. Therefore, the encoding apparatus can store the reconstructed picture (or reconstructed sample, reconstructed block) in a memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. may be further applied to the reconstructed picture.

[0116] A video / picture decoding procedure based on inter prediction may generally include, for example:

[0117] FIG. 5 illustrates an example of an inter-prediction based video / picture decoding method to which embodiments of the present disclosure can be applied.

[0118] 5, the decoding apparatus may perform operations corresponding to those performed by the encoding apparatus, such as performing prediction on a current block based on received prediction information, and deriving a prediction sample.

[0119] Specifically, the decoding apparatus may determine a prediction mode for the current block based on received prediction information (S500). The decoding apparatus may determine which inter prediction mode is applied to the current block based on prediction mode information in the prediction information.

[0120] For example, it may be determined whether the merge mode or the (A)MVP mode is applied to the current block based on a merge flag, or one of various inter prediction mode candidates may be selected based on a mode index. The inter prediction mode candidates may include skip mode, merge mode, and / or (A)MVP mode, or may include various inter prediction modes described below.

[0121] The decoding apparatus may derive motion information of the current block based on the determined inter prediction mode (S510). For example, when a skip mode or a merge mode is applied to the current block, the decoding apparatus may form a merge candidate list (described later) and select one merge candidate from among the merge candidates included in the merge candidate list. The selection may be performed based on the above-mentioned selection information (merge index). Motion information of the current block may be derived using motion information of the selected merge candidate. The motion information of the selected merge candidate may be used as motion information of the current block.

[0122] As another example, when the (A)MVP mode is applied to the current block, the decoding apparatus may construct an (A)MVP candidate list described below, and use a motion vector of an MVP candidate selected from the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. The selection may be performed based on the above-mentioned selection information (MVP flag or MVP index). In this case, the MVD of the current block may be derived based on information on the MVD, and the motion vector of the current block may be derived based on the MVP of the current block and the MVD. Also, the reference picture index of the current block may be derived based on the reference picture index information. A picture indicated by the reference picture index in the reference picture list for the current block may be derived as a reference picture referenced for inter prediction of the current block.

[0123] On the other hand, the motion information of the current block may be derived without constructing a candidate list as described below, in which case the motion information of the current block may be derived according to a procedure disclosed in a prediction mode as described below. In this case, the candidate list construction as described above may be omitted.

[0124] The decoding apparatus may generate a prediction sample for the current block based on the motion information of the current block (S520). In this case, the reference picture may be derived based on the reference picture index of the current block, and the prediction sample for the current block may be derived using a sample of the reference block indicated on the reference picture by the motion vector of the current block. In this case, as described below, a prediction sample filtering procedure may be further performed on all or a part of the prediction samples of the current block, depending on the circumstances.

[0125] For example, the inter prediction unit of the decoding device may include a prediction mode determination unit, a motion information induction unit, and a prediction sample induction unit, in which the prediction mode determination unit determines a prediction mode for the current block based on received prediction mode information, the motion information induction unit induces motion information (such as a motion vector and / or a reference picture index) of the current block based on information regarding the received motion information, and the prediction sample induction unit induces a prediction sample of the current block.

[0126] The decoding apparatus generates a residual sample for the current block based on the received residual information (S530). The decoding apparatus generates a reconstructed sample for the current block based on the predicted sample and the residual sample, and generates a reconstructed picture based on the reconstructed sample (S540). Then, an in-loop filtering procedure, etc. may be further applied to the reconstructed picture, as described above.

[0127] FIG. 6 illustrates an inter prediction procedure to which the embodiments of the present disclosure can be applied.

[0128] 6, as described above, the inter prediction procedure may include an inter prediction mode determination step, a motion information deriving step according to the determined prediction mode, and a prediction execution (prediction sample generation) step based on the induced motion information. The inter prediction procedure may be performed in an encoding device and a decoding device as described above. In this document, a coding device may include an encoding device and / or a decoding device.

[0129] Referring to FIG. 6, the coding apparatus determines an inter prediction mode for a current block (S600). Various inter prediction modes may be used for predicting a current block in a picture. For example, various modes such as merge mode, skip mode, motion vector prediction (MVP) mode, affine mode, sub-block merge mode, and merge with MVD (MMVD) mode may be used. Decoder side motion vector refinement (DMVR) mode, adaptive motion vector resolution (AMVR) mode, bi-prediction with CU-level weight (BCW), bi-directional optical flow (BDOF), and the like may be used in addition to or instead of the additional modes. According to an embodiment of the present disclosure, the above-mentioned inter prediction modes may include a multi-hyphesis prediction (MHP) mode. The multi-hyphesis prediction mode refers to a method of performing a prediction by weighting a prediction block generated based on additional motion information for a bi-directional prediction (or bi-prediction) block. The multi-hyphesis prediction mode will be described in detail later.

[0130] In the present disclosure, the affine mode may be referred to as an affine motion prediction mode. Also, the MVP mode may be referred to as an advanced motion vector prediction mode. In the present disclosure, a motion information candidate induced by some modes and / or some modes may be included as one of the motion information related candidates of other modes. For example, an HMVP candidate may be added as a merge candidate of the merge / skip mode, or may be added as a motion vector predictor candidate of the AMVP mode. When the HMVP candidate is used as a motion information candidate of the merge mode or skip mode, the HMVP candidate may be referred to as an HMVP merge candidate.

[0131] Prediction mode information indicating an inter prediction mode of the current block may be signaled from the encoding apparatus to the decoding apparatus. The prediction mode information may be included in a bitstream and received by the decoding apparatus. The prediction mode information may include index information indicating one of a plurality of candidate modes. Alternatively, the inter prediction mode may be indicated by hierarchical signaling of flag information.

[0132] In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether a skip mode is applied, and if the skip mode is not applied, a merge flag may be signaled to indicate whether a merge mode is applied, and if the merge mode is not applied, an MVP mode may be applied, or a flag for additional partitioning may be further signaled. The affine mode may be signaled as an independent mode, or as a mode dependent on the merge mode or MVP mode, etc. For example, the affine mode may include an affine merge mode and an affine MVP mode.

[0133] The coding apparatus may derive motion information for the current block (S610). The motion information deriving may be based on the inter prediction mode. The coding apparatus may perform inter prediction using the motion information of the current block. The encoding apparatus may derive optimal motion information for the current block through a motion estimation procedure.

[0134] For example, the encoding apparatus can search for a similar reference block with high correlation in a fractional pixel unit within a specified search range in the reference picture using an original block in the original picture for the current block, thereby deriving motion information. The similarity of the blocks can be derived based on a difference in phase-based sample values. For example, the similarity of the blocks can be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, the motion information can be derived based on the reference block with the smallest SAD within the search range. The derived motion information can be signaled to the decoding apparatus in various ways based on the inter prediction mode.

[0135] The coding apparatus may perform inter prediction based on motion information for the current block (S620). The coding apparatus may generate predictive samples for the current block based on the motion information. The current block including the predictive samples may be referred to as a predicted block.

[0136] The multiple hypothesis prediction mode will be described in detail below. As described above, the multiple hypothesis prediction mode refers to a prediction method in which an additional prediction block (or predictor) is considered for a bidirectional prediction block. The multiple hypothesis prediction mode may be selectively used as one of the various inter prediction modes described above. Of course, the multiple hypothesis prediction mode according to the embodiment of the present disclosure is not limited to such names. In the present disclosure, the multiple hypothesis prediction mode may be called an MHP mode, a multiple hypothesis inter prediction mode, an inter-inter joint prediction mode, a joint inter prediction mode, a joint prediction mode, a multiple inter prediction mode, a multiple prediction mode, etc.

[0137] FIG. 7 is a diagram illustrating an inter prediction method performed by a decoding apparatus 300 according to an embodiment of the present disclosure.

[0138] Referring to FIG. 7, a decoding apparatus may perform bidirectional prediction to generate a base prediction block (S700). When an MHP mode is applied, the decoding apparatus may generate and combine an additional prediction block in addition to a prediction block generated (or derived) by bidirectional prediction. As an example, the base prediction block may include an L0 prediction block and / or an L1 prediction block. In the present disclosure, the base prediction block may be referred to as an initial prediction block, a temporary prediction block, a reference prediction block, etc. Also, as an example, the base prediction block may be a prediction block obtained by weighting the L0 and L1 prediction blocks.

[0139] In one embodiment, weights may be used for weighted sum of L0 and L1 prediction blocks. The weights are weights used for weighted prediction and may collectively refer to BCW (Bi-prediction with CU based Weights) and CW (CU based Weights). The weights may be derived from a weight candidate list. The weight candidate list may include a plurality of weight candidates and may be predefined in the encoding / decoding device.

[0140] The weight candidate may be a set of weights (i.e., a first weight and a second weight) indicating weights to be applied to each of the bidirectional predicted blocks, or may be weights to be applied to a predicted block in either one of the two directions. When only a weight to be applied to a predicted block in either one direction is derived from the weight candidate list, a weight to be applied to a predicted block in the other direction may be derived based on a weight derived from the weight candidate list. For example, a weight to be applied to a predicted block in the other direction may be derived by subtracting a weight derived from the weight candidate list from a predetermined value.

[0141] In one embodiment, a weight index indicating a weight used in weighted prediction of a current block may be induced in a weight candidate list. In the present disclosure, the weight index may be referred to as bcw_idx or bcw index. The weight index may be induced by a decoding device or may be signaled from an encoding device. When induced by a decoding device, the weight index may be induced as a weight index of a specific merge candidate in a merge candidate list. As an example, a specific merge candidate may be identified in a merge candidate list by a merge index.

[0142] The decoding apparatus may generate an additional prediction block based on the MHP mode (S710). The decoding apparatus may generate the additional prediction block in addition to the base prediction block, and may combine (or weighted-sum) the base prediction block and the generated additional prediction block.

[0143] In one embodiment, when the MHP mode is applied, the decoding apparatus may generate and combine up to a predefined number of additional prediction blocks. In other words, the decoding apparatus may combine (or weighted-sum) up to a predefined number of additional prediction blocks with a base prediction block. For example, the predefined number may be 2. Or, for example, the predefined number may be one of 1, 2, 3, and 4. The predefined number may be referred to as an MHP maximum number.

[0144] In addition, when a plurality of additional prediction blocks are combined, the plurality of additional prediction blocks may be sequentially weighted-summed to the base prediction block. For example, when a maximum of two additional prediction blocks are generated, a prediction block may be generated by weighting the base prediction block and a first additional prediction block, and a final prediction block may be generated by weighting the generated prediction block and a second additional prediction block. The prediction block generated by weighting the base prediction block and the first additional prediction block may be called an intermediate prediction block.

[0145] Alternatively, when a plurality of additional prediction blocks are combined, the base prediction block and the generated plurality of additional prediction blocks may be collectively weighted-summed. That is, after the plurality of additional prediction blocks are generated, a weight may be applied to each of the plurality of additional prediction blocks and the base prediction block (or the L0 prediction block and the L1 prediction block), and the weighted-sum may be collectively performed.

[0146] Also, in one embodiment, the decoding device may determine whether or not MHP is applied. In this case, a step of determining whether or not MHP is applied may be added prior to step S710. For example, whether or not MHP is applied may be explicitly signaled or may be implicitly induced by the decoding device.

[0147] In addition, in one embodiment, whether or not MHP is applied may be signaled from the encoding apparatus to the decoding apparatus. For example, an MHP flag indicating whether or not MHP is applied may be signaled from the encoding apparatus to the decoding apparatus. In this case, a condition for signaling / parsing the MHP flag may be predefined. The signaling / parsing condition of the MHP flag may be an MHP availability condition. If the signaling / parsing condition is satisfied, the decoding apparatus may parse the MHP flag from the bitstream. Alternatively, in one embodiment, whether or not MHP is applied may be induced by the decoding apparatus based on predefined coding information. For example, whether or not MHP is applied may be defined to be the same as an MHP availability condition (or a signaling / parsing condition) described below.

[0148] Also, as an embodiment, the decoding apparatus may obtain MHP information (which may also be called MHP prediction information) to generate an additional prediction block. As an example, the MHP information may include weight information and / or prediction information. Based on the prediction information, a reference block according to an MHP mode, i.e., an additional prediction block, may be derived, and the additional prediction block derived based on the weight information may be weighted and summed with a base prediction block (or an intermediate prediction block). Also, as an example, the MHP information may further include an MHP flag indicating whether or not MHP is applied.

[0149] As an example, the prediction information may include mode information used to induce the additional prediction block and motion information according to the mode. The mode information may be inter-prediction mode information indicating merge mode or AMVP mode. For example, the mode information may be a merge flag. That is, the merge mode or AMVP mode may be used to induce the additional prediction block, and a flag syntax element for indicating this may be signaled. Alternatively, a predefined mode of the merge mode or the AMVP mode may be used to induce the additional prediction block. Alternatively, either the merge mode or the AMVP mode may be selected based on predefined coding information.

[0150] As an example, when a merge mode is used to derive the additional prediction block, the prediction information may include a merge index. The merge index may identify a merge candidate in a merge candidate list. When an AMVP mode is used to derive the additional prediction block, the prediction information may include a motion vector predictor flag, a reference index, and motion vector difference information. The motion vector predictor flag may identify a candidate in a motion vector predictor candidate list.

[0151] The decoding apparatus may generate a final predicted block by performing a weighted sum of the base predicted block and the additional predicted block (S720). As described above, the number of additional predicted blocks may be equal to or less than a predefined number. For example, if the number of additional predicted blocks is two, the final predicted block may be a block obtained by performing a weighted sum of the base predicted block and the two additional predicted blocks.

[0152] As described above, when multiple additional predictive blocks are combined, the multiple additional predictive blocks may be weighted-summed sequentially to the base predictive block, or the base predictive block and the generated multiple additional predictive blocks may be weighted-summed collectively.

[0153] FIG. 8 is a flowchart illustrating an example of a syntax parsing order according to an embodiment of the present disclosure.

[0154] Referring to FIG. 8, the decoding apparatus may obtain (or parse) a weight index (S800). The weight index is information (or syntax element) indicating weight information used for weighted sum of bidirectional prediction blocks, and may be a bcw index or bcw_idx. The decoding apparatus may generate a base prediction block by performing a weighted sum of the L0 and L1 reference blocks based on the weight index. The weight index may be as described in FIG. 7, and the relevant overlapping description will be omitted.

[0155] The decoding apparatus may obtain MHP information (S810). The decoding apparatus may generate additional prediction blocks based on the obtained MHP information. The MHP information may be as described in FIG. 7, and the description will not be repeated.

[0156] The decoding apparatus may obtain base prediction information (S820). The decoding apparatus may generate a base prediction block based on the obtained base prediction information. The decoding apparatus may identify L0 and L1 reference blocks based on the obtained base prediction information, and may generate a base prediction block by performing a weighted sum of the L0 and L1 reference blocks based on a weight index. The base prediction information may include a motion vector predictor flag, a reference index, and motion vector difference information. The base prediction information may be as described in FIGS. 4 to 7, and related overlapping descriptions will be omitted.

[0157] The decoding apparatus may obtain adaptive motion vector resolution (AMVR) information (S830). In the present disclosure, AMVR may also be referred to as integer motion vector (imv). The motion vector derived based on the prediction information obtained in step S820 may be transformed by a shift operation based on a predetermined variable for adaptive motion vector resolution (hereinafter, referred to as AmvrShift). A motion vector may be derived based on the sum of the transformed motion vector and a motion vector differential.

[0158] The variable AmvrShift may be determined based on at least one of a flag related to an adaptive motion vector resolution (hereinafter, referred to as an AMVR flag) or an index related to an adaptive motion vector resolution (hereinafter, referred to as an AMVR index). The AMVR information may include an AMVR flag and / or an AMVR index. Here, the AMVR flag may indicate whether an adaptive motion vector resolution is applied to the current block. Alternatively, the AMVR flag may be defined to indicate whether a motion vector difference of the current block has a predefined default resolution (e.g., 1 / 4 sample unit). The AMVR index may indicate one of a plurality of predefined motion vector resolution candidates in the same manner for the encoding apparatus and the decoding apparatus. The plurality of motion vector resolution candidates may include at least two of 1 / 16 sample units, 1 / 8 sample units, 1 / 4 sample units, 1 / 2 sample units, 1 sample units, or 4 sample units. The maximum number of motion vector resolution candidates available to the current block may be 3, 4, 5, or 6. The AMVR flag and AMVR index may be obtained for a first direction.

[0159] As an example, the variable AmvrShift may be determined as shown in Table 1 below.

[0160] [Table 1]

[0161] In Table 1, amvr_flag corresponds to the flag related to the adaptive motion vector resolution described above and may indicate whether the adaptive motion vector resolution is applied to the current block. amvr_precision_idx corresponds to the index related to the adaptive motion vector resolution described above and may indicate one of a plurality of predefined motion vector resolution candidates.

[0162] According to Table 1, if amvr_flag is 0, adaptive motion vector resolution is not applied to the current block, and the variable AmvrShift is determined to be 2. This may mean that the corrected first motion vector and the motion vector difference of the current block are converted to have a default resolution (i.e., a resolution of 1 / 4 sample units).

[0163] On the other hand, an adaptive motion vector resolution may be applied to the current block if amvr_flag is 1. In this case, the variable AmvrShift may be determined to be 3, 4, or 6 based on amvr_precision_idx.

[0164] That is, when amvr_precision_idx is 0, the variable AmvrShift is determined as 3, and the compensated first motion vector and the motion vector difference of the current block may be converted to have a resolution in 1 / 2 sample units. The compensated first motion vector and the motion vector difference of the current block may be bit-shifted by 3 to the left to obtain a first motion vector and a motion vector difference having a resolution in 1 / 2 sample units. Similarly, when amvr_precision_idx is 1, the variable AmvrShift is determined as 4, and the compensated first motion vector and the motion vector difference of the current block may be converted to have a resolution in 1 sample units. The compensated first motion vector and the motion vector difference of the current block may be bit-shifted by 4 to the left to obtain a first motion vector and a motion vector difference having a resolution in 1 sample units. When amvr_precision_idx is 2, the variable AmvrShift is determined to be 6, and the corrected first motion vector and the motion vector difference of the current block may be converted to have a resolution of 4 sample units. The corrected first motion vector and the motion vector difference of the current block may be bit-shifted by 6 to the left to obtain the first motion vector and the motion vector difference having a resolution of 4 sample units, respectively.

[0165] At least one of the AMVR flag or the AMVR index may be signaled by the bitstream only if a certain condition is met, where the certain condition may include at least one of the conditions described below.

[0166] The AMVR flag and / or the AMVR index may be signaled by a bitstream based on a first flag indicating whether adaptive motion vector resolution is allowed for a video unit to which the current block belongs. The video unit may represent at least one of a sequence, a picture, or a slice. The first flag may be signaled in at least one of a sequence parameter set, a picture parameter set, a picture header, or a slice header of the bitstream.

[0167] As an example, when the first flag is a first value, the AMVR flag and / or the AMVR index may be signaled by the bitstream, and when the first flag is a second value, the AMVR flag and / or the AMVR index may not be signaled by the bitstream.

[0168] The decoding apparatus may obtain a Motion Vector Sign Derivation (MVSD) index (S840). According to an embodiment of the present disclosure, for inter-prediction motion vector differential information, instead of separately signaling horizontal (x-axis) and vertical (y-axis) sign information of each L0, L1 motion vector differential, the sign of each L0, L1 motion vector differential may be signaled using an index. MVSD represents a method of signaling the sign of each L0, L1 motion vector differential using an index, and MVSD index represents the index used here.

[0169] As an example, the MVSD index may indicate one of the x, y code combinations of (+,+), (+,-), (-,+), and (-,-). As another example, the x, y code combinations of (+,+), (+,-), (-,+), and (-,-) may be sorted (or re-sorted) based on template matching. In this case, the costs may be compared based on template matching and sorted in ascending order of cost. The MVSD index may indicate a particular candidate within the sorted code combination candidate group.

[0170] According to an embodiment of the present disclosure, the above-mentioned MHP mode may not be applied when one or more of the conditions in Table 2 below are satisfied, taking into consideration a trade-off between performance and complexity. In other words, MHP unavailability conditions such as those in Table 2 below may be defined.

[0171] [Table 2]

[0172] The MHP mode may not be applied if one or more of the conditions defined in Table 2 are met. The MHP mode may be disabled if one or more of the conditions defined in Table 2 are met, in which case the MHP flag and MHP information indicating whether the MHP mode is applied or not may not be parsed.

[0173] The MHP information may be parsed from the bitstream in the order shown in Figure 8 (i.e., step S810). However, when an MHP unavailability condition is defined as shown in Table 2 above, the MHP information may have a parsing dependency according to the condition.

[0174] Specifically, if the merge flag value is not 1, affine mode is applied, and the imv value is not 0 (i.e., the AMVR flag value is not 0), the MHP mode may not be available and may not be parsed from the bitstream. That is, if the AMVR information is defined as the MHP information parsing condition as shown in Table 2, even though the AMVR information is obtained in step S830 after step S810, a problem occurs in that the AMVR information is checked as the MHP information parsing condition to parse the MHP information in step S810, which is a step prior to step S830, resulting in a parsing dependency.

[0175] Therefore, to solve this problem, a new MHP information parsing condition may be defined taking into consideration the inter prediction related syntax parsing order as shown in Fig. 8. As an embodiment, syntax information acquired after step S810 may not be considered as an MHP information parsing condition for parsing the MHP information. For example, AMVR information may not be considered for parsing the MHP information. As an example, an MHP unavailable condition may be defined as shown in Table 3 below.

[0176] [Table 3]

[0177] Referring to Table 3, the condition where the merge flag value is not 1, the affine mode is applied, and the imv value is not 0 (i.e., the AMVR flag value is not 0) may not be considered. The MHP information may also be parsed when the merge flag value is not 1, the affine mode is applied, and the imv value is not 0 (i.e., the AMVR flag value is not 0).

[0178] In addition, as an embodiment, MVSD may not be applied when MHP is applied, and MVSD may be applied when MHP is not applied. In this case, MHP information may be referenced in the process of parsing MVD and MVD sign index, as shown in Table 4 below.

[0179] [Table 4]

[0180] Referring to Table 4, if MHP is not available and MVSD is available, MVSD may be used. In this case, there is a problem that dependency occurs between MHP and MVSD. Therefore, in order to remove the dependency between MHP and MVSD, as shown in Table 5 below, whether or not to apply can be determined by considering whether MVD and MVSD can be used regardless of MHP.

[0181] [Table 5]

[0182] Referring to Table 5, it is not necessary to consider whether MHP is available for parsing the MVSD index or MVD code. This improves parsing dependency and improves compression efficiency. Specifically, by removing the parsing dependency between MHP and MVSD, the degree of freedom of the parsing order of MHP information can be improved, and compression efficiency can be improved by changing the parsing order of AMVR information, MVSD index, and MHP prediction information, which will be described later. In addition, according to an embodiment of the present disclosure, the parsing order can be changed to solve the parsing dependency. As described above, the parsing dependency can be a problem between MHP information and AMVR information. In addition, when the MHP information does not exist, the MVD code is parsed, which can cause a parsing dependency problem. A method of changing the parsing order to solve such a problem will be described.

[0183] In one embodiment, the decoding device may first obtain (or parse) the weight index. Then, the decoding device may obtain basic prediction information. Then, the decoding device may obtain AMVR information. Then, the decoding device may obtain the MVSD index. Then, the decoding device may obtain MHP information. That is, in this embodiment, the parsing order in FIG. 8 may be changed to S800, S820, S830, S840, and S810. According to this embodiment, the AMVR information is obtained before parsing the MHP information, thereby improving the parsing dependency described above.

[0184] Also, in one embodiment, the decoding device may first obtain a weight index. Then, the decoding device may obtain basic prediction information. Then, the decoding device may obtain an MVSD index. Then, the decoding device may obtain AMVR information. Then, the decoding device may obtain MHP information. That is, in this embodiment, the parsing order in FIG. 8 may be changed to S800, S820, S840, S830, and S810. According to this embodiment, the AMVR information is obtained prior to parsing the MHP information, thereby improving the parsing dependency described above.

[0185] Also, in one embodiment, the decoding device may first obtain basic prediction information. Then, the decoding device may obtain AMVR information. Then, the decoding device may obtain an MVSD index. Then, the decoding device may obtain a weight index. Then, the decoding device may obtain MHP information. That is, in this embodiment, the parsing order in FIG. 8 may be changed to S820, S830, S840, S800, and S810. According to this embodiment, the AMVR information is obtained before parsing the MHP information, thereby improving the parsing dependency described above.

[0186] Also, in one embodiment, the decoding device may first obtain basic prediction information. Then, the decoding device may obtain an MVSD index. Then, the decoding device may obtain AMVR information. Then, the decoding device may obtain a weight index. Then, the decoding device may obtain MHP information. That is, in this embodiment, the parsing order in FIG. 8 may be changed to S820, S840, S830, S800, and S810. According to this embodiment, the AMVR information is obtained before parsing the MHP information, thereby improving the parsing dependency described above.

[0187] As described above, in order to resolve the parsing dependency, the parsing order of the MHP information can be changed to after the parsing of the AMVR information. Also, as described above, the parsing order can be changed so that the MHP information is not referenced when parsing the MVD and MVSD that refer to the MHP information. Also, as in Tables 2 and 3 above, when the MHP information is parsed only when the BCW index is not the default value, the MHP information is parsed after the parsing of the BCW index, but the order can be the same as in the above embodiment.

[0188] FIG. 9 is a diagram showing a schematic configuration of the inter prediction unit 332 that performs the inter prediction method according to the present disclosure.

[0189] 9, the inter prediction unit 332 may include a base prediction block generator 900, an additional prediction block generator 910, and a final prediction block generator 920.

[0190] The basic prediction block generator 900 may generate a basic prediction block by performing bidirectional prediction. When the MHP mode is applied, the basic prediction block generator 900 may generate and combine an additional prediction block in addition to a prediction block generated (or induced) by bidirectional prediction. As an example, the basic prediction block may include an L0 prediction block and / or an L1 prediction block. In the present disclosure, the basic prediction block may be referred to as an initial prediction block, a temporary prediction block, a reference prediction block, etc. Also, as an example, the basic prediction block may be a prediction block obtained by weighting the L0 and L1 prediction blocks.

[0191] In one embodiment, weights may be used for weighted sum of L0 and L1 prediction blocks. The weights are weights used in weighted prediction and may collectively refer to BCW (Bi-prediction with CU based Weights) and CW (CU based Weights). The weights may be derived from a weight candidate list. The weight candidate list may include a plurality of weight candidates and may be predefined in the encoding / decoding device.

[0192] The weight candidate may be a set of weights (i.e., a first weight and a second weight) indicating weights to be applied to each of the bidirectional predicted blocks, or may be weights to be applied to a predicted block in either one of the two directions. When only a weight to be applied to a predicted block in either one direction is derived from the weight candidate list, a weight to be applied to a predicted block in the other direction may be derived based on a weight derived from the weight candidate list. For example, a weight to be applied to a predicted block in the other direction may be derived by subtracting a weight derived from the weight candidate list from a predetermined value.

[0193] In one embodiment, a weight index indicating a weight used in weighted prediction of a current block may be induced in a weight candidate list. In the present disclosure, the weight index may be referred to as bcw_idx or bcw index. The weight index may be induced by a decoding device or may be signaled from an encoding device. When induced by a decoding device, the weight index may be induced as a weight index of a specific merge candidate in a merge candidate list. As an example, a specific merge candidate may be identified in a merge candidate list by a merge index.

[0194] The additional prediction block generator 910 may generate an additional prediction block based on the MHP mode. The additional prediction block generator 910 may generate an additional prediction block in addition to the basic prediction block, and may combine (or weighted-sum) the basic prediction block and the generated additional prediction block.

[0195] As an embodiment, the additional predicted block generator 910 may generate up to a predefined number of additional predicted blocks when the MHP mode is applied. In other words, the additional predicted block generator 910 may combine (or weighted-sum) up to a predefined number of additional predicted blocks with a base predicted block. As an example, the predefined number may be 2. Or, as an example, the predefined number may be one of 1, 2, 3, and 4. The predefined number may be referred to as an MHP maximum number.

[0196] In addition, when a plurality of additional prediction blocks are combined, the plurality of additional prediction blocks may be sequentially weighted-summed to the base prediction block. For example, when a maximum of two additional prediction blocks are generated, a prediction block may be generated by weighting the base prediction block and a first additional prediction block, and a final prediction block may be generated by weighting the generated prediction block and a second additional prediction block. The prediction block generated by weighting the base prediction block and the first additional prediction block may be referred to as an intermediate prediction block.

[0197] Alternatively, when a plurality of additional prediction blocks are combined, the base prediction block and the generated plurality of additional prediction blocks may be collectively weighted-summed. That is, after the plurality of additional prediction blocks are generated, a weight may be applied to each of the plurality of additional prediction blocks and the base prediction block (or the L0 prediction block and the L1 prediction block), and the weighted-sum may be collectively performed.

[0198] Also, in one embodiment, the additional prediction block generator 910 may determine whether or not to apply MHP. For example, whether or not to apply MHP may be explicitly signaled or may be implicitly induced by a decoding device.

[0199] In addition, in one embodiment, whether or not MHP is applied may be signaled from the encoding apparatus to the decoding apparatus. For example, an MHP flag indicating whether or not MHP is applied may be signaled from the encoding apparatus to the decoding apparatus. In this case, a condition for signaling / parsing the MHP flag may be predefined. The signaling / parsing condition of the MHP flag may be an MHP availability condition. If the signaling / parsing condition is satisfied, the decoding apparatus may parse the MHP flag from the bitstream. Alternatively, in one embodiment, whether or not MHP is applied may be induced by the decoding apparatus based on predefined coding information. For example, whether or not MHP is applied may be defined to be the same as an MHP availability condition (or a signaling / parsing condition) described below.

[0200] Also, as an embodiment, the additional predicted block generator 910 may obtain MHP information (which may also be referred to as MHP prediction information) to generate an additional predicted block. As an example, the MHP information may include weight information and / or prediction information. Based on the prediction information, a reference block according to an MHP mode, i.e., an additional predicted block, may be derived, and the additional predicted block derived based on the weight information may be weighted and summed with a base predicted block (or an intermediate predicted block). Also, as an example, the MHP information may further include an MHP flag indicating whether or not MHP is applied.

[0201] As an example, the prediction information may include mode information and mode-based motion information used to induce the additional prediction block. The mode information may be inter-prediction mode information indicating merge mode or AMVP mode. For example, the mode information may be a merge flag. That is, the merge mode or AMVP mode may be used to induce the additional prediction block, and a flag syntax element for indicating this may be signaled. Alternatively, a predefined mode of the merge mode or the AMVP mode may be used to induce the additional prediction block. Alternatively, either the merge mode or the AMVP mode may be selected based on predefined coding information.

[0202] As an example, when a merge mode is used to derive the additional prediction block, the prediction information may include a merge index. The merge index may identify a merge candidate in a merge candidate list. When an AMVP mode is used to derive the additional prediction block, the prediction information may include a motion vector predictor flag, a reference index, and motion vector difference information. The motion vector predictor flag may identify a candidate in a motion vector predictor candidate list.

[0203] The final predicted block generator 920 may generate a final predicted block by performing a weighted sum of the base predicted block and the additional predicted block. As described above, the number of additional predicted blocks may be equal to or less than a predefined number. For example, if the number of additional predicted blocks is two, the final predicted block may be a block obtained by performing a weighted sum of the base predicted block and two additional predicted blocks.

[0204] As described above, when multiple additional predictive blocks are combined, the multiple additional predictive blocks may be weighted-summed sequentially to the base predictive block, or the base predictive block and the multiple generated additional predictive blocks may be weighted-summed collectively.

[0205] Meanwhile, in one embodiment, syntax parsing orders associated with various inter prediction modes including the MHP mode may be defined or changed. In this case, the method described in FIG. 8 may be applied, and therefore, a duplicated description will be omitted here.

[0206] Also, as an embodiment, a MHP information parsing condition may be newly defined in consideration of the inter prediction related syntax parsing order as shown in Fig. 8. For example, AMVR information may not be considered for parsing MHP information. In this case, the methods described in Fig. 8 and Tables 2 and 3 may be applied, and redundant description will be omitted here.

[0207] In addition, as an embodiment, the application of MVD and MVSD may be determined by considering whether they can be used regardless of MHP. In this case, the method described in FIG. 8, Table 4, and Table 5 may be applied, and the duplicated description will be omitted here.

[0208] Also, as an embodiment, a parsing dependency may be an issue between MHP information and AMVR information as described above. Also, a parsing dependency may be an issue when the MVD code is parsed in the absence of MHP information. To solve this problem, the parsing order may be changed in various ways. In this case, the method described in FIG. 8 may be applied, and a duplicated description will be omitted here.

[0209] FIG. 10 is a diagram illustrating an inter prediction method performed by the encoding device 200 according to an embodiment of the present disclosure.

[0210] Referring to FIG. 10, an encoding apparatus may perform bidirectional prediction to generate a base prediction block (S1000). When the MHP mode is applied, the encoding apparatus may generate and combine an additional prediction block in addition to the prediction block generated (or derived) by bidirectional prediction. As an example, the base prediction block may include an L0 prediction block and / or an L1 prediction block. In the present disclosure, the base prediction block may be referred to as an initial prediction block, a temporary prediction block, a reference prediction block, etc. Also, as an example, the base prediction block may be a prediction block obtained by weighting the L0 and L1 prediction blocks.

[0211] In one embodiment, weights may be used for weighted sum of L0 and L1 prediction blocks. The weights are weights used in weighted prediction and may collectively refer to BCW (Bi-prediction with CU based Weights) and CW (CU based Weights). The weights may be derived from a weight candidate list. The weight candidate list may include a plurality of weight candidates and may be predefined in the encoding / decoding device.

[0212] The weight candidate may be a set of weights (i.e., a first weight and a second weight) indicating weights to be applied to each of the bidirectional predicted blocks, or may be weights to be applied to a predicted block in either one of the two directions. When only a weight to be applied to a predicted block in either one direction is derived from the weight candidate list, a weight to be applied to a predicted block in the other direction may be derived based on a weight derived from the weight candidate list. For example, a weight to be applied to a predicted block in the other direction may be derived by subtracting a weight derived from the weight candidate list from a predetermined value.

[0213] In one embodiment, a weight index indicating a weight used in weighted prediction of a current block may be induced in a weight candidate list. In the present disclosure, the weight index may be referred to as bcw_idx or bcw index. The weight index may be induced by a decoding apparatus or may be signaled from an encoding apparatus. When induced by a decoding apparatus, the weight index may be induced as a weight index of a specific merge candidate in a merge candidate list. As an example, a specific merge candidate may be identified in a merge candidate list by a merge index.

[0214] The encoding apparatus may generate an additional prediction block based on the MHP mode (S1010). The encoding apparatus may generate the additional prediction block in addition to the base prediction block, and may combine (or weighted-sum) the base prediction block and the generated additional prediction block.

[0215] In one embodiment, when the MHP mode is applied, the encoding apparatus may generate up to a predefined number of additional prediction blocks. In other words, the encoding apparatus may combine (or weighted-sum) up to a predefined number of additional prediction blocks with a base prediction block. For example, the predefined number may be 2. Or, for example, the predefined number may be one of 1, 2, 3, and 4. The predefined number may be referred to as an MHP maximum number.

[0216] In addition, when a plurality of additional prediction blocks are combined, the plurality of additional prediction blocks may be sequentially weighted-summed to the base prediction block. For example, when a maximum of two additional prediction blocks are generated, a prediction block may be generated by weighting the base prediction block and a first additional prediction block, and a final prediction block may be generated by weighting the generated prediction block and a second additional prediction block. The prediction block generated by weighting the base prediction block and the first additional prediction block may be referred to as an intermediate prediction block.

[0217] Alternatively, when a plurality of additional prediction blocks are combined, the base prediction block and the generated plurality of additional prediction blocks may be collectively weighted-summed. That is, after the plurality of additional prediction blocks are generated, a weight may be applied to each of the plurality of additional prediction blocks and the base prediction block (or the L0 prediction block and the L1 prediction block), and the weighted-sum may be collectively performed.

[0218] Also, in one embodiment, the encoding device may determine whether or not MHP is applied. In this case, a step of determining whether or not MHP is applied may be added prior to step S1010. For example, whether or not MHP is applied may be explicitly signaled or may be implicitly induced by the decoding device.

[0219] In addition, in one embodiment, the application of MHP may be signaled from the encoding apparatus to the decoding apparatus. For example, an MHP flag indicating the application of MHP may be signaled from the encoding apparatus to the decoding apparatus. In this case, a condition for signaling / parsing the MHP flag may be predefined. The signaling / parsing condition of the MHP flag may be an MHP availability condition. If the signaling / parsing condition is satisfied, the encoding apparatus may signal the MHP flag in the bitstream. Alternatively, in one embodiment, the application of MHP may be induced by the decoding apparatus based on predefined coding information. For example, the application of MHP may be defined to be the same as an MHP availability condition (or a signaling / parsing condition) described below.

[0220] Also, as an embodiment, the encoding apparatus may obtain MHP information (which may also be referred to as MHP prediction information) to generate an additional prediction block. As an example, the MHP information may include weight information and / or prediction information. Based on the prediction information, a reference block according to an MHP mode, i.e., an additional prediction block, may be derived, and the additional prediction block derived based on the weight information may be weighted-summed with a base prediction block (or an intermediate prediction block). Also, as an example, the MHP information may further include an MHP flag indicating whether or not MHP is applied.

[0221] As an example, the prediction information may include mode information and mode-based motion information used to induce the additional prediction block. The mode information may be inter-prediction mode information indicating merge mode or AMVP mode. For example, the mode information may be a merge flag. That is, the merge mode or AMVP mode may be used to induce the additional prediction block, and a flag syntax element for indicating this may be signaled. Alternatively, a predefined mode of the merge mode or the AMVP mode may be used to induce the additional prediction block. Alternatively, either the merge mode or the AMVP mode may be selected based on predefined coding information.

[0222] As an example, when a merge mode is used to derive the additional prediction block, the prediction information may include a merge index. The merge index may identify a merge candidate in a merge candidate list. When an AMVP mode is used to derive the additional prediction block, the prediction information may include a motion vector predictor flag, a reference index, and motion vector difference information. The motion vector predictor flag may identify a candidate in a motion vector predictor candidate list.

[0223] The encoding apparatus may generate a final predicted block by performing a weighted sum of the base predicted block and the additional predicted block. As described above, the number of the additional predicted blocks may be equal to or less than a predefined number. For example, if the number of the additional predicted blocks is two, the final predicted block may be a block obtained by performing a weighted sum of the base predicted block and two additional predicted blocks.

[0224] As described above, when multiple additional predictive blocks are combined, the multiple additional predictive blocks may be weighted-summed sequentially to the base predictive block, or the base predictive block and the generated multiple additional predictive blocks may be weighted-summed collectively.

[0225] Meanwhile, in one embodiment, syntax parsing orders associated with various inter prediction modes including the MHP mode may be defined or changed. In this case, the method described in FIG. 8 may be applied, and therefore, a duplicated description will be omitted here.

[0226] Also, as an embodiment, a MHP information parsing condition may be newly defined in consideration of the inter prediction related syntax parsing order as shown in Fig. 8. For example, AMVR information may not be considered for parsing MHP information. In this case, the methods described in Fig. 8 and Tables 2 and 3 may be applied, and redundant description will be omitted here.

[0227] In addition, as an embodiment, the application of MVD and MVSD may be determined by considering whether they can be used regardless of MHP. In this case, the method described in FIG. 8, Table 4, and Table 5 may be applied, and the duplicated description will be omitted here.

[0228] Also, as an embodiment, a parsing dependency may be an issue between MHP information and AMVR information as described above. Also, a parsing dependency may be an issue when the MVD code is parsed in the absence of MHP information. To solve this problem, the parsing order may be changed in various ways. In this case, the method described in FIG. 8 may be applied, and a duplicated description will be omitted here.

[0229] FIG. 11 is a diagram showing a schematic configuration of the inter prediction unit 221 that performs the inter prediction method according to the present disclosure.

[0230] Referring to FIG. 11, the inter prediction unit 221 may include a base prediction block generation unit 1100, an additional prediction block generation unit 1110, and a final prediction block generation unit 1120.

[0231] The basic prediction block generator 1100 may generate a basic prediction block by performing bidirectional prediction. When the MHP mode is applied, the basic prediction block generator 1100 may generate and combine an additional prediction block in addition to a prediction block generated (or induced) by bidirectional prediction. As an example, the basic prediction block may include an L0 prediction block and / or an L1 prediction block. In the present disclosure, the basic prediction block may be referred to as an initial prediction block, a temporary prediction block, a reference prediction block, etc. Also, as an example, the basic prediction block may be a prediction block obtained by weighting the L0 and L1 prediction blocks.

[0232] In one embodiment, weights may be used for weighted sum of L0 and L1 prediction blocks. The weights are weights used in weighted prediction and may collectively refer to BCW (Bi-prediction with CU based Weights) and CW (CU based Weights). The weights may be derived from a weight candidate list. The weight candidate list may include a plurality of weight candidates and may be predefined in the encoding / decoding device.

[0233] The weight candidate may be a set of weights (i.e., a first weight and a second weight) indicating weights to be applied to each of the bidirectional predicted blocks, or may be weights to be applied to a predicted block in either one of the two directions. When only a weight to be applied to a predicted block in either one direction is derived from the weight candidate list, a weight to be applied to a predicted block in the other direction may be derived based on a weight derived from the weight candidate list. For example, a weight to be applied to a predicted block in the other direction may be derived by subtracting a weight derived from the weight candidate list from a predetermined value.

[0234] In one embodiment, a weight index indicating a weight used in weighted prediction of a current block may be induced in a weight candidate list. In the present disclosure, the weight index may be referred to as bcw_idx or bcw index. The weight index may be induced by a decoding device or may be signaled from an encoding device. When induced by a decoding device, the weight index may be induced as a weight index of a specific merge candidate in a merge candidate list. As an example, a specific merge candidate may be identified in a merge candidate list by a merge index.

[0235] The additional prediction block generator 1110 may generate an additional prediction block based on the MHP mode. The additional prediction block generator 1110 may generate an additional prediction block in addition to the basic prediction block, and may combine (or weighted-sum) the basic prediction block and the generated additional prediction block.

[0236] As an embodiment, the additional predicted block generator 1110 may generate up to a predefined number of additional predicted blocks when the MHP mode is applied. In other words, the additional predicted block generator 1110 may combine (or weighted-sum) up to a predefined number of additional predicted blocks with a base predicted block. As an example, the predefined number may be 2. Or, as an example, the predefined number may be one of 1, 2, 3, and 4. The predefined number may be referred to as an MHP maximum number.

[0237] In addition, when a plurality of additional prediction blocks are combined, the plurality of additional prediction blocks may be sequentially weighted-summed to the base prediction block. For example, when a maximum of two additional prediction blocks are generated, a prediction block may be generated by weighting the base prediction block and a first additional prediction block, and a final prediction block may be generated by weighting the generated prediction block and a second additional prediction block. The prediction block generated by weighting the base prediction block and the first additional prediction block may be referred to as an intermediate prediction block.

[0238] Alternatively, when a plurality of additional prediction blocks are combined, the base prediction block and the generated plurality of additional prediction blocks may be collectively weighted-summed. That is, after the plurality of additional prediction blocks are generated, a weight may be applied to each of the plurality of additional prediction blocks and the base prediction block (or the L0 prediction block and the L1 prediction block), and the weighted-sum may be collectively performed.

[0239] Also, in one embodiment, the additional prediction block generator 910 may determine whether or not to apply MHP. For example, whether or not to apply MHP may be explicitly signaled or may be implicitly induced by a decoding device.

[0240] In addition, in one embodiment, whether or not MHP is applied may be signaled from the encoding apparatus to the decoding apparatus. For example, an MHP flag indicating whether or not MHP is applied may be signaled from the encoding apparatus to the decoding apparatus. In this case, a condition for signaling / parsing the MHP flag may be predefined. The signaling / parsing condition of the MHP flag may be an MHP availability condition. If the signaling / parsing condition is satisfied, the decoding apparatus may parse the MHP flag from the bitstream. Alternatively, in one embodiment, whether or not MHP is applied may be induced by the decoding apparatus based on predefined coding information. For example, whether or not MHP is applied may be defined to be the same as an MHP availability condition (or a signaling / parsing condition) described below.

[0241] Also, as an embodiment, the additional predicted block generator 1110 may obtain MHP information (which may also be referred to as MHP prediction information) to generate an additional predicted block. As an example, the MHP information may include weight information and / or prediction information. Based on the prediction information, a reference block according to an MHP mode, i.e., an additional predicted block, may be derived, and the additional predicted block derived based on the weight information may be weighted and summed with a base predicted block (or an intermediate predicted block). As an example, the MHP information may further include an MHP flag indicating whether or not MHP is applied.

[0242] As an example, the prediction information may include mode information and mode-based motion information used to induce the additional prediction block. The mode information may be inter-prediction mode information indicating merge mode or AMVP mode. For example, the mode information may be a merge flag. That is, the merge mode or AMVP mode may be used to induce the additional prediction block, and a flag syntax element for indicating this may be signaled. Alternatively, a predefined mode of the merge mode or the AMVP mode may be used to induce the additional prediction block. Alternatively, either the merge mode or the AMVP mode may be selected based on predefined coding information.

[0243] As an example, when a merge mode is used to derive the additional prediction block, the prediction information may include a merge index. The merge index may identify a merge candidate in a merge candidate list. When an AMVP mode is used to derive the additional prediction block, the prediction information may include a motion vector predictor flag, a reference index, and motion vector difference information. The motion vector predictor flag may identify a candidate in a motion vector predictor candidate list.

[0244] The final predicted block generator 1120 may generate a final predicted block by performing a weighted sum of the base predicted block and the additional predicted block. As described above, the number of additional predicted blocks may be equal to or less than a predefined number. For example, if the number of additional predicted blocks is two, the final predicted block may be a block obtained by performing a weighted sum of the base predicted block and two additional predicted blocks.

[0245] As described above, when multiple additional predictive blocks are combined, the multiple additional predictive blocks may be weighted-summed sequentially to the base predictive block, or the base predictive block and the multiple generated additional predictive blocks may be weighted-summed collectively.

[0246] Meanwhile, in one embodiment, syntax parsing orders associated with various inter prediction modes including the MHP mode may be defined or changed. In this case, the method described in FIG. 8 may be applied, and therefore, a duplicated description will be omitted here.

[0247] Also, as an embodiment, a MHP information parsing condition may be newly defined in consideration of the inter prediction related syntax parsing order as shown in Fig. 8. For example, AMVR information may not be considered for parsing MHP information. In this case, the methods described in Fig. 8 and Tables 2 and 3 may be applied, and redundant description will be omitted here.

[0248] In addition, as an embodiment, the application of MVD and MVSD may be determined by considering whether they can be used regardless of MHP. In this case, the method described in FIG. 8, Table 4, and Table 5 may be applied, and the duplicated description will be omitted here.

[0249] Also, as an embodiment, a parsing dependency may be an issue between MHP information and AMVR information as described above. Also, a parsing dependency may be an issue when the MVD code is parsed in the absence of MHP information. To solve this problem, the parsing order may be changed in various ways. In this case, the method described in FIG. 8 may be applied, and a duplicated description will be omitted here.

[0250] In the above-described embodiments, the method is described based on a flowchart as a series of steps or blocks, but the embodiments are not limited to the order of steps, and certain steps may occur in different steps and in different orders or simultaneously than those described above. Furthermore, those skilled in the art will appreciate that the steps shown in the flowcharts are not exclusive, and other steps may be included or one or more steps of the flowcharts may be omitted without affecting the scope of the embodiments of this document.

[0251] The methods according to the embodiments of the present document described above may be embodied in software form, and the encoding device and / or decoding device according to the present document may be included in a device that performs image processing, such as a TV, a computer, a smartphone, a set-top box, or a display device.

[0252] When the embodiments in this document are embodied as software, the methods described above may be embodied as modules (steps, functions, etc.) performing the functions described above. The modules may be stored in a memory and executed by a processor. The memory may be internal or external to the processor and may be coupled to the processor by various means known in the art. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. That is, the embodiments described in this document may be embodied and performed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure may be embodied and performed on a computer, processor, microprocessor, controller, or chip. In this case, information (e.g., information on instructions) or algorithms for the implementation may be stored in a digital storage medium.

[0253] In addition, the decoding device and encoding device to which the embodiments of the present specification are applied may be included in a multimedia broadcast transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a custom video (VoD) service providing device, an over-the-top video (OTT) device, an internet streaming service providing device, a three-dimensional (3D) video device, a virtual reality (VR) device, an argument reality (AR) device, an image telephone video device, a transportation means terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), and a medical video device, and may be used to process a video signal or a data signal. For example, an over-the-top video (OTT) device may include a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), etc.

[0254] In addition, a processing method to which the embodiments of the present specification are applied may be produced in the form of a program executed by a computer and may be stored in a computer-readable recording medium. Multimedia data having a data structure according to the embodiments of the present specification may also be stored in a computer-readable recording medium. The computer-readable recording medium includes any kind of storage device and distributed storage device in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium may also include a medium embodied in the form of a carrier wave (e.g., transmission via the Internet). In addition, a bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network.

[0255] Furthermore, the embodiments of the present specification may be embodied as a computer program product by program code, the program code being executable by a computer according to the embodiments of the present specification. The program code may be stored on a computer readable carrier.

[0256] FIG. 12 illustrates an example of a content streaming system to which the embodiments of the present disclosure can be applied.

[0257] Referring to FIG. 12, a content streaming system to which the embodiments of the present specification are applied may broadly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0258] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, when a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0259] The bitstream may be generated by an encoding method or a bitstream generation method to which the embodiments of this specification are applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0260] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server plays a role of controlling commands / responses between the devices in the content streaming system.

[0261] The streaming server may receive the content from a media storage and / or an encoding server. For example, when receiving the content from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a certain period of time in order to provide a smooth streaming service.

[0262] Examples of the user devices include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glass, head mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0263] Each server in the content streaming system may be operated as a distributed server, in which case data received by each server may be processed in a distributed manner.

[0264] The claims described herein may be combined in various ways. For example, the technical features of the method claims herein may be combined and embodied as an apparatus, the technical features of the apparatus claims herein may be combined and embodied as a method, the technical features of the method claims herein and the technical features of the apparatus claims herein may be combined and embodied as an apparatus, and the technical features of the method claims herein and the technical features of the apparatus claims herein may be combined and embodied as a method.

Claims

1. generating a base prediction block of a current block by performing bidirectional prediction based on a first inter prediction mode; generating an additional prediction block for the current block based on a second inter prediction mode; generating a final predicted block of the current block by performing a weighted sum of the base predicted block and the additional predicted block.

2. The method of claim 1, wherein the first inter prediction mode includes at least one of a merge mode, an AMVP mode, or a combined merge-AMVP mode.

3. The method of claim 2 , wherein the second inter prediction mode comprises a multiple hypothesis prediction mode.

4. The image decoding method of claim 1 , wherein, when a plurality of additional prediction blocks are generated, the final prediction block is generated by sequentially performing a weighted sum of the plurality of additional prediction blocks and the base prediction block.

5. The method of claim 1 , further comprising obtaining information about the second inter-prediction mode from a bitstream.

6. The information about the second inter prediction mode includes at least one of weight information or prediction information, The weight information indicates a weight used in a weighted sum of the additional predicted block, The image decoding method of claim 5 , wherein the prediction information represents information for deriving the additional prediction block.

7. The video decoding method of claim 6 , wherein the prediction information includes a mode flag indicating whether a merge mode or an AMVP mode is used to derive the additional predicted block.

8. When the mode flag indicates that a merge mode is used to derive the additional prediction block, the prediction information further includes a merge index; 8. The video decoding method of claim 7, wherein when the mode flag indicates that an AMVP mode is used to derive the additional prediction block, the prediction information includes at least one of a motion vector predictor flag, a reference index, or motion vector differential information.

9. The information about the second inter prediction mode is obtained from a bitstream when a predefined condition is satisfied, The image decoding method of claim 5, wherein the predefined condition is defined without taking into account adaptive motion vector resolution (AMVR) information.

10. The method further includes obtaining Adaptive Motion Vector Resolution (AMVR) information from the bitstream; The image decoding method of claim 5 , wherein the obtaining of the AMVR information is performed before the obtaining of information about the second inter prediction mode.

11. The method further includes obtaining a motion vector sign derivation (MVSD) index from the bitstream, the motion vector sign derivation index identifying a candidate from among a plurality of candidate code combinations; The method of claim 5 , wherein the obtaining of the MVSD index is performed before the obtaining of information about the second inter prediction mode.

12. generating a base prediction block of a current block by performing bidirectional prediction based on a first inter prediction mode; generating an additional prediction block for the current block based on a second inter prediction mode; generating a final predicted block of the current block by performing a weighted sum of the base predicted block and the additional predicted block.

13. 13. A computer-readable storage medium storing a bitstream generated by the video encoding method of claim 12.

14. generating a base prediction block of a current block by performing bidirectional prediction based on a first inter prediction mode; generating an additional prediction block for the current block based on a second inter prediction mode; generating a final predicted block of the current block by performing a weighted sum of the base predicted block and the additional predicted block; generating a bitstream by encoding the current block based on the final predicted block; transmitting data including the bitstream.

Images