Encoding method, decoding method, computer-readable recording medium, and transmission method

Abstract

A decoding method according to one aspect of the present disclosure comprises: obtaining information on a conventional prediction mode; generating a regular prediction block based on the information on the regular prediction mode; obtaining information on an additional prediction mode; generating an additional prediction block based on the information on the additional prediction mode; and generating a final prediction block based on a combination of the regular prediction block and an additional prediction block, where the additional prediction mode may include at least one of an intra prediction mode and an intra block copy (IBC) prediction mode, and the information on the additional prediction mode may include at least one of information indicating whether to apply the intra prediction mode to the generation of the additional prediction block and information indicating whether to apply the IBC prediction mode to the generation of the additional prediction block.

Description

TECHNICAL FIELD

[0001] The disclosure relates to an encoding / decoding method of image information, a computer-readable recording medium storing image information, and a method of transmitting image information. BACKGROUND

[0002] Recently, the demand for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images has been increasing in various application fields, and thus, efficient image compression techniques are being discussed.

[0003] There are various techniques such as an inter prediction technique of predicting pixel values included in a current picture from pictures before or after the current picture using a video compression technique, an intra prediction technique of predicting pixel values included in a current picture by using pixel information in the current picture, an entropy encoding technique of assigning a short symbol to a value having a high frequency of occurrence and assigning a long symbol to a value having a low frequency of occurrence, etc., and these image compression techniques can be used to efficiently compress and transmit or store image data.

[0004] Therefore, there is a need for an efficient image compression technique for efficiently transmitting, storing, and reproducing information of high-resolution and high-quality images. SUMMARY

[0005] TECHNICAL PROBLEM

[0006] The disclosure provides a method and apparatus for performing inter prediction based on a multi-prediction block mode.

[0007] The disclosure provides a method and apparatus for signaling / parsing information about a multi-prediction block mode.

[0008] The disclosure provides a method and apparatus for deriving information about a multi-prediction block mode.

[0009] TECHNICAL SOLUTION

[0010] A decoding method according to an aspect of the disclosure includes obtaining information about a regular prediction mode, generating a regular prediction block based on the information about the regular prediction mode, obtaining information about an additional prediction mode, generating an additional prediction block based on the information about the additional prediction mode, and generating a final prediction block based on a combination of the regular prediction block and the additional prediction block, wherein the additional prediction mode includes at least one of an intra prediction mode or an IBC (Intra Block Copy) prediction mode, and the information about the additional prediction mode can include at least one of information indicating whether to apply the intra prediction mode to the generation of the additional prediction block or information indicating whether to apply the IBC prediction mode to the generation of the additional prediction block.

[0011] A decoding method according to an aspect of the disclosure includes deriving information about prediction from a neighbor block based on a prediction mode for a current block being determined as a merge mode, and generating a prediction block for the current block based on the information about prediction derived from the neighbor block, wherein the generating the prediction block for the current block can include including information about a multi-prediction block mode in the information about prediction derived from the neighbor block, and applying the multi-prediction block mode to the current block.

[0012] An encoding method according to an aspect of the disclosure includes generating information indicating whether a multi-prediction block mode is applied to a current block, generating information about a regular prediction mode for generating a regular prediction block and information about an additional prediction mode for generating an additional prediction block based on the multi-prediction block mode being applied, and encoding the information indicating whether the multi-prediction block mode is applied, the information about the regular prediction mode, and the information about the additional prediction mode, wherein the additional prediction mode includes at least one of an intra prediction mode or an IBC prediction mode, and the information about the additional prediction mode can include at least one of information indicating whether the intra prediction mode is applied to the generation of the additional prediction block or information indicating whether the IBC prediction mode is applied to the generation of the additional prediction block.

[0013] An encoding method according to an aspect of the disclosure includes generating information indicating that a merge mode is applied to a current block and generating a merge candidate list based on a prediction mode for the current block being determined as the merge mode, selecting one candidate among candidates included in the merge candidate list and generating information indicating the selected candidate, and encoding the information indicating that the merge mode is applied and the information indicating the selected candidate, wherein the selected candidate corresponds to a neighbor block to which a multi-prediction block mode is applied, and information about the multi-prediction block mode applied to the selected candidate can be derived as information about prediction of the current block.

[0014] A recording medium according to an aspect of the disclosure is a computer-readable recording medium storing a bitstream, wherein the bitstream includes information indicating whether a multi-prediction block mode is applied to a current block, information about a regular prediction mode for generating a regular prediction block, and information about an additional prediction mode for generating an additional prediction block, wherein the additional prediction mode includes at least one of an intra prediction mode or an IBC prediction mode, and the information about the additional prediction mode can include at least one of information indicating whether the intra prediction mode is applied to the generation of the additional prediction block or information indicating whether the IBC prediction mode is applied to the generation of the additional prediction block.

[0015] According to one aspect of this disclosure, the recording medium is a computer-readable recording medium storing a bit stream, wherein the bit stream includes: information indicating the application of a merge mode to the current block; information indicating the selection of a candidate from among candidates included in a merge candidate list; and information regarding a prediction of the selected candidate, wherein the selected candidate corresponds to a neighboring block to which a multi-prediction block mode is applied, and the information regarding the multi-prediction block mode applied to the selected candidate can be derived as information regarding a prediction of the current block.

[0016] According to one aspect of the present disclosure, a method for transmitting image information includes: generating image information comprising a bitstream, wherein the bitstream includes information indicating whether a multi-prediction block mode is applied to the current block, information about a regular prediction mode for generating a regular prediction block, and information about an additional prediction mode for generating an additional prediction block; and transmitting the image information comprising the bitstream, wherein the additional prediction mode includes at least one of an intra-prediction mode or an IBC prediction mode, and the information about the additional prediction mode may include at least one of information indicating whether an intra-prediction mode is applied to the generation of an additional prediction block or information indicating whether an IBC prediction mode is applied to the generation of an additional prediction block.

[0017] According to one aspect of this disclosure, a method for transmitting image information includes: generating image information comprising a bitstream, wherein the bitstream includes information indicating whether a merging mode is applied to the current block, information indicating a candidate selected from candidates included in a merging candidate list, and information about a prediction of the selected candidate; and transmitting the image information comprising the bitstream, wherein the selected candidate corresponds to a neighboring block to which a multi-prediction block mode is applied, and the information about the multi-prediction block mode applied to the selected candidate can be derived as information about a prediction of the current block.

[0018] Beneficial effects

[0019] According to this disclosure, a method and apparatus for performing inter-frame prediction based on a multi-prediction block pattern can be provided.

[0020] According to this disclosure, methods and apparatus can be provided for signaling / resolving information about multi-prediction block patterns.

[0021] According to this disclosure, methods and apparatus for deriving information about multi-prediction block patterns can be provided.

[0022] The effects that can be obtained from this disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. Attached Figure Description

[0023] FIG. 1 An example of a video / image coding system according to this disclosure.

[0024] FIG. 2 A schematic block diagram illustrating an encoding apparatus that can apply embodiments of the present disclosure and perform encoding of video / image signals is shown.

[0025] FIG. 3 A schematic block diagram illustrating a decoding apparatus that can apply embodiments of the present disclosure and perform decoding of video / image signals is shown.

[0026] FIG. 4 Examples of video / image decoding methods to which embodiments of the present disclosure may be applied are provided.

[0027] FIG. 5 Examples of video / image coding methods to which embodiments of the present disclosure may be applied are provided.

[0028] FIG. 6 Example of a reference sample (R) defined in PDPC based on various prediction models. x,-1 R -1,y R -1,-1 ).

[0029] FIG. 7 An example of how intra-frame sub-partitions are split.

[0030] FIG. 8 A diagram illustrating the matrix-based intra-frame prediction process is shown.

[0031] FIG. 9 and FIG. 10 Examples of video / image coding methods based on inter-frame prediction that can be applied to embodiments of this disclosure are provided.

[0032] FIG. 11 and FIG. 12 Examples of video / image decoding methods based on inter-frame prediction that can be applied to embodiments of this disclosure are provided.

[0033] FIG. 13 An example of an inter-frame prediction process that can be applied to embodiments of this disclosure is shown.

[0034] FIG. 14 An example of a block used to configure a list of candidates to be merged.

[0035] FIG. 15 Examples of four types of motion that can be represented in an affine motion model are shown.

[0036] FIG. 16 An example of a control point motion vector used in affine motion prediction is shown.

[0037] FIG. 17An example of the candidate positions in the configured affine merge pattern.

[0038] FIG. 18 An example of a neighboring block of the current block.

[0039] FIG. 19 Example of GPM splitting grouped by the same angle.

[0040] FIG. 20 An example of the top-left neighbor block used in CIIP weight derivation.

[0041] FIG. 21 Examples of available IPM candidates include inter-frame and intra-frame predictions of the GPM.

[0042] FIG. 22 Example of a sub-block and sub-PU at the CU / PU boundary in ATMVP mode.

[0043] FIG. 23 An example of an extended CU region used in BDOF.

[0044] FIG. 24 An inter-frame prediction method performed by a decoding device according to an embodiment of the present disclosure is illustrated, and FIG. 25 Examples of prediction blocks used in a multi-prediction block mode according to embodiments of the present disclosure are illustrated.

[0045] FIG. 26 This is a flowchart illustrating an example of an operation for obtaining multi-prediction block mode information in a decoding method performed by a decoding device according to an embodiment.

[0046] FIG. 27 This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information in a decoding method performed by a decoding device according to an embodiment.

[0047] FIG. 28 This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information in a decoding method performed by a decoding device according to an embodiment.

[0048] FIG. 29 This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information in a decoding method performed by a decoding device according to an embodiment.

[0049] FIG. 30 This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information in a decoding method performed by a decoding device according to an embodiment.

[0050] FIG. 31This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information in a decoding method performed by a decoding device according to an embodiment.

[0051] FIG. 32 An example is shown of performing inheritance from a neighboring block that applies multi-prediction block mode while excluding MHP-intra-frame mode.

[0052] FIG. 33 An example is shown where the number of intra-frame modes inherited from a neighboring block that applies a multi-prediction block mode is limited to one.

[0053] FIG. 34 This is a flowchart illustrating an example of deriving information about MHP-intra-frame modes on the decoder side in a decoding method performed by a decoding device according to an embodiment.

[0054] FIG. 35 This is a flowchart illustrating another example of deriving information about MHP-intra-frame modes on the decoder side in a decoding method performed by a decoding device according to an embodiment.

[0055] FIG. 36 This is an example diagram illustrating the error between neighboring samples of the current block and neighboring samples of a regular prediction block.

[0056] FIG. 37 This is a flowchart illustrating an example of the operation of signaling / parsing information about MHP-intra-frame mode.

[0057] FIG. 38 This is a flowchart illustrating another example of the operation of signaling / parsing information about MHP-intra-frame modes.

[0058] FIG. 39 This is a flowchart illustrating yet another example of the operation of signaling / parsing information about MHP-intra-frame modes.

[0059] FIG. 40 Examples showing the positions of the current block's neighboring and non-neighboring blocks.

[0060] FIG. 41 This is a flowchart illustrating an example of the operation of signaling / parsing the information required to generate an MHP_IBC block.

[0061] FIG. 42 This is a flowchart illustrating another example of the operation of signaling / parsing the information required to generate an MHP_IBC block.

[0062] FIG. 43 This is a flowchart illustrating an example of the operation of signaling / parsing weight information for MHP-intra-frame mode.

[0063] FIG. 44 This is a flowchart illustrating another example of the operation of signaling / parsing weight information for MHP-intra-frame mode.

[0064] FIG. 45 An example is given where the current block is split when weights are applied to an MHP-intra-prediction block and different weights are applied to the corresponding split regions.

[0065] FIG. 46 This is a flowchart illustrating an example of a decoding method performed in MHP-intra-frame mode in a decoding method according to an embodiment.

[0066] FIG. 47 Examples illustrating the locations of reference samples used to generate MHP-intra-prediction blocks and samples within regular prediction blocks.

[0067] FIG. 48 An example is shown where, in order to compare the activity of samples corresponding to the same location, the location of samples in a regular prediction block is changed and applied to the region adjacent to the prediction block.

[0068] FIG. 49 This example demonstrates how a strong filter is applied to the current block.

[0069] FIG. 50 This is a flowchart illustrating an example of the operation of signaling / parsing filtered information for MHP-intra-frame mode.

[0070] FIG. 51 This is a flowchart illustrating another example of the operation of signaling / parsing filtered information for MHP-intra-frame mode.

[0071] FIG. 52 The illustrations are provided to exemplify a content streaming system that can be applied according to embodiments of the present disclosure. Detailed Implementation

[0072] Because this disclosure can be modified in various ways and has several embodiments, specific embodiments will be illustrated in the accompanying drawings and described in detail in the detailed description. However, this disclosure is not intended to be limited to the specific embodiments and should be understood to include all variations, equivalents, and substitutions included within the spirit and scope of this disclosure. Similar reference numerals are used for similar components in the description of each drawing.

[0073] Terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. These terms are used only to distinguish one component from other components. For example, without departing from the scope of this disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. Terms and / or combinations of any one or more of the relevant statement items are included.

[0074] When a component is described as "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to another component, but there may also be another component in between. On the other hand, when a component is described as "directly connected" or "directly linked" to another component, it should be understood that there is no other component in between.

[0075] The terminology used in this application is for describing particular embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, it should be understood that terms such as “comprising” or “having” are intended to designate the presence of features, numbers, steps, operations, components, portions, or combinations thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, portions, or combinations thereof.

[0076] This disclosure relates to video / image coding. For example, the methods / implementations disclosed herein can be applied to methods disclosed in the Universal Video Coding (VVC) standard. Additionally, the methods / implementations disclosed herein can be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second-generation Audio Video Coding (AVS2) standard, or next-generation video / image coding standards (e.g., H.267 or H.268).

[0077] This specification presents various implementations of video / image encoding, and unless otherwise stated, these implementations may be combined with each other to perform the task.

[0078] Here, "video" can refer to a collection of images over time. "Image" generally refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms part of an image during encoding. A slice / tile can include at least one Code Tree Unit (CTU). An image can consist of at least one slice / tile. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of an image. A tile column is a rectangular area of ​​CTUs with the same height as the image and a width assigned by the syntax requirements of the image parameter set. A tile row is a rectangular area of ​​CTUs with the same height assigned by the image parameter set and a width equal to the width of the image. CTUs within a tile can be arranged consecutively according to a CTU raster scan, and tiles within an image can be arranged consecutively according to a tile raster scan. A slice can include an integer number of complete tiles or an integer number of consecutive complete CTU rows that can be exclusively included within a single NAL unit of an image. Simultaneously, an image can be divided into at least two sub-images. A sub-image can be a rectangular area of ​​at least one slice within an image.

[0079] A pixel, cell, or pixel unit can refer to the smallest unit that makes up a picture (or image). Additionally, "sample" can be used as the term corresponding to a pixel. A sample can typically represent a pixel or pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component.

[0080] A unit can represent a conventional unit in image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, units may be used interchangeably with terms such as block or region. In general, an MxN block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0081] Here, "A or B" can refer to "A only", "B only", or "both A and B". In other words, "A or B" can be interpreted as "A and / or B". For example, "A, B or C" can refer to "A only", "B only", "C only", or "any combination of A, B and C".

[0082] The forward slash ( / ) or comma used in this article can refer to "and / or". For example, "A / B" can refer to "A and / or B". Therefore, "A / B" can refer to "A only", "B only", or "both A and B". For example, "A, B, C" can refer to "A, B, or C".

[0083] Here, "at least one of A and B" can refer to "only A", "only B" or "both A and B". Furthermore, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same way as "at least one of A and B".

[0084] Additionally, here, "at least one of A, B, and C" can refer to "A only", "B only", "C only" or "any combination of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" can refer to "at least one of A, B, and C".

[0085] Additionally, the parentheses used in this document can refer to "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction". In other words, "prediction" here is not limited to "intra-frame prediction", and "intra-frame prediction" can be cited as an example of "prediction". Furthermore, even when the indication is "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction".

[0086] Here, a technical feature described individually in a single figure may be implemented individually or simultaneously.

[0087] FIG. 1 A video / image encoding system according to this disclosure is shown.

[0088] refer to FIG. 1 A video / image encoding system may include a first device (source device) and a second device (receiving device).

[0089] A source device can transmit encoded video / image information or data to a receiving device in the form of a file or stream via digital storage media or a network. The source device may include a video source, an encoding device, and a transmitting unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may consist of a separate device or external components.

[0090] A video source can acquire video / images through the process of capturing, compositing, or generating video / images. A video source can include devices for capturing video / images and devices for generating video / images. Devices for capturing video / images can include at least one camera, video / image archives containing previously captured video / images, etc. Devices for generating video / images can include computers, tablets, smartphones, etc., and can generate video / images (electronically). For example, virtual video / images can be generated by computers, etc., and in this case, the process of capturing video / images can be replaced by the process of generating related data.

[0091] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization for compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0092] The transmitting unit can send encoded video / image information or data, output as a bitstream, to the receiving unit of the receiving device in the form of a file or stream, via digital storage media or a network. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiving unit can receive / extract the bitstream and send it to a decoding device.

[0093] Decoding devices can decode video / images by performing a series of processes, such as dequantization, inverse transform, and prediction, that correspond to the operations of encoding devices.

[0094] The renderer can render decoded video / images. The rendered video / images can be displayed through a display unit.

[0095] FIG. 2 A rough block diagram of an encoding apparatus that can be applied to embodiments of the present disclosure and perform encoding of video / image signals is shown.

[0096] refer to FIG. 2The encoding device 200 may consist of an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image partitioner 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). Additionally, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory 270 as an internal / external component.

[0097] Image partitioner 210 can partition an input image (or picture, frame) input to encoding device 200 into at least one processing unit. As an example, a processing unit can be referred to as a coding unit (CU). In this case, the coding unit can be recursively partitioned from coding tree unit (CTU) or maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure.

[0098] For example, a coding unit can be split into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure can be applied first, and a binary tree structure and / or a ternary structure can be applied later. Alternatively, a binary tree structure can be applied before the quadtree structure. The coding process according to this specification can be performed based on a final coding unit that is no longer split. In this case, based on coding efficiency according to image characteristics, the largest coding unit can be used directly as the final coding unit, or if necessary, the coding unit can be recursively divided into deeper coding units, and the coding unit with the optimal size can be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction, as described later.

[0099] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be partitioned or segmented from the aforementioned final encoding unit, respectively. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0100] In some cases, a unit can be used interchangeably with terms such as block or region. Generally, an MxN block can represent a set of transform coefficients or samples consisting of M columns and N rows. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. Samples can be used as a term to correspond a picture (or image) to pixels or cells.

[0101] Encoding device 200 can subtract the prediction signal (prediction block, prediction sample array) output from inter-frame predictor 221 or intra-frame predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is sent to converter 232. In this case, the unit in encoding device 200 that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called subtractor 231.

[0102] Predictor 220 can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a block of predictions including prediction samples for the current block. Predictor 220 can determine whether to apply intra-frame prediction or inter-frame prediction on a block or CU basis. Predictor 220 can generate various information about the prediction, such as prediction mode information, and send it to entropy encoder 240, as described later in the description of each prediction mode. The information about the prediction can be encoded in entropy encoder 240 and output as a bitstream.

[0103] Intra-predictor 222 can predict the current block by referencing samples within the current image. Depending on the prediction mode, the referenced samples can be located near the current block or at a distance from it. In intra-prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. The non-directional mode can include at least one of a DC mode or a planar mode. Depending on the level of detail of the prediction direction, the directional modes can include 33 or 65 directional modes. However, this is just an example, and more or fewer directional modes can be used depending on the configuration. Intra-predictor 222 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0104] Inter-frame predictor 221 can derive a prediction block for the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a juxtaposed reference block, juxtaposed CU (colCU), etc., and the reference image including the temporally neighboring block may be referred to as a juxtaposed image (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes, and for example, for skip mode and merge mode, the inter-frame predictor 221 can use motion information of neighboring blocks as motion information of the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, motion vectors of surrounding blocks are used as motion vector predictors, and motion vector differences are signaled to indicate the motion vector of the current block.

[0105] Predictor 220 can generate a prediction signal based on various prediction methods described later. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but also apply both intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as a combined intra-frame and inter-frame prediction (CIIP) mode. Alternatively, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for prediction against a block. The IBC prediction mode or palette mode can be used for content image / video coding such as screen content coding (SCC) in games, etc. IBC essentially performs prediction within the current image, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current image. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. A palette mode can be considered an example of intra-frame coding or intra-frame prediction. When a palette mode is applied, sample values ​​within the image can be signaled based on information about the palette table and palette index. The prediction signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal.

[0106] Transformer 232 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graphical Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to the transform obtained from a graphic when the relationship information between pixels is expressed as a graphic. CNT refers to the transform obtained based on generating a prediction signal using all previously reconstructed pixels. Furthermore, the transform process can be applied to square pixel blocks of the same size or to non-square blocks of variable size.

[0107] Quantizer 233 can quantize the transform coefficients and send them to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the quantized transform coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and can generate information about the quantized transform coefficients based on the one-dimensional vector form of the quantized transform coefficients.

[0108] The entropy encoder 240 can perform various encoding methods, such as exponential Columbus coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 240 can encode information necessary for video / video image reconstruction (e.g., values ​​of syntax elements, etc.) in addition to the transform coefficients quantized together or individually.

[0109] Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the network abstraction layer (NAL) unit level. The video / image information may further include information about various parameter sets such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). Additionally, the video / image information may further include general constraint information. Here, information transmitted from the encoding device / signaled to the decoding device and / or syntax elements can be included in the video / image information. The video / image information can be encoded by the above-described encoding process and included in the bitstream. The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. Transmission units (not shown) for transmission and / or storage units (not shown) for storing signals output from the entropy encoder 240 can be configured as internal / external components of the encoding device 200, or the transmission unit may also be included in the entropy encoder 240.

[0110] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients using dequantizer 234 and inverse transformer 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). When there is no residual for the block to be processed, such as when a skip mode is applied, the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed within the current image, and can also be used for inter-frame prediction of the next image by filtering, which will be described later. Meanwhile, a luminance mapping with chroma scaling (LMCS) can be applied during image encoding and / or reconstruction.

[0111] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270, specifically in the DPB of memory 270. Various filtering methods can include deblocking filtering, sample adaptive shifting, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various information about the filtering and send it to entropy encoder 240. The information about the filtering can be encoded in entropy encoder 240 and output as a bitstream.

[0112] The modified reconstructed image sent to memory 270 can be used as a reference image in inter-frame predictor 221. When inter-frame prediction is applied through it, the encoding device can avoid prediction mismatch in encoding device 200 and decoding device, and can also improve encoding efficiency.

[0113] The DPB of memory 270 can store modified reconstructed images for use as reference images in inter-frame predictor 221. Memory 270 can store motion information of blocks from which motion information in the current image is derived (or encoded) and / or motion information of blocks in the pre-reconstructed image. The stored motion information can be sent to inter-frame predictor 221 to be used as motion information for spatially or temporally neighboring blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and send them to intra-frame predictor 222.

[0114] Image information output from encoding device 200 in bitstream form can be sent to decoding device 300.

[0115] FIG. 3 A rough block diagram of a decoding device that can be implemented using embodiments of the present disclosure and perform decoding of video / image signals is shown.

[0116] Image information sent from encoding device 200 in bitstream form can be received by decoding device 300.

[0117] refer to FIG. 3 The decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 331 and an intra-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321.

[0118] According to the implementation, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 described above can be configured by a single hardware component (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded image buffer (DPB) and can be configured by a digital storage medium. The hardware component may further include the memory 360 as an internal / external component.

[0119] When the input includes a bitstream containing video / image information, the decoding device 300 can respond to... FIG. 2The process of processing video / image information in an encoding device reconstructs an image. For example, decoding device 300 can derive units / blocks based on block segmentation information obtained from the bitstream. Decoding device 300 can perform decoding by using processing units applied in the encoding device. Therefore, the decoding processing unit can be an encoding unit, and the encoding unit can be segmented from encoding tree units or maximally encoded units according to a quadtree structure, binary tree structure, and / or ternary tree structure. At least one transform unit can be derived from the encoding unit. Furthermore, the reconstructed image signal decoded and output by decoding device 300 can be played back by a playback device.

[0120] Decoding device 300 can receive data in bitstream form from... FIG. 2The signal output by the encoding device and the received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may further include information about various parameter sets such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may further include general constraint information. The decoding device can further decode the picture based on the information about the parameter sets and / or the general constraint information. The information sent / received by the signal and / or the syntax elements described later herein can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 can decode the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element from the bitstream, determine a context model using information about the syntax element to be decoded, decoding information of surrounding blocks and the block to be decoded, or information about symbols / bins decoded in the previous step, perform arithmetic decoding on the bins by predicting the occurrence probability of the bins based on the determined context model, and generate symbols corresponding to the value of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about the decoded symbols / bins for the context model used for the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​of entropy decoding performed on them in the entropy decoder 310, i.e., the quantized transform coefficients and related parameter information, can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, information about filtering in the information decoded in the entropy decoder 310 can be provided to the filter 350. Meanwhile, the receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300 or the receiving unit can be a component of the entropy decoder 310.

[0121] Furthermore, the decoding device according to this specification can be referred to as a video / image / picture decoding device, and the decoding device can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0122] Dequantizer 321 can dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can obtain the transform coefficients by performing dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information).

[0123] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0124] Predictor 320 can perform prediction on the current block and generate a prediction block including prediction samples for the current block. Predictor 320 can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from entropy decoder 310, and determine a specific intra-frame / inter-frame prediction mode.

[0125] Predictor 320 can generate prediction signals based on various prediction methods described later. For example, predictor 320 can not only apply intra-frame prediction or inter-frame prediction to predict a block, but also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as a combined intra-frame and inter-frame prediction (CIIP) mode. Alternatively, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for block prediction. The IBC prediction mode or palette mode can be used for content image / video coding such as screen content coding (SCC) in games, etc. IBC essentially performs prediction within the current frame, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current frame. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. Palette mode can be considered an example of intra-frame coding or intra-frame prediction. When a palette mode is applied, information about the palette table and palette index can be included in the video / image information and transmitted as a signal.

[0126] Intra-predictor 331 can predict the current block by referencing samples within the current image. Depending on the prediction mode, the referenced samples can be located near the current block or at a certain distance away from the current block. In intra-prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. Intra-predictor 331 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0127] Inter-frame predictor 332 can derive a prediction block for the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information of neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and derive the motion vector and / or reference image index of the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode used for the current block.

[0128] Adder 340 can add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331) to generate a reconstruction signal (reconstructed image, reconstruction block, reconstruction sample array). When there is no residual for the block to be processed, such as when a skip mode is applied, the prediction block can be used as the reconstruction block.

[0129] Adder 340 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, output by filtering as described later, or it can be used for inter-frame prediction of the next image. Meanwhile, a luminance map with chroma scaling (LMCS) can be applied during image decoding.

[0130] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and send the modified reconstructed image to memory 360, specifically the DPB of memory 360. Various filtering methods can include deblocking filtering, adaptive sampling offset, adaptive loop filtering, bilateral filtering, etc.

[0131] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in inter-frame prediction unit 332. Memory 360 can store motion information of blocks derived (or decoded) from motion information in its current image and / or motion information of blocks in the pre-reconstructed image. The stored motion information can be sent to inter-frame predictor 260 as motion information for spatially or temporally neighboring blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and send them to intra-frame predictor 331.

[0132] The embodiments described here in the filter 260, inter-frame predictor 221 and intra-frame predictor 222 of the encoding device 200 can also be applied equally or correspondingly to the filter 350, inter-frame predictor 332 and intra-frame predictor 331 of the decoding device 300.

[0133] FIG. 4 Examples of video / image decoding methods to which embodiments of the present disclosure may be applied are provided.

[0134] In video / image coding, the images that make up a video / image can be encoded / decoded according to a series of decoding sequences. The image order corresponding to the output order of the decoded images can be set to be different from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed in inter-frame prediction.

[0135] exist FIG. 4 In this process, S400 can be executed by the entropy decoder 310 of the aforementioned decoding device 300, S410 can be executed by the predictor 330, S420 can be executed by the residual processor 320, S430 can be executed by the adder 340, and S440 can be executed by the filter 350. S400 may include the decoding process according to this disclosure, S410 may include the inter-frame / intra-frame prediction process according to this disclosure, S420 may include the residual processing process according to this disclosure, S430 may include the block / image reconstruction process according to this disclosure, and S440 may include the intra-loop filtering process according to this disclosure.

[0136] Reference FIG. 4 The decoding device can obtain video / image information from the bitstream (S400), perform prediction based on the obtained video / image information (S410), and reconstruct the image through residual processing (S420, dequantizing and inverse transforming the quantized transform coefficients) (S430).

[0137] Through the reconstruction process, the in-loop filtering process (S440) can be applied to the reconstructed image generated by the reconstruction process to produce a modified reconstructed image. The modified reconstructed image can be output as a decoded image and can also be stored in the buffer or memory of the decoding device, and used as a reference image in the inter-frame prediction process when decoding the next image. In some cases, the in-loop filtering process can be omitted, and in this case, the reconstructed image can be output as a decoded image, and can also be stored in the buffer or memory of the decoding device and used as a reference image in the inter-frame prediction process when decoding subsequent images.

[0138] The in-loop filtering process (S440) may include a deblocking filtering process, a SAO (Sample Adaptive Offset) process, an ALF (Adaptive Loop Filter) process, and / or a bilateral filtering process, and some or all of them may be omitted. Furthermore, one or more of the deblocking filtering process, the SAO process, the ALF process, and the bilateral filtering process may be applied sequentially, or all of them may be applied sequentially. For example, the SAO process may be performed after the deblocking filtering process has been applied to the reconstructed image. Alternatively, for example, the ALF process may be performed after the deblocking filtering process has been applied to the reconstructed image. The same process may be performed in the encoding device.

[0139] FIG. 5 Examples of video / image coding methods to which embodiments of the present disclosure may be applied are provided.

[0140] exist FIG. 5 In this process, the prediction step (S500) can be performed by the predictor 220 of the encoding device 200, the residual processing based on the prediction result (S510) can be performed by the residual processor 230, and the step of encoding video information including prediction information and residual information (S520) can be performed by the entropy encoder 240. S500 may include the inter-frame / intra-frame prediction process according to this disclosure, S510 may include the residual processing process according to this disclosure, and S520 may include the encoding process according to this disclosure.

[0141] The encoding process may optionally include not only encoding the information used for image reconstruction (e.g., prediction information, residual information, partitioning information, etc.) and outputting it as a bitstream, but also generating a reconstructed image for the current image and applying in-loop filtering to the reconstructed image.

[0142] Encoding device 200 can derive (modified) residual samples from quantized transform coefficients using dequantizer 234 and inverse transformer 235, and can generate a reconstructed image based on the predicted sample as the output of S500 and the (modified) residual samples. The reconstructed image generated in this way can be the same as the reconstructed image generated in decoding device 300 described above. The modified reconstructed image can be generated through an in-loop filtering process for reconstructing the image, and the modified reconstructed image can be stored in a buffer or memory and used as a reference image in the inter-frame prediction process when encoding subsequent images, similar to the case of the decoding device.

[0143] As mentioned above, in some cases, some or all of the in-loop filtering process can be omitted. When performing the in-loop filtering process, the filtering-related information (parameters) can be encoded by the entropy encoder 240 and output as a bit stream, and the decoding device 300 can perform the in-loop filtering process in the same manner as the encoding device based on the filtering-related information.

[0144] This in-loop filtering process reduces noise (such as artifacts and ringing) that occurs during image / video encoding and improves subjective / objective image quality. Furthermore, by performing the in-loop filtering process in both the encoding device 200 and the decoding device 300, both devices can derive the same prediction results, thereby improving the reliability of image encoding and reducing the amount of data that needs to be transmitted for image encoding.

[0145] As described above, the image reconstruction process can be performed not only in the decoding device 300 but also in the encoding device 200. Reconstructed blocks can be generated on a per-block basis based on intra-frame prediction / inter-frame prediction, and a reconstructed image including these blocks can be generated. When the current image / slice / tile group is an I-image / slice / tile group, the blocks included in the current image / slice / tile group can be reconstructed based solely on intra-frame prediction. Meanwhile, when the current image / slice / tile group is a P-image / slice / tile group or a B-image / slice / tile group, the blocks included in the current image / slice / tile group can be reconstructed based on either intra-frame prediction or inter-frame prediction. In this case, inter-frame prediction can be applied to some blocks within the current image / slice / tile group, and intra-frame prediction can be applied to other blocks.

[0146] The color components of an image may include luminance components and chrominance components, and unless expressly limited in this disclosure, embodiments of this disclosure may be applied to both luminance and chrominance components.

[0147] Intra prediction

[0148] The predictors 220 and 330 of the encoding device 200 and the decoding device 300 can derive reference samples from the neighboring samples of the current block according to the intra-frame prediction mode of the current block, and can generate prediction samples of the current block based on the reference samples.

[0149] For example, (i) the predicted sample can be derived based on the average or interpolation of the neighboring reference samples of the current block, and (ii) the predicted sample can be derived based on the reference sample among the neighboring reference samples of the current block that is located in a specific (predictive) direction of the predicted sample. Case (i) can be called non-directional mode or non-angular mode, and case (ii) can be called directional mode or angular mode.

[0150] In addition, linear interpolation intra-prediction (LIP) can be applied, where intra-prediction is performed for the current block by linearly interpolating the predicted sample values ​​generated by the intra-prediction mode based on the current block.

[0151] Furthermore, the provisional prediction sample for the current block can be derived based on filtered neighbor reference samples, and the prediction sample for the current block can be derived by performing a weighted sum of at least one reference sample derived according to the intra-prediction mode from the existing neighbor reference samples (i.e., unfiltered neighbor reference samples) with the provisional prediction sample. Such a prediction can be called PDPC (Location Dependent Intra-Prediction Combination).

[0152] Furthermore, intra-frame prediction coding can be performed by selecting the reference sample line with the highest prediction accuracy among multiple neighboring reference sample lines in the current block, deriving the prediction sample using reference samples in the prediction direction located on the selected line, and indicating (signaling) the reference sample line used to the decoding device. This approach can be called multi-reference line intra-frame prediction (MRL) or MRL-based intra-frame prediction.

[0153] Furthermore, intra-frame prediction can be performed using the same intra-frame prediction mode by dividing the current block into vertical or horizontal sub-partitions, and neighbor reference samples can be derived and used on a sub-partition basis. That is, in this case, the intra-frame prediction mode used for the current block is applied equivalently to the sub-partitions, but in some cases, deriving and using neighbor reference samples on a sub-partition basis can improve prediction performance. Such a prediction method can be called intra-frame sub-partitioning (ISP) or ISP-based intra-frame prediction.

[0154] Furthermore, when the prediction direction based on the predicted sample points between neighboring reference samples, that is, when the prediction direction points to the fractional sample position, the value of the predicted sample can be derived by interpolating multiple reference samples located around the prediction direction (around the fractional sample position).

[0155] The aforementioned intra-prediction methods can be referred to as intra-prediction types to distinguish them from intra-prediction modes. Intra-prediction types can be referred to by various terms such as intra-prediction techniques, intra-prediction tools, detailed intra-prediction modes, or additional intra-prediction modes. For example, an intra-prediction type (or additional intra-prediction mode, etc.) can include at least one of LIP, PDPC, MRL, or ISP. Information about intra-prediction types can be encoded by the encoding device, included in the bitstream, and transmitted to the decoding device. Information about intra-prediction types can be implemented and transmitted in various forms, such as flag information indicating whether each intra-prediction type is applied or index information indicating one of multiple intra-prediction types.

[0156] The MPM list used to derive the above intra-prediction modes can be configured differently depending on the intra-prediction type. Alternatively, the MPM list can be configured uniformly regardless of the intra-prediction type.

[0157] PDPC (Position Dependent Intra Prediction Combination) prediction

[0158] According to PDPC, the filtered reference sample can be derived by performing filtering based on a predefined filter, and the provisional prediction sample for the current block can be derived based on the intra-prediction mode of the current block and the filtered reference sample. The prediction sample for the current block can be derived by performing a weighted sum of at least one reference sample derived from the intra-prediction mode among the existing reference samples (i.e., the unfiltered reference samples) and the provisional prediction sample. Here, the predefined filter can be one of five 7-tap filters. Alternatively, the predefined filter can be one of a 3-tap filter, a 5-tap filter, or a 7-tap filter. The 3-tap filter, 5-tap filter, and 7-tap filter can represent a filter with three filter coefficients, a filter with five filter coefficients, and a filter with seven filter coefficients, respectively.

[0159] For example, the prediction results of intra-plane mode can be further modified using PDPC.

[0160] Alternatively, as an example, PDPC can be applied to intra-plane mode, intra-DC mode, horizontal intra-prediction mode, vertical intra-prediction mode, lower-left intra-prediction mode (i.e., intra-prediction mode 2), eight directional intra-prediction modes adjacent to the lower-left intra-prediction mode, upper-right intra-prediction mode, and eight directional intra-prediction modes adjacent to the upper-right intra-prediction mode without separate signaling.

[0161] Specifically, when applying PDPC, the predicted sample at coordinates (x, y) predicted based on a linear combination of intra-frame prediction mode and reference sample can be derived as shown in Equation 1 below.

[0162] [Formula 1]

[0163] Here, R x,-1 Let R-1,y represent the top reference sample and the left reference sample located above and to the left of the current sample at coordinates (x, y), and R0... -1,-1 This represents the top-left reference sample located at the top-left corner of the current block.

[0164] Meanwhile, when PDPC is applied to intra-plane mode, intra-DC mode, horizontal intra-prediction mode, and vertical intra-prediction mode, additional boundary filters such as the DC mode boundary filter or the vertical / horizontal mode edge filter of conventional HEVC may not be necessary.

[0165] FIG. 6 Example of a reference sample (R) defined in PDPC based on various prediction models. x,-1 R -1,y R -1,-1 ).

[0166] Reference FIG. 6 In PDPC, reference samples can be derived from (a) the diagonal upper right pattern, (b) the diagonal lower left pattern, (c) the adjacent diagonal upper right pattern and (d) the adjacent diagonal lower left pattern.

[0167] The weights of PDPC can be derived based on the prediction pattern as shown in Table 1 below.

[0168] [Table 1]

[0169] PDPC generates prediction samples using reference samples based on the prediction mode, and then refines the prediction samples using surrounding reference samples. Based on 65 directional intra-frame prediction modes, PDPC can be applied preferentially to the plane, DC, mode 2 (bottom right directional mode), VDIA (top left directional mode), Hor (horizontal directional mode), Ver (vertical directional mode), neighboring modes of mode 2 (modes 3 to 10), and neighboring modes of VDIA mode (modes 58 to 65). Furthermore, PDPC can be applied variably, taking into account the block size, rather than being applied to all prediction samples within the block to be encoded.

[0170] ISP (Intra Sub Partition) prediction

[0171] Conventional intra-frame prediction performs encoding by treating the block to be encoded as a single coding unit without partitioning. However, intra-fragmentation (ISP) prediction methods perform intra-frame predictive coding by splitting the block to be encoded horizontally or vertically. In this case, encoding / decoding is performed on a block-by-block basis to generate a reconstructed block, which is then used as a reference block for the next split block. Currently, intra-fragmentation (ISP) can be split according to the block size as shown in Table 2 below.

[0172] [Table 2]

[0173] The ISP tool can split a block of luminance intra-frame prediction into two or four sub-partitions in the vertical or horizontal direction, depending on the block size. For example, the minimum block size for ISP is 4×8 (or 8×4). When the block size is larger than 4×8 (or 8×4), the block can be split into four sub-partitions.

[0174] FIG. 7 An example of how intra-frame sub-partitions are split.

[0175] Reference FIG. 7 And [Table 2], a 4×8 block can be split horizontally or vertically and divided into two sub-partitions, as illustrated in (a). Blocks of sizes other than 4×4, 4×8 and 8×4 can be split horizontally or vertically and divided into four sub-partitions, as illustrated in (b).

[0176] To reduce coding complexity, the intra-framing sub-partitioning method generates a list of most probable modes (MPMs) for each partitioning method (horizontal and vertical partitioning) and compares the predicted modes in the generated MPM list from a rate-distortion optimization (RDO) perspective to determine the optimal mode. Furthermore, when using multi-reference line (MRL) intra-framing prediction, the above intra-framing sub-partitioning method may not be used. That is, the intra-framing sub-partitioning method is only applied when using the 0th reference line (i.e., the value of intra_luma_ref_idx is 0). Additionally, when using the above intra-framing sub-partitioning method, the aforementioned PDPC may not be used.

[0177] In the intra-subpartition method, whether intra-subpartition is applied is first sent based on the block, and if the current block uses intra-subpartition (intra_subpartitions_mode_flag), then the information indicating whether the partition is horizontal or vertical is encoded / decoded (intra_subpartitions_split_flag).

[0178] When applying the intra-fractional sub-partitioning method, the intra-prediction mode used for the current block is applied equally to the sub-partitions, and intra-prediction performance can be improved by deriving and using neighboring reference samples on a sub-partition basis. In other words, when applying the intra-fractional sub-partitioning method, residual sample processing is performed on a sub-partition basis. That is, prediction samples are derived for each sub-partition, and reconstructed samples are obtained by adding the residual signal (residual sample) of the corresponding sub-partition to it.

[0179] The residual signal (residual sample) can be derived from the residual information (quantized transform coefficient information or residual coding syntax) in the bitstream described above through a dequantization / inverse transform process. That is, the predicted sample of the first sub-partition can be derived, the residual sample can be derived, and the reconstructed sample of the first sub-partition can be derived based on it. In this case, when deriving the predicted sample of the second sub-partition, some reconstructed samples from the reconstructed samples in the first sub-partition (e.g., the left or top neighbor reference samples of the second sub-partition) can be used as neighbor reference samples of the second sub-partition.

[0180] Similarly, the predicted samples for the second sub-partition can be derived, the residual samples can be derived, and the reconstructed samples for the second sub-partition can be derived based on them. In this case, when deriving the predicted samples for the third sub-partition, some reconstructed samples from the reconstructed samples in the second sub-partition (e.g., the left or top neighbor reference samples of the third sub-partition) can be used as neighbor reference samples for the third sub-partition. The same description can be applied to subsequent sub-partitions.

[0181] Matrix-based Intra Prediction (MIP)

[0182] FIG. 8 The matrix-based intra-frame prediction process is illustrated schematically.

[0183] Matrix-based intra prediction (MIP) can also be called Affine Linear Weighted Intra Prediction (ALWIP) or Matrix Weighted Intra Prediction (MWIP). To predict samples from rectangular blocks of equal width and height, MIP uses the reconstructed neighbor boundary samples from the left row of the block and the top row of the block as input. When reconstructed samples are unavailable, reconstructed samples are generated as performed in regular intra prediction.

[0184] like FIG. 8 As shown, the generation of the prediction signal can be performed according to the following three steps.

[0185] 1. When W=H=4, extract four samples from the boundary sample by average, and extract eight samples (average) in all other cases.

[0186] 2. Matrix-vector multiplication is performed using the averaged samples as input and an offset is added, resulting in a reduction of the predicted signal for the subsampled sample set in the original block (matrix-vector multiplication).

[0187] 3. Generate the prediction signal at the remaining positions by linear interpolation, which performs a step-by-step linear interpolation (interpolation) on the prediction signal in each direction of the subsample set.

[0188] The matrices and offset vectors required to generate the prediction signal (prediction block or prediction sample) are taken from three matrix sets S0, S1, and S2. Set S0 consists of 18 matrices. (Each matrix has 16 rows and 4 columns) and 18 offset vectors Composition (each offset vector is 16 in size). The matrices and offset vectors of this set are used for blocks of size 4×4. Set S1 consists of 10 matrices. (Each matrix has 16 rows and 8 columns) and 10 offset vectors Composed of (each offset vector is 16 in size). The matrices and offset vectors of this set are used for blocks of sizes 4×8, 8×4, and 8×8. Finally, set S2 consists of 6 matrices. (Each matrix has 64 rows and 8 columns) and 6 offset vectors Composition (each offset vector is 64 in size). The set of matrices and offset vectors, or some of the matrices and offset vectors, are used for all other block shapes.

[0189] The total number of multiplications required to compute matrix-vector multiplication is less than or equal to In other words, in MIP mode, each sample requires a maximum of four multiplications.

[0190] Decoder-side Intra mode derivation (DIMD)

[0191] In DIMD (decoder-side intra-frame mode derivation), intra-frame prediction can be derived as a weighted average between a plane and two derivation directions. Two angular modes can be selected from the HoG (histogram of gradients) calculated based on the neighboring pixels of the current block. When two angular modes are selected, predictors for the angular modes and a plane predictor are calculated, and their weighted average is used as the final predictor for the corresponding block. To determine the weights, the magnitude of the corresponding HoG is used for each of the two modes.

[0192] The deduced intra-frame mode is included in the initial list of MPMs, so the DIMD process is performed before the MPM list is constructed. The first deduced intra-frame mode of a DIMD block is stored with the block and can be used to construct the MPM list of neighboring blocks.

[0193] Template-based Intra mode derivation (TIMD) fusion

[0194] For each intra-prediction of the MPM, the SATD (Sum of Absolute Transform Differences) between the predicted and reconstructed samples of the template is calculated. The two intra-prediction modes with the smallest SATD can be selected as TIMD modes. The two TIMD modes are fused by applying weights, and the weighted intra-prediction is used for coding the current CU. PDPC can be included in the derivation of the TIMD modes.

[0195] The costs of the two selected modes are compared with a threshold, and a cost factor of 2 is applied during the test, as follows: costMode2 < 2 costMode1.

[0196] When this condition is met, apply fusion; otherwise, use only mode 1.

[0197] The weights of the patterns are calculated based on SATD costs as follows: weight1 = costMode2 / (costMode1 + costMode2) weight2 = 1 - weight1

[0198] Spatial Geometry Partition Mode (SGPM)

[0199] SGPM is an intra-frame mode of an inter-frame coding tool similar to GPM, generating two prediction parts during intra-frame prediction. In this mode, a candidate list is constructed, where each entry includes a partition split and two intra-frame prediction modes, and a combination is formed using one partition mode and three intra-frame prediction modes. The length of the candidate list can be set to 16, and the selected candidate index can be signaled.

[0200] The candidate list is reordered using a template, where the SAD (Situational Aspect Difference) between the template's prediction and reconstruction is used for sorting. The template size can be fixed at 1.

[0201] For each partition mode, the same intra-frame to inter-frame GPM list derivation is used to derive the IPM list for each partition. The size of the IPM list can be set to 3. Within the list, the TIMD derivation mode can be replaced with two derivation modes in the horizontal and vertical directions.

[0202] The SGPM mode can be applied to the following limited block sizes: 4 ≤ width ≤ 64, 4 ≤ height ≤ 64, width < height × 8, height < width × 8, width × height ≥ 32.

[0203] Adaptive blending is also used in spatial GPM, and the blending depth τ can be derived as follows: - If minimum (width, height) = 4, choose 1 / 2τ.

[0204] - Otherwise, if the minimum (width, height) = 8, choose τ.

[0205] Otherwise, if the minimum (width, height) = 16, choose 2τ.

[0206] Otherwise, if the minimum (width, height) = 32, choose 4τ.

[0207] - Otherwise, choose 8τ.

[0208] Intra Block Copy (IBC)

[0209] IBC is a tool used in the HEVC extension of SCC. It is well known that the coding efficiency of screen content material is significantly improved. Since IBC mode is implemented as a block-level coding mode, block matching (BM) can be performed at the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector can be used to represent the displacement from the current block to a reference block that has already been reconstructed within the current image. The luma block vector of an IBC-encoded CU can have an integer resolution (or precision). The chroma block vector can be rounded to an integer resolution. Then, the IBC mode combined with AMVR can switch between 1-pixel and 4-pixel motion vector resolutions. IBC-encoded CUs can be treated as a third prediction mode, rather than an intra-frame or inter-frame prediction mode. IBC mode can be applied to CUs whose width and height are both less than or equal to 64 luma samples.

[0210] On the encoder side, hash-based motion estimation can be performed for IBC. The encoder can perform BD checks on blocks with a width and height of no more than 16 luminance samples. For non-merging modes, a block vector search can be performed first using a hash-based search. When the hash search does not return valid candidates, a local search based on block matching can be performed.

[0211] In hash-based searches, hash key matching (32-bit CRC) between the current block and the reference block can be extended to all allowed block sizes. The hash key calculation for all locations in the current image is based on 4×4 sub-blocks. For a current block with a larger size, a hash key match with the reference block's hash key can be determined when all hash keys of the 4×4 sub-blocks match the hash key of the corresponding reference location.

[0212] When multiple predicted blocks are found to match the hash key of the current block, the block vector cost of each matching reference is calculated, and the minimum cost can be selected.

[0213] In block matching searches, the search scope can be set to cover both the previous CTU and the current CTU. At the CU level, flags are used to signal the IBC mode, and the IBC mode can be signaled as IBC AMVP mode or IBC skip / merge mode as follows.

[0214] - IBC Skip / Merge Mode: Merge candidate indices can be used to indicate which block vectors from the list of neighboring candidate IBC encoded blocks are used to predict the current block. The merge list can include spatial, HMVP, and paired candidates.

[0215] -IBC AMVP Mode: Block vector differences can be encoded in the same way as motion vector differences. The block vector prediction method can use two candidates (in the case of IBC encoding): a predictor from the left neighbor and a predictor from the top neighbor. When neither neighbor is available, a default block is used as the predictor. A signal flag can be used to indicate the block vector predictor index.

[0216] Simultaneously, when applying inter-frame prediction, the predictor of the encoding / decoding device can derive prediction samples by performing inter-frame prediction on a block-by-block basis. Inter-frame prediction can be represented as a prediction derived in a manner that depends on data elements (e.g., sample values ​​or motion information) of images other than the current image. When inter-frame prediction is applied to the current block, the prediction block (prediction sample array) for the current block can be derived based on the reference block (reference sample array) specified by the motion vectors in the reference image indicated by the reference image index.

[0217] To reduce the amount of motion information transmitted in inter-frame prediction mode, the motion information of the current block can be predicted on a block, sub-block, or sample basis based on the correlation between the motion information of neighboring blocks and the current block. Motion information may include motion vectors and / or reference image indices. Motion information may also include inter-frame prediction type information (L0 prediction, L1 prediction, bidirectional prediction, etc.). When applying inter-frame prediction, neighboring blocks may include spatial neighboring blocks existing in the current image and temporal neighboring blocks existing in the reference image.

[0218] The reference image including the reference block and the reference image including the temporal neighbor block can be the same or different. The temporal neighbor block can be called a co-located reference block or co-located CU (colCU), and the reference image including the temporal neighbor block can be called a co-located image (colPic). For example, a candidate list of motion information can be constructed based on the neighbor blocks of the current block, and a signal can be sent to indicate which candidate to select (use) to derive the motion vector and / or reference image index of the current block, and a flag or index information.

[0219] Inter-frame prediction can be performed based on various prediction modes. For example, in skip and merge modes, the motion information of the current block can be the same as that of the selected neighboring blocks. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, the motion vectors of the selected neighboring blocks are used as motion vector predictors, and motion vector difference (MVD) signals can be sent. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0220] Depending on the inter-frame prediction type (L0 prediction, L1 prediction, bidirectional prediction, etc.), motion information can include L0 motion information and / or L1 motion information. The motion vector in the L0 direction can be called the L0 motion vector or MVL0, and the motion vector in the L1 direction can be called the L1 motion vector or MVL1. Prediction based on the L0 motion vector can be called L0 prediction, prediction based on the L1 motion vector can be called L1 prediction, and prediction based on both L0 and L1 motion vectors can be called bidirectional prediction. Here, the L0 motion vector can represent the motion vector associated with the reference image list L0, and the L1 motion vector can represent the motion vector associated with the reference image list L1. The reference image list L0 can include images that are earlier than the current image in the output order, and the reference image list L1 can include images that are later than the current image in the output order. The earlier image can be called the forward reference image, and the later image can be called the backward reference image.

[0221] The reference image list L0 can also include images that are later in the output order than the current image. In this case, in reference image list L0, earlier images can be indexed first, and later images can be indexed later. The reference image list L1 can also include images that are earlier in the output order than the current image. In this case, in reference image list L1, later images can be indexed first, and earlier images can be indexed later. Here, the output order can correspond to the Image Order Count (POC) order.

[0222] FIG. 9 andFIG. 10 Examples of video / image coding methods based on inter-frame prediction that can be applied to embodiments of this disclosure are provided.

[0223] Reference FIG. 9 The encoding device (200) can perform inter-frame prediction (S600) on the current block. The encoding device can deduce the inter-frame prediction mode and motion information of the current block and generate prediction samples for the current block. Here, the processes of determining the inter-frame prediction mode, deduce motion information, and generate prediction samples can be performed simultaneously, or one process can be performed before the other. For example, as... FIG. 10 As shown, the inter-frame predictor (221) of the encoding device (200) may include a prediction mode determiner (221a), a motion information deducer (221b), and a prediction sample deducer (221c). The prediction mode determiner (221a) can determine the prediction mode of the current block, the motion information deducer (221b) can deduce the motion information of the current block, and the prediction sample deducer (221c) can deduce the prediction samples of the current block.

[0224] For example, the inter-frame predictor of the encoding device can search for blocks similar to the current block within a predetermined region (search region) of a reference image through motion estimation, and can deduce reference blocks that have the smallest difference from the current block or whose difference is less than or equal to a predetermined threshold. Based on this, a reference image index indicating the reference image where the reference block is located can be derived, and motion vectors can be derived based on the positional difference between the reference block and the current block. The encoding device can determine the mode to be applied to the current block from various prediction modes. The encoding device can compare the RD costs of various prediction modes and determine the optimal prediction mode for the current block.

[0225] For example, when a skip mode or merge mode is applied to the current block, the encoding device can construct a merge candidate list, described later, and deduce a reference block with the smallest difference or a difference less than or equal to a predetermined threshold based on the sample difference (e.g., SAD or SATD) between the current block and a reference block indicated by a merge candidate included in the merge candidate list. In this case, a merge candidate associated with the deduced reference block is selected, and merge index information indicating the selected merge candidate can be generated and signaled to the decoding device. The motion information of the current block can be deduced using the motion information of the selected merge candidate.

[0226] As another example, when the (A)MVP mode is applied to the current block, the encoding device can construct an (A)MVP candidate list, described later, and can use the motion vector of a motion vector predictor candidate selected from the motion vector predictor (MVP) candidates included in the (A)MVP candidate list as the motion vector predictor for the current block. In this case, for example, the motion vector of the reference block derived through the motion estimation described above can be used as the motion vector of the current block, and the motion vector predictor candidate with the motion vector having the smallest difference from the motion vector of the current block can be selected as the motion vector predictor candidate. The motion vector difference (MVD) can be derived, which is the difference between the motion vector of the current block and the motion vector predictor. In this case, information about the MVD can be signaled to the decoding device. In addition, when the (A)MVP mode is applied, the value of the reference picture index can be configured as reference picture index information and can be signaled to the decoding device separately.

[0227] The encoding device can derive residual samples based on the predicted samples (S610). The encoding device can derive residual samples by comparing the original samples and the predicted samples of the current block.

[0228] The encoding device can encode video / image information including prediction information and residual information (S620). The encoding device can output the encoded video / image information in the form of a bitstream. The prediction information may include information related to the prediction process, such as prediction mode information (e.g., skip flag, merge flag, or merge index) and / or motion information. The motion information may include candidate selection information (e.g., merge index, MVP flag, or MVP index), which is information used to derive motion vectors. In addition, the motion information may include information about the aforementioned MVD and / or reference image index information. Furthermore, the motion information may include information indicating whether L0 prediction, L1 prediction, or bidirectional prediction is applied. The residual information is information related to the residual samples. The residual information may include information about the quantized transform coefficients of the residual samples.

[0229] The output bitstream can be stored in a (digital) storage medium and sent to a decoding device, or it can be sent to a decoding device via a network.

[0230] Simultaneously, as mentioned above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This is to derive the same prediction results in the encoding device as those performed in the decoding device, thereby improving encoding efficiency. Therefore, the encoding device can store the reconstructed image (or reconstructed samples or reconstructed blocks) in memory and use it as a reference image for inter-frame prediction. As mentioned above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0231] FIG. 11 and FIG. 12 Examples of video / image decoding methods based on inter-frame prediction that can be applied to embodiments of this disclosure are provided.

[0232] The video / image decoding process based on inter-frame prediction can schematically include, for example, the following.

[0233] Reference FIG. 11 The decoding device 300 can perform operations corresponding to those performed in the encoding device 200. The decoding device can perform prediction on the current block and derive prediction samples based on the received prediction information.

[0234] Specifically, the decoding device can determine the prediction mode of the current block based on the received prediction information (S700). The prediction mode determiner 332a of the decoding device 300 can determine which inter-frame prediction mode to apply to the current block based on the prediction mode information in the prediction information.

[0235] For example, a merge flag can be used to determine whether to apply a merge mode or an (A)MVP mode to the current block. Alternatively, a variety of inter-frame prediction mode candidates can be selected based on a mode index. Inter-frame prediction mode candidates may include skip mode, merge mode, and / or (A)MVP mode, or may include various inter-frame prediction modes described later.

[0236] The decoding device can derive motion information for the current block based on the determined inter-frame prediction mode (S710). For example, when a skip mode or merge mode is applied to the current block, the motion information derivator 332b of the decoding device 300 can configure a merge candidate list, described later, and select a merge candidate from among the merge candidates included in the merge candidate list. Such selection can be performed based on the selection information (merge index) described above. The motion information for the current block can be derived using the motion information of the selected merge candidate. The motion information of the selected merge candidate can be used as the motion information for the current block.

[0237] As another example, when the (A)MVP mode is applied to the current block, the decoding device can configure the (A)MVP candidate list, described later, and can use the motion vector of the MVP (Motion Vector Predictor) candidate selected from the MVP candidates included in the (A)MVP candidate list as the MVP of the current block. Such selection can be performed based on the selection information described above (MVP flag or MVP index). In this case, the MVD of the current block can be derived based on information about the MVD, and the motion vector of the current block can be derived based on the MVP and MVD of the current block. Additionally, the reference image index of the current block can be derived based on reference image index information. In the reference image list for the current block, the image indicated by the reference image index can be derived as the reference image referenced for inter-frame prediction of the current block.

[0238] Furthermore, as will be described later, the motion information of the current block can be derived without configuring a candidate list, and in this case, the motion information of the current block can be derived based on the process disclosed in the prediction pattern, which will be described later. In this case, the configuration of the aforementioned candidate list can be omitted.

[0239] The decoding device can generate a prediction sample for the current block based on the motion information of the current block (S720). In this case, the prediction sample derivator 332c of the decoding device 300 can derive a reference image based on the reference image index of the current block, and can derive the prediction sample for the current block by using samples of the reference block on the reference image indicated by the motion vector of the current block. In this case, as will be described later, in some cases, a prediction sample filtering process for all or some of the prediction samples of the current block can be further performed.

[0240] In other words, the inter-frame predictor 332 of the decoding device 300 may include a prediction mode determiner 332a, a motion information inferrer 332b, and a prediction sample inferrer 332c. The prediction mode determiner 332a can determine the prediction mode of the current block based on the received prediction mode information, the motion information inferrer 332b can infer the motion information (motion vector and / or reference image index, etc.) of the current block based on the received motion information-related information, and the prediction sample inferrer 332c can derive or generate prediction samples for the current block.

[0241] The decoding device generates residual samples for the current block based on the received residual information (S730). The decoding device 300 can generate reconstructed samples for the current block based on the predicted samples and residual samples, and can generate a reconstructed image based on them (S740). Thereafter, as described above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0242] FIG. 13An inter-frame prediction process that can be applied to embodiments of this disclosure is illustrated by way of example.

[0243] Reference FIG. 13 As described above, the inter-frame prediction process (S600) may include an inter-frame prediction mode determination step, a motion information derivation step based on the determined prediction mode, and a prediction execution (prediction sample generation) step based on the derivation of motion information. As described above, the inter-frame prediction process may be executed in an encoding apparatus and a decoding apparatus. In this document, the encoding apparatus may include an encoding apparatus and / or a decoding apparatus.

[0244] Reference FIG. 13 The encoding device determines the inter-frame prediction mode for the current block (S800). Various inter-frame prediction modes can be used for the prediction of the current block in the image. For example, various modes can be used, such as merging mode, skipping mode, MVP (Motion Vector Prediction) mode, affine mode, sub-block merging mode, and MMVD (Merging with MVD) mode. DMVR (Decoder-Side Motion Vector Refinement) mode, AMVR (Adaptive Motion Vector Resolution) mode, BCW (Bidirectional Prediction with CU-Level Weights), BDOF (Bidirectional Optical Flow), etc., can be used as additional modes. Furthermore, according to one embodiment of this disclosure, the above-mentioned inter-frame prediction mode may include a multi-hypothesis prediction (MHP) mode. The multi-hypothesis prediction mode indicates a method of performing prediction by weighted summation of an additional prediction block generated based on additional motion information and the inter-frame prediction block. The multi-hypothesis prediction mode will be described in detail later.

[0245] In this disclosure, the affine pattern can also be referred to as the affine motion prediction pattern. Additionally, the MVP pattern can also be referred to as the AMVP (Advanced Motion Vector Prediction) pattern. In this disclosure, some patterns and / or motion information candidates derived from some patterns can be included as one of the motion information related candidates for another pattern. For example, HMVP candidates can be additionally included as merge candidates for merge / skip patterns, or they can also be additionally included as motion vector predictor candidates for AMVP patterns. When an HMVP candidate is used as a motion information candidate for a merge or skip pattern, the HMVP candidate can be referred to as an HMVP merge candidate.

[0246] Prediction mode information, indicating the inter-frame prediction mode of the current block, can be signaled from the encoding device to the decoding device. The prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information may include index information indicating one of several candidate modes. Alternatively, the inter-frame prediction mode can also be indicated by hierarchical signaling of flag information.

[0247] In this context, the prediction pattern information may include one or more flags. For example, a skip flag may be signaled to indicate whether a skip pattern is applied, a merge flag may be signaled to indicate whether a merge pattern is applied when a skip pattern is not applied, and when a merge pattern is not applied, it may indicate the application of the MVP pattern, or further signal flags for additional differentiation. Affine patterns may be signaled as independent patterns or as patterns dependent on either the merge or MVP patterns. For example, affine patterns may include an affine merge pattern and an affine MVP pattern.

[0248] The encoding device can derive motion information for the current block (S810). The motion information can be derived based on the inter-frame prediction mode determined in the above steps. The encoding device can perform inter-frame prediction using the motion information of the current block. The encoding device can derive the optimal motion information for the current block through the motion estimation process.

[0249] For example, an encoding device can search for highly correlated similar reference blocks in a predetermined search range within a reference image, using original blocks from the original image for the current block, on a fractional pixel basis, and can derive motion information from them. Block similarity can be derived based on the difference in phase-based sample values. For example, block similarity can be calculated based on the SAD (Self-Average Difference) between the current block (or its template) and a reference block (or its template). In this case, motion information can be derived based on the reference block with the minimum SAD within the search range. The derived motion information can be signaled to the decoding device in various ways based on inter-frame prediction modes.

[0250] The encoding device can perform inter-frame prediction based on the motion information of the current block to generate prediction samples (S820). The current block that includes the prediction samples can be called the prediction block.

[0251] Simultaneously, a signal can be sent indicating whether the aforementioned List 0 (L0) prediction, List 1 (L1) prediction, or bidirectional prediction is used for the current block (current coding unit). Such information can be referred to as motion prediction direction information, inter-frame prediction direction information, or inter-frame prediction indication information, and can be configured / encoded / signaled, for example, in the form of the `inter_pred_idc` syntax element. That is, the `inter_pred_idc` syntax element can indicate whether the aforementioned List 0 (L0) prediction, List 1 (L1) prediction, or bidirectional prediction is used for the current block (current coding unit). In this document, for ease of description, the inter-frame prediction type (L0 prediction, L1 prediction, or BI prediction) indicated by the `inter_pred_idc` syntax element can be indicated as the motion prediction direction. L0 prediction can also be represented as `pred_L0`, L1 prediction can also be represented as `pred_L1`, and bidirectional prediction can also be represented as `pred_BI`. For example, the prediction type shown in Table 3 below can be indicated based on the value of the `inter_pred_idc` syntax element.

[0252] [Table 3]

[0253] As described above, an image can include one or more slices. Slices can have one of the following slice types: intra-frame (I) slices, prediction (P) slices, and bidirectional prediction (B) slices. Such slice types can be indicated based on slice type information. For blocks within an I slice, prediction does not use inter-frame prediction and can only use intra-frame prediction. Of course, even in this case, the original sample values ​​can be encoded and signaled without prediction. For blocks within a P slice, either intra-frame or inter-frame prediction can be used, and when inter-frame prediction is used, only unidirectional prediction can be used. Similarly, for blocks within a B slice, either intra-frame or inter-frame prediction can be used, and when inter-frame prediction is used, at most bidirectional prediction can be used.

[0254] L0 and L1 can include reference images encoded / decoded before the current image. For example, L0 can include reference images that are before and / or after the current image in POC order, and L1 can include reference images that are after and / or before the current image in POC order. In this case, in L0, a relatively lower reference image index can be assigned to a reference image that is before the current image in POC order, and in L1, a relatively lower reference image index can be assigned to a reference image that is after the current image in POC order. In the case of B-slices, bidirectional prediction can be applied, and even in this case, unidirectional bidirectional prediction or bidirectional bidirectional prediction can be applied. Bidirectional bidirectional prediction can be referred to as true bidirectional prediction.

[0255] Inter-frame prediction can be performed using motion information from the current block. The encoding device can derive the optimal motion information for the current block through a motion estimation process. For example, the encoding device can search for highly correlated similar reference blocks in a predetermined search range within a reference image, using original blocks from the original image for the current block, on a fractional-pixel basis, and derive motion information from them. Block similarity can be derived based on the difference in phase-based sample values. For example, block similarity can be calculated based on the SAD between the current block (or its template) and a reference block (or its template). In this case, motion information can be derived based on the reference block with the minimum SAD within the search area. The derived motion information can be signaled to the decoding device based on the inter-frame prediction mode according to various methods.

[0256] Merge mode and skip mode

[0257] When the merge mode is applied, the motion information of the currently predicted block is not sent directly; instead, it is inferred by using the motion information of neighboring predicted blocks. Therefore, the motion information of the currently predicted block can be indicated by sending a flag indicating the use of the merge mode and a merge index indicating which neighboring predicted block to use. The merge mode can also be called the regular merge mode.

[0258] To perform the merging mode, the encoder can search for candidate blocks to merge, which are used to derive motion information for the current predicted block. For example, up to five candidate blocks can be used, but this disclosure is not limited to this. Additionally, a maximum number of candidate blocks can be sent in the slice header or tile group header, but this disclosure is not limited to this. After finding candidate blocks, the encoder can generate a list of candidate blocks and select the one with the lowest cost from among them as the final candidate block to merge.

[0259] FIG. 14 An example of a block used to configure a list of candidates to be merged.

[0260] This disclosure provides various implementations of merge candidate blocks for configuring a merge candidate list.

[0261] The merge candidate list can, for example, use five merge candidate blocks. For instance, four spatial merge candidates and one temporal merge candidate could be used. As a concrete example, in the case of spatial merge candidates, FIG. 14 The blocks illustrated below can be used as spatial merge candidates. In the following text, spatial merge candidates, or spatial MVP candidates as described later, may be referred to as SMVPs, and temporal merge candidates, or temporal MVP candidates as described later, may be referred to as TMVPs.

[0262] The list of merge candidates for the current block can be configured, for example, based on the following process: - Insert spatial merge candidates derived from search space neighbor blocks into the merge candidate list. - Insert time-merging candidates derived by searching time neighbor blocks into the merge candidate list. - Compare the current number of merge candidates with the maximum number of merge candidates. - When the current number of merge candidates is less than the maximum number of merge candidates, add additional merge candidates to the merge candidate list. The above process describes how the encoding device (encoder / decoder) inserts spatial merging candidates derived by searching the spatial neighbor blocks of the current block into the merging candidate list. For example, spatial neighbor blocks can include the neighbor blocks around the lower left corner, the left neighbor block, the neighbor blocks around the upper right corner, the upper neighbor block, and the neighbor blocks around the upper left corner of the current block. However, this is just an example, and in addition to the spatial neighbor blocks mentioned above, supplementary neighbor blocks such as the right neighbor block, the lower neighbor block, and the lower right corner neighbor block can also be used as spatial neighbor blocks. The encoding device can search spatial neighbor blocks based on priority to detect available blocks and can derive spatial merging candidates from the motion information of the detected blocks. For example, the encoder and decoder can search in the order of A1, B1, B0, A0, and B2. FIG. 14 The example shows five blocks, and the available candidates can be indexed sequentially to configure the list of merged candidates.

[0263] The encoding device inserts temporal merge candidates derived by searching temporal neighboring blocks of a current block into a merge candidate list. The temporal neighboring blocks may be located on a reference picture that is a picture different from the current picture in which the current block is located. The reference picture on which the temporal neighboring blocks are located may be referred to as a collocated picture or a col picture. The temporal neighboring blocks may be searched on the col picture in the order of neighboring blocks and a lower-right center block around the lower-right corner of the collocated block for the current block.

[0264] Meanwhile, when motion data compression is applied, specific motion information may be stored as representative motion information for each predetermined storage unit in a collocated picture. In this case, it is not necessary to store motion information for all blocks within the predetermined storage unit, and a motion data compression effect can be obtained thereby. In this case, the predetermined storage unit may be predefined, for example, in units of 16×16 samples or 8×8 samples, or information about the size of the predetermined storage unit may be signaled from an encoder to a decoder. When motion data compression is applied, the motion information of the temporal neighboring blocks may be replaced with the representative motion information of the predetermined storage unit in which the temporal neighboring blocks are located. That is, in this case, from an implementation aspect, the temporal merge candidates may be derived based on the motion information of a prediction block that covers a position obtained by arithmetically right-shifting the coordinates of the temporal neighboring blocks by a predetermined value and then arithmetically left-shifting the value (instead of the prediction block located at the coordinates of the temporal neighboring blocks). For example, when the predetermined storage unit is in units of 2 n ×2 n samples, if the coordinates of the temporal neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the corrected position (((xTnb>>n)<<n), ((yTnb>>n)<<n)) may be used for the temporal merge candidates. Specifically, for example, when the predetermined storage unit is in units of 16×16 samples, if the coordinates of the temporal neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the corrected position (((xTnb>>4)<<4), ((yTnb>>4)<<4)) may be used for the temporal merge candidates. Alternatively, for example, when the predetermined storage unit is in units of 8×8 samples, if the coordinates of the temporal neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the corrected position (((xTnb>>3)<<3),((yTnb>>3)<<3)) may be used for the temporal merge candidates.

[0265] The encoding device can compare the current number of merge candidates with the maximum number of merge candidates. The maximum number of merge candidates can be predefined or signaled to the decoder from the encoder. For example, the encoder can generate information about the maximum number of merge candidates, encode it, and send it to the decoder as a bitstream. When the maximum number of merge candidates is full, the subsequent candidate addition process can be skipped.

[0266] When the number of current merge candidates is less than the maximum number of merge candidates as a result of comparison, the encoding device inserts additional merge candidates into the merge candidate list. Additional merge candidates may include at least one of the following: history-based merge candidates, pairwise average merge candidates, ATMVP, combined bidirectional prediction merge candidates (when the current slice / tile group's slice / tile group type is type B), and / or zero-vector merge candidates, which will be described later.

[0267] When the current number of merge candidates is not less than the maximum number of merge candidates as a result of comparison, the encoding device can terminate the configuration of the merge candidate list. In this case, the encoder can select the optimal merge candidate from the merge candidates in the configured merge candidate list based on the RD (rate-distortion) cost, and can signal selection information (e.g., merge index) indicating the selected merge candidate to the decoder. The decoder can select the optimal merge candidate based on the merge candidate list and the selection information.

[0268] As described above, the motion information of the selected merging candidate can be used as the motion information of the current block, and the predicted sample of the current block can be derived based on the motion information of the current block. The encoder can derive the residual sample of the current block based on the predicted sample, and can encode residual information about the residual sample and send it to the decoder. As described above, the decoder can generate reconstructed samples based on the residual samples derived based on the sent residual information and the predicted samples, and can generate a reconstructed image based on them.

[0269] When the skip mode is applied, the motion information of the current block can be derived in the same way as when the merge mode is applied. However, when the skip mode is applied, the residual signal of the corresponding block is omitted, and therefore the predicted sample can be directly used as the reconstructed sample.

[0270] Merge mode with MVD (MMVD)

[0271] The MMVD modes mentioned above will be described. Besides the merging mode where implicitly derived motion information is directly used for the predicted samples of the current block, a merging mode using motion vector differences (MMVD) can be used. MMVD can also be applied to the skip mode because it uses a similar motion information derivation method as the skip and merging modes. After signaling the skip and merging flags, MMVD flag information (e.g., mmvd_flag) can be signaled to indicate whether to use the MMVD mode for the current block.

[0272] In MMVD, after selecting a merge candidate, that candidate can be further refined based on the MVD information signaled. When MMVD is applied to the current block (i.e., when mmvd_flag is 1), additional information about MMVD can be signaled. This additional information may include a merge candidate flag (e.g., mmvd_merge_flag) indicating whether the first or second candidate in the merge candidate list is used with the motion vector difference, a distance index indicating the motion magnitude (e.g., mmvd_distance_idx), and a direction index indicating the motion direction (e.g., mmvd_direction_idx). In MMVD mode, one of the top two candidates in the merge list can be selected as the MV basis. The merge candidate flag is signaled to indicate the candidate to be used.

[0273] The distance index indicates motion amplitude information and a predefined offset from the starting point. The offset is added to the horizontal or vertical component of the initial MV. The relationship between the distance index and the predefined offset can be represented by Table 4 below.

[0274] [Table 4]

[0275] Here, when slice_fpel_mmvd_enabled_flag equals 1, it indicates that the merge mode with motion vector difference uses integer sample precision in the current slice. When slice_fpel_mmvd_enabled_flag equals 0, it indicates that the merge mode with motion vector difference can use fractional sample precision in the current slice. The slice_fpel_mmvd_enabled_flag syntax element is signaled through the slice header, or it can be included in the slice header.

[0276] The direction index indicates the MVD direction relative to the starting point. The direction index can indicate one of the four directions shown in Table 5 below. The meaning of the MVD symbol can vary depending on the information of the starting MV. When the starting MV is a non-predictive MV or a bidirectional MV where both lists point to the same side of the current image (i.e., when both reference POCs are greater than or less than the current image's POC), the symbols in Table 5 below can indicate the sign of the MV offset added to the starting MV. When the starting MV is a bidirectional predictive MV and the two MVs point to different sides of the current image (i.e., when one reference POC is greater than the current image's POC and the other reference POC is less than the current image's POC), the symbols in Table 5 below indicate the sign of the MV offset added to the list 0 MV component of the starting MV, and the signs of the list 1 MVs have the opposite values.

[0277] [Table 5]

[0278] The two elements of merging and adding MVD offset MmvdOffset[x0][y0] can be derived as follows.

[0279]

[0280] Motion vector prediction (MVP)

[0281] The MVP (Motion Vector Prediction) pattern can also be called the AMVP (Advanced Motion Vector Prediction) pattern. When applying the MVP pattern, an MVP (Motion Vector Predictor) candidate list can be generated using the motion vectors of reconstructed spatial neighbor blocks and / or the motion vectors corresponding to temporal neighbor blocks (or sibling blocks). That is, the motion vectors of reconstructed spatial neighbor blocks and / or the motion vectors corresponding to temporal neighbor blocks can be used as motion vector predictor candidates. When applying bidirectional prediction, the MVP candidate list for deriving L0 motion information and the MVP candidate list for deriving L1 motion information can be generated and used separately.

[0282] The aforementioned prediction information (or information about prediction) may include selection information (e.g., an MVP flag or MVP index) indicating the optimal motion vector predictor candidate to be selected from the list of motion vector predictor candidates. In this case, the predictor can select the motion vector predictor for the current block from the motion vector predictor candidates included in the list of motion vector candidates by using the selection information.

[0283] The predictor of the encoding device can obtain the motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and can encode it and output it as a bitstream. That is, the MVD can be obtained as the value obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the predictor of the decoding device can obtain the motion vector difference included in the prediction information, and can deduce the motion vector of the current block by adding the motion vector difference to the motion vector predictor. The predictor of the decoding device can obtain or deduce the reference image index indicating the reference image from the prediction information. For example, the motion vector predictor candidate list can be configured as follows: - Search for spatial candidate blocks for motion vector prediction and insert them into the prediction candidate list. - Check if the number of space candidate blocks is less than 2 - If the number of spatial candidate blocks is less than 2, search for temporal candidate blocks and append them to the prediction candidate list. - If time candidate blocks are unavailable, use the zero motion vector. - If the number of spatial candidate blocks is not less than 2, then terminate the configuration of the motion vector predictor candidate list. Furthermore, when applying the MVP pattern, the reference image index can be explicitly signaled. In this case, the reference image index (refidxL0) used for L0 prediction and the reference image index (refidxL1) used for L1 prediction can be signaled separately. For example, when applying the MVP pattern and applying bidirectional prediction, both information about refidxL0 and information about refidxL1 can be signaled.

[0284] MVD (Motion Vector Difference) coding

[0285] When the MVP pattern is applied, information about the MVD derived in the encoding device as described above can be signaled or encoded and sent to the decoding device. Information about the MVD may include, for example, information representing the x and y components of the absolute value of the MVD and their signs. In this case, information such as whether the absolute value of the MVD is greater than 0, whether the absolute value of the MVD is greater than 1, and the remainder of the MVD can be signaled incrementally. For example, information indicating whether the absolute value of the MVD is greater than 1 can only be signaled when the value of the flag indicating whether the absolute value of the MVD is greater than 0 is equal to 1.

[0286] For example, information about MVD can be configured using the following syntax, and can be encoded in the encoding device and signaled to the decoding device.

[0287] [Table 6]

[0288] For example, MVD[compIdx] can be based on abs_mvd_greater0_flag[compIdx] ( abs_mvd_minus2[compIdx] + 2 ) (1 - 2) The expression `mvd_sign_flag[compIdx]` is used to deduce the value. Here, `compIdx` (or `cpIdx`) represents the index of each component and can have a value of 0 or 1. A value of `compIdx` equal to 0 can represent the x-component, and a value of `compIdx` equal to 1 can represent the y-component. However, this is just an example, and the values ​​of the corresponding components can be represented using coordinate systems other than the x and y coordinate systems.

[0289] Simultaneously, the MVD (MVDL0) used for L0 prediction and the MVD (MVDL1) used for L1 prediction can be signaled separately, and information about MVD can include information about MVDL0 and / or information about MVDL1. For example, when the MVP mode is applied to the current block and bidirectional prediction is applied, both information about MVDL0 and information about MVDL1 can be signaled.

[0290] Affine prediction

[0291] Conventional video coding systems use only one motion vector to represent the motion of a coded block (i.e., using a translational motion model). While this approach can represent optimal motion at the block level, it is not optimal for every pixel. Determining the optimal motion vector at the pixel level would improve coding efficiency. To this end, an affine motion prediction method using an affine motion model for coding will be described.

[0292] FIG. 15 Examples of four types of motion that can be represented in an affine motion model are given.

[0293] Affine motion prediction methods can represent pixel-level motion vectors of a block using two, three, or four motion vectors.

[0294] like FIG. 15 As shown, affine motion models can represent four types of motion. Among the motions that can be represented by affine motion models, those representing three types of motion (translation, scaling, and rotation) are called similar (or simplified) affine motion models, and the proposed method will be described below based on similar affine motion models. However, the disclosed implementations are not limited to the corresponding motion models.

[0295] FIG. 16 An example of a control point motion vector used in affine motion prediction is shown.

[0296] like FIG. 16 As shown, affine motion prediction can determine the motion vectors of the pixel positions included in a block by using two or more control point motion vectors (CPMVs). In this case, a set of motion vectors is called an affine motion vector field (MVF) and can be determined by the following formula.

[0297] For a 4-parameter affine motion model, the motion vector at the sample position (x, y) of the block can be derived according to Equation 2 below.

[0298] [Equation 2]

[0299] For a 6-parameter affine motion model, the motion vector at the sample position (x, y) of the block can be derived according to Equation 3 below.

[0300] [Formula 3]

[0301] Here, It is the CPMV of the CP at the top left corner of the coded block. It is the CPMV of the CP in the upper right corner, and It is the CPMV of the CP at the bottom left corner. Additionally, W corresponds to the width of the current block, H corresponds to the height of the current block, and... It is the motion vector at position {x, y}.

[0302] During the encoding / decoding process, the affine MVF can be determined based on pixels or predefined sub-blocks. When determined based on pixels, the motion vector is obtained based on each pixel value; when determined based on sub-blocks, the block's motion vector is obtained based on the pixel value at the center of the sub-block (i.e., the bottom right sample among the four center samples). For example, as... FIG. 16 As shown, the affine MVF can be determined based on 4×4 sub-blocks. However, this is just an example, and the size of the sub-blocks applied to the affine prediction can be modified in various ways.

[0303] When affine prediction is available, the motion model applicable to the current block can include three models: translational motion model, 4-parameter affine motion model, and 6-parameter affine motion model. Here, the translational motion model can represent a model using regular block-level motion vectors, the 4-parameter affine motion model can represent a model using two CPMVs, and the 6-parameter affine motion model can represent a model using three CPMVs.

[0304] Affine motion prediction can include affine MVP (or affine inter-frame) mode and affine merging mode. In affine motion prediction, the motion vector of the current block can be derived based on samples or sub-blocks.

[0305] Affine merge

[0306] In affine merging mode, CPMV can be determined based on the affine motion model of neighboring blocks encoded using affine motion prediction. Affine-encoded neighboring blocks can be used in affine merging mode according to the search order. When one or more neighboring blocks are encoded using affine motion prediction, the current block can be encoded using affine merging.

[0307] That is, when applying affine merge mode, the CPMV of the current block can be derived using the CPMV of neighboring blocks. In this case, the CPMV of the neighboring blocks can be directly used as the CPMV of the current block, or the CPMV of the neighboring blocks can be modified based on the size of the neighboring blocks and the size of the current block and used as the CPMV of the current block.

[0308] Meanwhile, in the case of affine merging based on subblock derivation of MV, this pattern can be called the subblock merging pattern, and can be indicated based on merge_subblock_flag (value 1). In this case, the affine merging candidate list, described later, can also be called the subblock merging candidate list. In this case, the subblock merging candidate list can further include candidates derived through SbTMVP. In this case, candidates derived through SbTMVP can be used as the candidate with index 0 in the subblock merging candidate list. In other words, candidates derived through SbTMVP can be located within the subblock merging candidate list before the inherited affine candidates and constructed affine candidates, described later.

[0309] When applying the affine merge pattern, an affine merge candidate list can be constructed to derive the CPMV of the current block. The affine merge candidate list can include at least one of the following candidates: 1) Inherited affine candidates 2) Constructing affine candidates 3) Zero MV candidates Here, inherited affine candidates are candidates derived from the CPMV of neighboring blocks when neighboring blocks are encoded in affine mode. Constructing affine candidates involves constructing candidates derived from the CPMV of neighboring blocks around each CP, and a zero-MV candidate can represent a candidate consisting of CPMV values ​​of 0. For example, a zero-MV candidate can be optionally inserted into the candidate list when the current number of candidates is less than the maximum number of candidates.

[0310] FIG. 16An example illustrating the inheritance of control point motion vectors, and FIG. 17 An example of constructing candidate positions in an affine merge pattern.

[0311] Two affine candidates for inheritance can be derived from the affine motion model of the neighboring blocks. One of them can be derived from the left neighbor CU, and the other can be derived from the upper CU.

[0312] Refer to the above again FIG. 14 For the left predictor, the scan order is A0->A1, and for the top predictor, the scan order is B0->B1->B2. Only the first inheritance candidate on each side can be selected, and no pruning check is performed between the two inheritance candidates.

[0313] When neighboring affine CUs are identified, the control point motion vectors can be used to derive CPMVP candidates from the affine merging list of the current CU. (See reference...) FIG. 16 When the neighboring lower-left block A is encoded in affine mode, motion vectors v2, v3, and v4 at the top-left, top-right, and bottom-left corners of the CU including block A can be obtained. When block A is encoded using a 4-parameter affine model, the two CPMVs of the current CU can be calculated based on v2 and v3. When block A is encoded using a 6-parameter affine model, the three CPMVs of the current CU can be calculated based on v2, v3, and v4.

[0314] FIG. 18 An example of a neighboring block of the current block.

[0315] Constructing affine candidates can refer to constructing candidates by combining neighboring translational motion information around each control point. The motion information for each control point can be derived from, for example... FIG. 18 The derivation is based on the specified spatial neighbor blocks and temporal neighbor blocks shown.

[0316] CPMV k (k=1, 2, 3, 4) represents the k-th control point. For CPMV1, blocks are checked in the order B2->B3->A2, and the MV of the first available block is used. For CPMV2, blocks are checked in the order B1->B0, and for CPMV3, blocks are checked in the order A1->A0. When available, TMVP can be used as CPMV4.

[0317] After obtaining the motion vectors of the four control points, affine merging candidates can be constructed based on the motion information. The following combinations of control point MVs can be used sequentially for construction: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}.

[0318] Combining three CPMVs constructs a 6-parameter affine merge candidate, and combining two CPMVs constructs a 4-parameter affine merge candidate. To omit the motion scaling process, combinations of corresponding control point MVs are not used when the reference indices of the control points are different.

[0319] DMVR (Decoder-side Motion Vector Refinement)

[0320] DMVR (Decoder-Side Motion Vector Refinement) is a method of performing motion prediction on the decoder side by refining the motion information of neighboring blocks. When DMVR is applied, the decoder can derive refined motion information in merge / skip modes by cost comparison based on templates generated using the motion information of neighboring blocks. By doing so, the accuracy of motion prediction can be improved and compression performance can be enhanced without additional signaling information.

[0321] In this disclosure, the decoder is described primarily for ease of explanation, but DMVR can also be implemented in the encoder in the same way.

[0322] Meanwhile, in terms of refining motion vectors, DMVR can be regarded as an example of the motion information derivation process, and in terms of generating prediction samples based on reference blocks indicated by refined MV pairs, it can also be regarded as an example of the prediction sample generation process.

[0323] During bidirectional prediction, a refined MV can be searched around the initial MV in the reference image lists L0 and L1. The BM method calculates the distortion between two candidate blocks from the reference image lists L0 and L1.

[0324] SAD is calculated for each MV candidate within the search range surrounding the initial MV. The MV candidate with the lowest SAD becomes the refined MV and is used to generate the BCW signal.

[0325] In one implementation, the decoder may perform a DMVR procedure to improve the accuracy of the initial motion compensation prediction (i.e., motion compensation prediction using a regular merge / skip mode). For example, the decoder may perform a DMVR procedure when the prediction mode for the current block is merge or skip mode and a BCW (Browser-Based Wrapper) is applied to the current block with the reference image in the opposite direction relative to the current image in display order.

[0326] For example, DMVR can be applied to CUs encoded according to the following modes and conditions: -Use BCWMV's CU-level merge mode - Relative to the current image, one reference image is a past image and the other is a future image. - The distances (i.e., the difference in point of view) from the two reference images to the current image are the same. -CU includes more than 64 luminance samples - Both CU height and CU width are greater than or equal to 8 luminance samples. -BCW weight index indicates the same weight -WP is not available for the current block. The refined MV derived through the DMVR process is used to generate inter-frame prediction samples and can also be used for temporal motion vector prediction in the encoding of the next image. The original MV is used in the deblocking process and can also be used for spatial motion vector prediction in the encoding of subsequent CUs.

[0327] GPM (Geometry Partition Mode)

[0328] In VVC, GPM (Gross Frame Prediction) can be supported for inter-frame prediction. GPM is a type of merging mode and can be signaled using CU-level flags. Other merging modes can include regular merging mode, MMVD mode, CIIP mode, and sub-block merging mode. For each possible CU size... It can support a total of 64 partition types, among which And exclude 8×64 and 64×8.

[0329] When using this mode, the CU can be divided into two parts by a geometrically positioned straight line. The position of the partition line can be mathematically derived from the angle and offset parameters of a specific partition. Each part of the geometric partition of the CU can be used for inter-frame prediction using its own motion. Only unidirectional prediction is allowed per partition; that is, each part has one MV and one reference index. Unidirectional prediction motion constraints can be applied to ensure that each CU requires only two motion-compensated predictions, similar to regular bidirectional prediction.

[0330] FIG. 19 Example of GPM partitions grouped at the same angle.

[0331] Unidirectional predicted motion for each partition can be used FIG. 19 The process is illustrated in the example to derive the result.

[0332] When GPM is used for the current block, the geometric partition index representing the partitioning pattern (angle and offset) and two merge indices (one per partition) can be additionally signaled. The maximum number of GPM candidate sizes is explicitly signaled in the sequence parameter set, and the syntax binarization of the GPM merge index can be specified. After each part of the geometric partition is predicted, the sample values ​​can be adjusted using a blending process with adaptive weights along the edges of the geometric partition, as described below. This becomes the prediction signal for the entire CU, and the transformation and quantization processes can be applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using GPM can be stored in a motion field storage device for the geometric partitioning mode.

[0333] CIIP (Combined Intra Inter Prediction)

[0334] Combined Intra-Out-of-Frame Prediction (CIIP) can be applied to the current block. Additional flags (e.g., ciip_flag) can be signaled to indicate whether CIIP mode is applied to the current CU. For example, when the CU is encoded in merged mode, if the CU includes at least 64 luma samples (i.e., the product of the CU width and CU height is greater than or equal to 64) and both the CU width and CU height are less than 128 luma samples, an additional flag can be signaled to indicate whether CIIP mode is applied to the current CU.

[0335] CIIP prediction combines inter-frame prediction signals and intra-frame prediction signals. The inter-frame prediction signal (P_inter) in CIIP mode can be derived using the same inter-frame prediction procedure applied to regular combining mode, and the intra-frame prediction signal (P_intra) can be derived using the regular intra-frame prediction procedure in planar mode. The intra-frame and inter-frame prediction signals can then be combined using a weighted average. FIG. 20 This example illustrates the top-left neighbor block used in CIIP weight derivation. Here, the weight values ​​can be calculated based on the encoding pattern of the top-left neighbor block as follows: - If the upper neighbor is available and intra-coded, isIntraTop is set to 1; otherwise, isIntraTop is set to 0. - If the left neighbor is available and intra-coded, isIntraLeft is set to 1; otherwise, isIntraLeft is set to 0. - If (isIntraTop + isIntraLeft) = 2, then wt is set to 3. Otherwise, if (isIntraTop + isIntraLeft) = 1, then wt is set to 2. Otherwise, wt is set to 1. CIIP predictions can be configured according to Equation 4 below.

[0336] [Formula 4]

[0337] GPM including intra and inter prediction

[0338] In GPM, which includes intra-frame and inter-frame prediction, the final prediction samples can be generated by weighting the inter-frame and intra-frame prediction samples for each region separated by GPM. Inter-frame prediction samples are derived from inter-frame GPM, while intra-frame prediction samples can be derived from the intra-frame prediction mode (IPM) candidate list and the index signaled from the encoder. The size of the IPM candidate list can be predefined as 3.

[0339] FIG. 21 Examples of available IPM candidates for GPM including intra-frame and inter-frame prediction are shown.

[0340] Available IPM candidates can be parallel angle modes (parallel mode), perpendicular angle modes (perpendicular mode), and planar modes relative to the GPM block boundary, such as... FIG. 21 Examples are given in (a), (b), and (c). Furthermore, as... FIG. 21 As illustrated in (d), the GPM, including intra-frame and inter-frame prediction, can be limited to reduce signaling overhead for IPM and prevent an increase in the size of the intra-frame prediction circuitry in the hardware decoder. Additionally, direct motion vectors and IPM storage for the GPM mixing region can be introduced to further improve coding performance.

[0341] In IPM derivation based on DIMD and neighboring modes, parallel modes can be registered first. Therefore, when no identical IPM candidates exist in the list, up to two IPM candidates can be registered using the decoder-side intra-frame mode derivation (DIMD) method and / or derivation from neighboring blocks.

[0342] In the deduced neighbor pattern, there are up to five locations for available neighbor blocks, but these can be limited by the GPM block boundary angles already used in a GPM that uses template matching.

[0343] Table 7 illustrates the locations of available neighboring blocks used to derive IPM candidates based on GPM block boundary angles. A and L can represent the top and left sides of the predicted block.

[0344] [Table 7]

[0345] GPM frames can be combined with GPM merge modes that have motion vector differences (GPM-MMVD). To further improve coding performance, TIMD can be used as an IPM candidate within GPM frames. Parallel modes can be registered first, followed by TIMD, DIMD, and IPM candidates from neighboring blocks.

[0346] MHP (Multi-Hypothesis Prediction)

[0347] In multi-hypothesis inter-frame prediction mode, in addition to the regular bidirectional prediction signal, one or more additional motion compensation prediction signals are signaled, and the resulting overall prediction signal can be obtained by weighted summation of samples. This is achieved by using bidirectional prediction signals. and the first additional inter-frame prediction signal / hypothesis The obtained prediction signal The following can be obtained.

[0348] [Formula 5]

[0349] The weighting factor α can be specified by the new syntax element add_hyp_weight_idx according to the mapping shown in Table 8 below.

[0350] [Table 8]

[0351] Similarly, one or more additional prediction signals can be used. The resulting overall prediction signal can be iteratively accumulated together with each additional signal according to Equation 6 below.

[0352] [Formula 6]

[0353] The resulting overall prediction signal can be obtained as the final (p_n) (i.e., (p_n) with the largest index (n)). For example, at most two additional prediction signals can be used (i.e., (n) is limited to 2).

[0354] The motion parameters of each additional prediction hypothesis can be explicitly signaled by specifying a reference index, a motion vector prediction index, and a motion vector difference, or implicitly signaled by specifying a merging index. A separate multi-hypothesis merging flag can distinguish between these two signaling modes.

[0355] For inter-frame AMVP mode, MHP can only be applied if unequal weights of BCW are selected in bidirectional prediction mode.

[0356] The combination of MHP and BDOF is feasible, but BDOF can only be applied to the bidirectional signal portion of the predicted signal (i.e., the two general first assumptions).

[0357] OBMC (Overlapped Block Motion Compensation)

[0358] OBMC can be performed on all motion-compensated block boundaries within the right and bottom boundaries of a CU. Therefore, it can be applied to both the luma and chroma components. To handle CU / subblock boundaries uniformly, OBMC can be performed at the subblock level on all MC (motion-compensated) block boundaries.

[0359] FIG. 22 Examples of sub-blocks at the CU / PU boundary and sub-PUs in ATMVP mode are shown. Here, the size of the sub-block can be set to 4×4.

[0360] When OBMC is applied to the current sub-block, in addition to the current motion vector, four connected neighboring sub-blocks that are available and different from the current motion vector can also be used to derive the prediction block for the current sub-block. These multiple prediction blocks based on multiple motion vectors can be weighted to generate the final prediction signal for the current sub-block.

[0361] The predicted block based on the motion vectors of neighboring sub-blocks is represented as P. N (where N represents the index of the neighboring sub-blocks in the up, down, left, and right directions), and the predicted block based on the motion vector of the current block is represented as P. C When P N Belongs to P C When using the same PU (because they contain the same motion information), it is not necessary to use P. N Execute OBMC. Otherwise, P N All pixels can be added to P C The corresponding pixel. That is, P N Four rows / columns can be added to P C The weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} can be used for P. N Furthermore, the weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} can be used for P. C The exception is small MC blocks, where only P... N Two rows / columns are added to P C (That is, when the CU size is 8×4 or 4×8, or when the CU is encoded in sbTMVP mode). In this case, the weighting factor {1 / 4, 1 / 8} can be used for P. N Furthermore, the weighting factors {3 / 4, 7 / 8} can be used for P. CFor P generated based on the motion vectors of its vertical (or horizontal) neighboring sub-blocks N Located in P N Pixels in the same row (or column) can be added to P with the same weighting factor. C .

[0362] LIC (Local Illumination Compensation)

[0363] Inter-frame prediction (LIC) is a technique used to model the local illumination change between the current block and the predicted block as a variation function between the current block template and the reference block template. The parameters of this function can be expressed as a scaling factor α and an offset β forming a linear equation. That is, the illumination change can be modeled as a function of the change between the current block template and the reference block template. To compensate, p[x] is the reference sample pointed to by the MV at position x of the reference image. When surround motion compensation is enabled, surround offset can be considered to crop the MV. Since α and β can be derived based on the current block template and the reference block template, no signaling overhead is required for them except for signaling the LIC flag in AMVP mode to indicate the use of LIC.

[0364] LIC can be used in inter-frame CUs with the following characteristics: Intra-frame neighbor samples can be used for LIC parameter derivation. - For blocks with fewer than 32 luminance samples, LIC can be disabled. - For both non-subblock mode and affine mode, LIC parameter derivation can be performed based on the template block sample corresponding to the current CU, rather than the partial template block sample corresponding to the first top-left 16×16 cell. - Samples of the reference block template can be generated using block MV and MC without rounding them to integer pixel resolution (precision).

[0365] For bidirectional inter-frame prediction CU, two sets of LIC parameters can be derived separately for L0 and L1 predicted samples. An iterative method can be applied to derive the L0 and L1 LIC parameters. Specifically, the L0 LIC parameters can first be derived by minimizing the difference between the L0 template prediction (T_0) and the template (T), and the samples of (T) can be updated by subtracting the corresponding samples of (T_0). Then, the L1 parameters can be calculated to minimize the difference between the L1 template prediction (T_1) and the updated template. Finally, the L0 parameters can be readjusted in the same manner.

[0366] According to this disclosure, when bidirectional prediction is applied to the current block, the predicted samples can be derived based on a weighted average. Conventionally, the bidirectional prediction signal (i.e., the bidirectional prediction sample) can be derived by a simple average of the L0 prediction signal (L0 prediction sample) and the L1 prediction signal (L1 prediction sample). That is, the bidirectional prediction sample is derived as the average of the L0 prediction sample based on the L0 reference image and MVL0 and the L1 prediction sample based on the L1 reference image and MVL1. However, according to this disclosure, when bidirectional prediction is applied, the bidirectional prediction signal (bidirectional prediction sample) can be derived by a weighted average of the L0 prediction signal and the L1 prediction signal according to Equation 7 below.

[0367] [Formula 7]

[0368] In weighted bidirectional prediction, five weights are allowed. For each bidirectional prediction CU, the weight w can be determined using one of the following two methods: 1) For non-merged CUs, the weight index can be signaled after the motion vector difference.

[0369] 2) For merging CUs, the weight index can be inferred from neighboring blocks based on the merge candidate index.

[0370] Weighted bidirectional prediction can be applied only to CUs with 256 or more luminance samples (i.e., CU width × CU height is greater than or equal to 256). For low-latency images, all five weights can be used, and for non-low-latency images, only three weights can be used. ).

[0371] In the encoder, fast search algorithms can be applied to find the weight indices without significantly increasing the encoder's complexity. These algorithms can be summarized as follows: - When combined with AMVR, it can conditionally check for unequal weights for 1-pixel and 4-pixel MV precision when the current image is a low-latency image.

[0372] - When combined with affine motion, affine motion estimation for unequal weights can only be performed if the affine mode is selected as the current best mode.

[0373] - When the two reference images in bidirectional prediction are the same, unequal weights can be conditionally checked.

[0374] -Based on the POC distance between the current image and the reference image, the QP encoding, and the time level, non-equal weights may not be searched when certain conditions are met.

[0375] The BCW weight index can be encoded using a context-coded bin and a bypass-coded bin. The first context-coded bin indicates whether equal weights are used, and when unequal weights are used, the bypass-coded bin signals the additional bin to indicate which unequal weight to use.

[0376] Weighted Prediction (WP) is an encoding tool used to efficiently encode video content in the presence of fade-ins and fade-outs. WP allows signaling of weight parameters (weights and offsets) for each reference picture in the reference picture lists L0 and L1. During motion compensation, the weights and offsets of the corresponding reference pictures can be applied.

[0377] WP and BCW weights can be designed for different types of video content. To avoid the mutual influence between WP and BCW weights that complicates the decoder design, when the CU uses WP, it does not signal the BCW weight index, and (w) can be inferred to be 4 (i.e., apply equal weights).

[0378] For merging CUs, the weight index can be inferred from neighboring blocks based on the merge candidate index. This method can be applied to both regular merge mode and inherited affine merge mode. In constructing the affine merge mode, affine motion information can be constructed based on the motion information of up to three blocks. The derivation of the BCW index of the CU using the constructed affine merge mode is as follows.

[0379] 1) The BCW index range {0,1,2,3,4} is divided into three groups: {0}, {1,2,3}, and {4}. When the BCW indices of all control points belong to the same group, the BCW index is derived according to step 2); otherwise, the BCW index is set to 2.

[0380] 2) When two or more control points have the same BCW index, the BCW index value is assigned to the candidate; otherwise, the BCW index of the currently constructed candidate is set to 2.

[0381] Bi-directional Optical Flow (BDOF)

[0382] Bidirectional optical flow (BDOF) can be used to improve bidirectional prediction signals. For example, BDOF can be applied at the 4×4 sub-block level and can be applied to CUs that meet the following conditions: - The height of the CU is not 4, and the CU size is not 4×8. -CU does not use affine mode or ATMVP merge mode for encoding. -CU uses a "true" bidirectional prediction mode for encoding (i.e., one of the two reference images is displayed earlier than the current image, and the other is displayed later than the current image). BDOF can be applied to the luminance component only, the chrominance component only, or both the luminance and chrominance components.

[0383] BDOF mode is based on the assumption that the motion of an object is a smooth optical flow. For each 4×4 sub-block, motion refinement can be calculated by minimizing the difference between the L0 and L1 prediction samples. Then, motion refinement can be used to adjust the bidirectional prediction sample values ​​of the 4×4 sub-blocks. The following steps can be applied to the BDOF process.

[0384] First, the horizontal and vertical gradients of two predicted samples can be obtained by directly calculating the difference between two neighboring samples according to Equations 8 and 9 below. and (k=0,1).

[0385] [Formula 8]

[0386] [Formula 9]

[0387] Here, Let represent the sample value of the predicted signal at coordinate (i,j) in list k (k=0,1), and shift1 can be calculated based on the brightness bit depth bitDepth. In addition, the autocorrelation and cross-correlation of the gradient S1, S2, S3, S5 and S6 can be calculated according to Equation 10 below.

[0388] [Formula 10]

[0389] Here, It can be calculated according to the following formula 11.

[0390] [Equation 11]

[0391] Ω represents a 6×6 window surrounding a 4×4 sub-block.

[0392] Motion refinement The following equation 12 can be used to derive it.

[0393] [Equation 12]

[0394] Here, , , ,and It is a floor function.

[0395] Based on motion refinement and gradients, adjustments can be calculated for each sample of a 4×4 sub-block according to Equation 13.

[0396] [Equation 13]

[0397] Finally, the BDOF samples of CU can be calculated by adjusting the bidirectional prediction samples according to Equation 14 below.

[0398] [Formula 14]

[0399] Above, n a n b and n s2 The values ​​are 3, 6, and 12, respectively. These values ​​are chosen so that the multiplier does not exceed 15 bits during the BDOF process, and the maximum bit width of the intermediate parameters of the BDOF process is maintained within 32 bits.

[0400] To derive the gradient values, a list of predicted samples k (k=0,1) outside the current CU boundary should be generated. . FIG. 23 An example of an extended CU region used in BDOF. See reference. FIG. 23 BDOF uses an extended row / column around the CU boundary. To control the computational complexity of generating prediction samples outside the boundary, a bilinear filter is used to generate prediction samples in the extended region, while a normal 8-tap motion-compensated interpolation filter is used to generate prediction samples within the CU. These extended sample values ​​are used only for gradient calculation. For the remaining steps of the BDOF process, whenever any samples and gradient values ​​outside the CU boundary are needed, they are filled from their nearest neighbor blocks (i.e., repeated).

[0401] When the width and / or height of a CU is greater than 16 luminance samples, the CU can be divided into sub-blocks with a width and / or height equal to 16 luminance samples, and the boundaries of the sub-blocks can be used as CU boundaries during the BDOF process. The maximum cell size for the BDOF process is limited to 16×16.

[0402] BDOF is not allowed when BCW weights are permitted for the current block, i.e., when the BCW weight index indicates unequal weights. Similarly, BDOF is not allowed when WP is permitted for the current block, i.e., when one of the two reference images has a luma_weight_lx_flag equal to 1. BDOF is also not allowed when the CU is encoded in symmetric MVD mode.

[0403] The Multiple Hypothesis Prediction (MHP) mode will be described in detail below. As mentioned above, the multiple hypothesis prediction mode refers to a method that uses additional prediction blocks (or predictors) in addition to regular prediction blocks. The multiple hypothesis prediction mode can be selectively used as one of the various inter-frame prediction modes described above. The multiple hypothesis prediction mode according to embodiments of this disclosure is not limited to this name. In this specification, the multiple hypothesis prediction mode may also be referred to as a multiple reference mode, multiple reference prediction, multiple reference prediction mode, multiple prediction block mode, MHP mode, multiple hypothesis inter-frame prediction mode, inter-frame combined prediction mode, combined inter-frame prediction mode, combined prediction mode, multiple inter-frame prediction mode, multiple prediction mode, additional reference prediction mode, additional reference mode, or multiple prediction block.

[0404] FIG. 24 Examples of methods for performing prediction by a decoding device according to embodiments of the present disclosure are illustrated, and FIG. 25 An example of a prediction block used in a multi-prediction block mode according to an embodiment of the present disclosure is illustrated. In this example, the prediction is described as being performed by a decoding device (300), but the same prediction could also be performed by an encoding device (200).

[0405] Reference FIG. 24 The decoding device can perform inter-frame prediction to generate regular prediction blocks (S900). This mode corresponds to a multi-prediction block mode when additional prediction blocks are generated in addition to those generated by unidirectional or bidirectional prediction in the general inter-frame prediction process. When applying the multi-prediction block mode, in addition to motion information in each direction during the inter-frame prediction process, additional motion information and weight information can be signaled or derived. For example, the decoding device can generate (or derive) additional prediction blocks in addition to those generated (or derived) by unidirectional or bidirectional prediction, and can combine them with each other.

[0406] To distinguish them from additional prediction blocks, prediction blocks generated (or derived) through unidirectional or bidirectional prediction supported in general inter-frame prediction will be called basic (regular) prediction blocks, and prediction blocks generated (or derived) according to the multi-prediction block mode will be called additional prediction blocks.

[0407] Additionally, in this disclosure, to distinguish prediction blocks generated (or derived) through conventional inter-frame prediction and prediction blocks generated (or derived) through supplementary prediction from the final prediction block, they may also be referred to as reference blocks, and the terms prediction block and reference block are used interchangeably. In the embodiments described later, they are primarily referred to as conventional prediction blocks and supplementary prediction blocks; however, any block falling within the scope of conventional prediction blocks or supplementary prediction blocks defined and described in this disclosure, even if referred to by another term such as reference block, should be considered to correspond to a conventional prediction block or supplementary prediction block of this disclosure. That is, regardless of the name, any block generated by the methods described in this disclosure and used according to the description of this disclosure may fall within the scope of prediction blocks or reference blocks of this disclosure.

[0408] Additionally, in this disclosure, multiple prediction blocks or multiple reference blocks may refer to additional prediction blocks, and in some cases may be used to include both additional prediction blocks and regular prediction blocks.

[0409] Furthermore, the terms "derive" and "generate" can be used in the same sense in this disclosure. That is, describing the derivation of a reference block or prediction block can mean that the reference block or prediction block is generated, and describing the generation of a reference block or prediction block can mean that the reference block or prediction block is derived.

[0410] As an example, a regular prediction block may include an L0 prediction block and / or an L1 prediction block. As mentioned above, a regular prediction block may also be referred to as a regular reference block, and may also be referred to as an initial prediction block, initial reference block, temporary prediction block, temporary reference block, reference prediction block, basic prediction block, or basic reference block.

[0411] Additionally, as an example, a regular prediction block can mean an L0 prediction block or an L1 prediction block, or it can mean a block obtained by weighting and combining L0 and L1 prediction blocks.

[0412] As a specific example, the decoding device can derive a first and a second regular prediction block for the current block by performing bidirectional prediction, and can weight and combine the first and second regular prediction blocks to combine with an additional prediction block. Alternatively, the decoding device can generate a regular prediction block for the current block by performing unidirectional prediction. That is, a regular prediction block can mean each of a plurality of prediction blocks generated by bidirectional prediction, can mean the result obtained by weighting and combining these prediction blocks, or can mean a prediction block generated by unidirectional prediction.

[0413] In one implementation, weights can be used to weight and combine L0 and L1 prediction blocks. Here, weights can collectively refer to BCW (bidirectional prediction with CU-based weights) and CW (CU-based weights). Weights can be derived from a weight candidate list. The weight candidate list can include multiple weight candidates and can be predefined in the encoding and decoding devices.

[0414] Weight candidates can be a set of weights indicating the weights (i.e., the first weight and the second weight) applied to the corresponding bidirectional prediction blocks, or they can be weights applied to prediction blocks in one of the bidirectional prediction directions. When the weights applied to a prediction block in one direction are derived only from the weight candidate list, the weights applied to the prediction block in the other direction can be derived based on the weights derived from the weight candidate list. For example, the weights applied to the prediction block in the other direction can be derived by subtracting the weights derived from the weight candidate list from a predetermined value.

[0415] In one implementation, a weight index indicating the weights used for weighted prediction of the current block can be derived from a list of weight candidates. In this disclosure, the weight index may be referred to as bcw_idx or the BCW index. The weight index can be derived by the decoding device or signaled by the encoding device. When derived by the decoding device, the weight index can be derived as the weight index of a specific merge candidate in the merge candidate list. As an example, a specific merge candidate can be specified in the merge candidate list by the merge index.

[0416] The decoding device can generate additional prediction blocks based on the multi-prediction block mode (S910) and can generate the final prediction block by combining the regular prediction block and the additional prediction block (S920).

[0417] In one implementation, when applying a multi-prediction block mode, the decoding device can derive a maximum predefined number of additional prediction blocks and can combine the derived additional prediction blocks with regular prediction blocks. In other words, the decoding device can combine additional prediction blocks of a number equal to or less than the predefined number with regular prediction blocks. As an example of a method for combining regular prediction blocks and additional prediction blocks, a weighted sum can be applied. As an example, the predefined number can be 2. Alternatively, the predefined number can be one of 1, 2, 3, or 4. Here, the predefined number can be referred to as the maximum number of the multi-prediction block mode. The above-described number of additional prediction blocks is merely an example applicable to this disclosure, and various numbers of additional prediction blocks can also be applied to the multi-prediction block mode according to this disclosure.

[0418] like FIG. 25The example illustrates that bidirectional prediction can be applied as regular prediction to generate a first regular prediction block (P0) and a second regular prediction block (P1), and two additional prediction blocks (P2, P3) can be generated. This example demonstrates generating two prediction blocks for each prediction direction, but the number of prediction blocks for each prediction direction can vary.

[0419] When multiple additional prediction blocks (P2, P3) are combined, they can be weighted sequentially and summed with the regular prediction block (P′). For example, when generating at most two additional prediction blocks, a prediction block can be generated by weighting and summing the regular prediction block and the first additional prediction block (P2), and a final prediction block (P) can be generated by weighting and summing the generated prediction block and the second additional prediction block (P3). The prediction block generated by weighting and summing the regular prediction block and the first additional prediction block (P2) can be called an intermediate prediction block. This process can be represented by Equation 15.

[0420] [Formula 15]

[0421] Step 1:

[0422] Step 2:

[0423] Step 3:

[0424] Here, w0 represents the weight of the first additional prediction block, and w1 represents the weight of the second additional prediction block. Additionally, P′ represents the regular prediction block obtained by weighting and combining the first and second regular prediction blocks, and P” can represent an intermediate prediction block. In this example, the first regular prediction block (P0) and the second regular prediction block (P1) can be assigned the same weight (0.5), but different weights can also be assigned. In this disclosure, a regular prediction block can represent the weighted sum of the first and second regular prediction blocks, can represent one of the first and second regular prediction blocks, or can represent both the first and second regular prediction blocks. When performing a one-way prediction to generate a regular prediction block, the regular prediction block can represent a single prediction block generated by the one-way prediction.

[0425] Alternatively, when combining multiple additional prediction blocks, the regular prediction block and the multiple additional prediction blocks can be weighted together and summed. That is, after generating multiple additional prediction blocks, weights can be applied to the multiple additional prediction blocks and the regular prediction block (or the first regular prediction block and the second regular prediction block), and they can be weighted together and summed.

[0426] At the same time, despite FIG. 24As not illustrated above, in order for the decoding device (300) to perform decoding, the decoding device can receive a bitstream from the encoding device (200) and obtain image information from the received bitstream. Therefore, in order to generate a regular prediction block (S900), information about the regular prediction mode can be obtained from the bitstream, and in order to generate an additional prediction block (S910), information about the additional prediction mode can be obtained from the bitstream. In addition, in order to generate the final prediction block (S920), weight information applied to the regular prediction block and the additional prediction block can be obtained from the bitstream.

[0427] Additionally, the decoding device (300) can determine whether to apply the multi-prediction block mode. In this case, a step of determining whether to apply the multi-prediction block mode can be added before step S910. Whether to apply the multi-prediction block mode can be explicitly signaled and sent, or it can be implicitly deduced (or determined) by the decoding device.

[0428] As an example, whether to apply a multi-prediction block mode can be signaled and encoded by the encoding device, and then sent to the decoding device. Specifically, information indicating whether a multi-prediction block mode is applied (hereinafter referred to as multi-prediction block mode information) (e.g., a multi-prediction block mode flag) can be signaled by the encoding device and sent to the decoding device. In this case, the conditions for signaling / resolving the multi-prediction block mode flag can be predefined. The signaling / resolving conditions for the multi-prediction block mode flag can be the conditions for enabling multi-prediction block mode. When the signaling / resolving conditions for the multi-prediction block mode flag are met, the decoding device can parse the multi-prediction block mode flag from the bitstream.

[0429] Alternatively, in another example, whether to apply the multi-prediction block mode can be deduced by the decoding device based on predefined encoding information. As an example, whether to apply the multi-prediction block mode can be defined in the same way as the multi-prediction block mode enabling conditions (or signaling / parsing conditions) described later.

[0430] For example, multi-prediction block mode information can be represented by syntax elements such as `mhp_flag`, and whether a multi-prediction block mode is applied (i.e., whether an additional prediction block exists) can be signaled / inferred by the value of `mhp_flag`. When the value of `mhp_flag` is 0, it can indicate that the multi-prediction block mode is not applied or that an additional prediction block does not exist, and when the value of `mhp_flag` is 1, it can indicate that the multi-prediction block mode is applied or that an additional prediction block exists.

[0431] For example, when the maximum number of additional prediction blocks (MaxNum) is 2, the value of mhp_flag can indicate the presence of additional prediction blocks in the manner shown in Table 9 below.

[0432] [Table 9]

[0433] When a multi-prediction block mode is applied to the current block, the decoding device can obtain information about the additional prediction mode (hereinafter referred to as additional prediction mode information) to generate additional prediction blocks. As an example, the additional prediction mode information may include prediction information and / or weight information. Based on this prediction information, additional prediction blocks according to the multi-prediction block mode can be generated, and based on this weight information, these additional prediction blocks can be combined with regular prediction blocks (or intermediate prediction blocks).

[0434] Specifically, the prediction information may include information indicating the prediction mode applied to the generation of the additional prediction block (hereinafter referred to as prediction mode information), information indicating the detailed prediction mode required to generate the additional prediction block according to the prediction mode (hereinafter referred to as detailed prediction mode information), and various other information required to generate the additional prediction block by applying the prediction mode and the detailed prediction mode. Weight information may represent the weights used to combine the additional prediction block with the regular prediction block.

[0435] In conventional multi-prediction block mode, the prediction modes applied to the generation of additional prediction blocks are restricted to inter-frame prediction modes. In this disclosure, encoding / decoding efficiency can be improved by including intra-frame prediction and / or IBC prediction in the prediction modes applied to the generation of additional prediction blocks.

[0436] For example, the additional prediction mode may include at least one of inter-frame mode, intra-frame mode, or IBC mode. Furthermore, inter-frame modes may include merge mode and MVP mode (or AMVP mode). Prediction mode information may include information indicating whether the additional prediction mode is inter-frame mode, intra-frame mode, or IBC mode. Additionally, prediction mode information may include information indicating whether the additional prediction mode within inter-frame mode is merge mode or MVP mode.

[0437] For example, prediction mode information can be represented by syntax elements indicating whether one of the aforementioned additional prediction modes is applied. As an example, prediction mode information may include a merging flag indicating whether a merging mode is used as an additional prediction mode, or an intra flag indicating whether an intra mode is applied. Alternatively, predefined prediction modes can be used to generate additional prediction blocks. Alternatively, additional prediction modes can be selected based on predefined coding information.

[0438] For example, when the merge pattern is used to generate additional prediction blocks, the prediction information may include a merge index. The merge index can indicate a specific merge candidate in the merge candidate list. When the AMVP pattern is used to derive additional prediction blocks, the prediction information may include a motion vector predictor index, a reference index, and motion vector difference information. The motion vector predictor index can indicate a specific candidate in the motion vector predictor candidate list.

[0439] As mentioned above, additional prediction mode information can include weight information. Weight information can represent the weights used when additional prediction blocks are combined with regular prediction blocks. When multiple additional prediction blocks exist, weight information for each of the additional prediction blocks can be included in the prediction information. As an example, weight information can be indicated by a weight index.

[0440] The additional prediction patterns used to derive additional prediction blocks and the regular prediction patterns used to derive regular prediction blocks can be distinguished from each other. For example, the MVP candidate list used to generate regular prediction blocks and the MVP candidate list used to generate additional prediction blocks can be configured independently. Furthermore, the merged candidate list used to generate regular prediction blocks and the merged candidate list used to generate additional prediction blocks can also be configured independently. However, according to implementations, the same candidate list can be used while using indices indicating different candidates, or one candidate list can be derived from another candidate list.

[0441] Furthermore, information regarding regular prediction blocks and information regarding multiple prediction blocks can be distinguished from each other. For example, when the value of the multiple prediction block mode flag (mhp_flag) indicating whether the multiple prediction block mode is applied is 1, the merge flag indicating whether the merge mode or inter-frame mode is used to generate additional prediction blocks can be represented as mhp_mrg or mhp_mrg_flag. That is, syntax elements representing information about additional prediction modes and syntax elements representing information about regular prediction modes can be distinguished from each other.

[0442] The grammar names described in this disclosure are merely examples, and all of these names may be changed. Even if a grammar uses a name different from the grammar names described in this disclosure, if a grammar includes or represents information described in this disclosure, that grammar falls within the scope of the grammars described in this disclosure.

[0443] In the embodiments described below, for ease of description, the inter-frame mode used as an additional prediction mode may be referred to as MHP-INTER, MHP-inter-frame mode, MHP-inter-frame prediction mode, or MHP-inter-frame prediction, and the intra-frame mode used as an additional prediction mode may be referred to as MHP-INTRA, MHP-intra-frame mode, MHP-intra-frame prediction mode, or MHP-intra-frame prediction. Furthermore, the IBC mode used as an additional prediction mode may be referred to as MHP-IBC, MHP-IBC mode, MHP-IBC prediction mode, or MHP-IBC prediction. Additionally, the merging mode used as an additional prediction mode may be referred to as MHP-merging mode, and the AMVP mode used as an additional prediction mode may be referred to as MHP-AMVP mode.

[0444] Furthermore, among the prediction mode information mentioned above, the information indicating whether the MHP-inter-prediction mode is applied can be called MHP-inter-frame information, and as an example, MHP-inter-frame information may include the MHP-inter-frame flag (mhp_inter_flag).

[0445] In addition, information indicating whether MHP-intra-prediction mode is applied can be called MHP-intra-information, and as an example, MHP-intra-information may include MHP-intra-flag (mhp_intra_flag).

[0446] In addition, information indicating whether an MHP-merge mode is applied can be called MHP-merge information, and as an example, MHP-merge information may include MHP-merge flags (mhp_mrg_flag or mhp_mrg).

[0447] Furthermore, in embodiments according to this disclosure, sending a signal notification information or signal may correspond to generating the information or signal, or may include generating the information or signal. Sending a signal notification information or signal may be performed by an encoding device, and the encoding device may encode the information or signal being sent or generated. Alternatively, sending a signal notification information or signal may be used with the same meaning as encoding, or may be used to include both generating and encoding the information or signal.

[0448] Furthermore, in embodiments according to this disclosure, parsing information or signals may correspond to obtaining the information or signals from a bitstream, or may include obtaining the information or signals. Parsing information or signals may be performed by a decoding device, which may parse the information or signals and reconstruct blocks or images based on the parsed information or signals.

[0449] Implementation 1

[0450] According to one implementation, when a multi-prediction block mode is applied, a signal can be sent to notify / parse whether the intra-prediction mode is applied to the generated MHP-intra-frame information of the additional prediction blocks. In the implementation described below, the case where the MHP-intra-frame flag (mhp_intra_flag) is used as the MHP-intra-frame information will be described as an example.

[0451] FIG. 26 This is a flowchart illustrating the operation of obtaining additional prediction mode information in a decoding method performed by a decoding device according to one embodiment.

[0452] The decoding device (300) can receive the bit stream generated by the encoding device (200) and obtain additional prediction mode information from the received bit stream. As described above, the additional prediction mode information may include prediction information and weight information, and the prediction information may include information about the prediction mode used to generate the additional prediction block (prediction mode information) and motion information.

[0453] Reference FIG. 26 When the maximum number of additional prediction blocks that can be generated is MaxNum, the decoding device can... FIG. 26 The process illustrated in the example is repeated MaxNum times to perform the multi-prediction block mode. That is, the loop can be executed up to the maximum number of additional prediction blocks (until the result of S1100 is negative).

[0454] The decoding device can parse the multi-prediction block mode flag (mhp_flag) from the bitstream (S1110). The multi-prediction block mode flag (mhp_flag) can indicate whether an additional prediction block exists. For example, when the value of the multi-prediction block mode flag (mhp_flag) is 1, it can indicate that an additional prediction block exists. When the value of the multi-prediction block mode flag (mhp_flag) is not 1, for example, when it is 0, it can indicate that an additional prediction block does not exist.

[0455] When the Multi-Prediction Block Mode Flag (mhp_flag) is 1 (Yes in S1120), the MHP-intra-frame information indicating whether intra-frame prediction is used for the generation of additional prediction blocks is parsed (S1130). In this embodiment, before determining whether the merge mode or AMVP is applied to the generation of additional prediction blocks, it can be determined whether intra-frame prediction is applied. As an example, the MHP-intra-frame information may include the MHP-intra-frame flag (mhp_intra_flag). When the MHP-intra-frame flag (mhp_intra_flag) is 1, it can indicate that intra-frame prediction is applied to the generation of additional prediction blocks. When the MHP-intra-frame flag (mhp_intra_flag) is not 1, for example, when it is 0, it can indicate that intra-frame prediction is not applied to the generation of additional prediction blocks.

[0456] When the value of the MHP-intra_flag is 1 (yes in S1140), additional information used for intra-frame prediction can be parsed (S1150). A detailed description will be given later.

[0457] When the value of the MHP-intra_flag is not 1 (no in S1140), the MHP-merging information indicating whether the merging mode was used in the generation of the additional prediction block is parsed (S1160). Alternatively, the MHP-merging information may include MHP-merging flags such as mhp_mrg or mhp_mrg_flag. When the value of the MHP-merging flag is 1, it may indicate that the merging mode was applied to the generation of the additional prediction block. When the value of the MHP-merging flag is not 1, for example, 0, it may indicate that the merging mode was not applied to the generation of the additional prediction block.

[0458] When the value of the MHP-merge flag (mhp_mrg) is 1 (yes in S1170), additional information used for the merge mode is parsed and obtained, such as the merge index (merge_idx) and weight information (weight_idx) (S1180).

[0459] When the value of the MHP-merge flag is not 1 (no in S1170), the additional information used for AMVP is parsed and obtained (S1190). As a specific example, the reference index (refIdx), motion vector predictor index (mvp_idx), motion vector difference information (mvd_data), and weight information (weight_idx) can be parsed and obtained.

[0460] although FIG. 26The decoding method is described in the text, but operations with the same intent can also be applied to the encoding method performed by the encoding device. The encoding device (200) can generate and encode information about the multi-prediction block pattern. The encoded information about the multi-prediction block pattern can be output as a bit stream and sent to the decoding device (300).

[0461] Specifically, the encoding device (200) can generate information indicating whether a multi-prediction block mode is applied to the current block, and can generate information about the regular prediction mode used to generate the regular prediction block and information about the additional prediction mode used to generate the additional prediction block based on the application of the multi-prediction block mode. The generated information can be encoded and output as a bitstream.

[0462] For example, information indicating the presence of additional prediction blocks (e.g., a multi-prediction block mode flag (mhp_flag)) can be generated or signaled and encoded, MHP-intra-frame information indicating whether intra-frame prediction is used for the generation of additional prediction blocks (e.g., mhp_intra_flag) can be used, and information indicating whether merging mode is used for the generation of additional prediction blocks (e.g., mhp_mrg or mhp_mrg_flag) can be used.

[0463] Furthermore, when intra-frame prediction is used to generate additional prediction blocks, additional information required for intra-frame prediction can be generated or signaled and encoded.

[0464] Furthermore, when the merge pattern is used to generate additional prediction blocks, additional information required for applying the merge pattern (e.g., merge index, etc.) can be generated or signaled and encoded.

[0465] Furthermore, when inter-frame prediction modes (such as AMVP mode) are used to generate additional prediction blocks, additional information required for applying AMVP mode (such as reference index, motion vector predictor index, motion vector difference information, etc.) can be generated or signaled and encoded.

[0466] In addition, weight information can be generated or signaled and encoded for combining additional prediction blocks with regular prediction blocks or combining multiple additional prediction blocks with each other.

[0467] As in this embodiment, combining the inter-frame pattern of the block most similar to the current block in the reference image with the intra-frame pattern derived from the neighboring samples of the current block not only improves compression efficiency but also removes the discontinuity between the current block and its neighboring samples.

[0468] Furthermore, when signaling / parsing whether intra-frame prediction is applied to the generated MHP-intra-frame information of the additional prediction block, the corresponding information can be signaled / parsed only if predetermined conditions are met. Therefore, the overhead of signaling / parsing can be reduced, and the complexity in both the encoding and decoding devices can be decreased. This will be described in detail below.

[0469] FIG. 27 This is a flowchart illustrating another example of the operation of obtaining additional prediction mode information in a decoding method performed by a decoding device according to one embodiment.

[0470] Reference FIG. 27 When the value of the multi-prediction block mode flag (mhp_flag) is 1 (yes in S1120), the decoding device (300) can determine whether a predetermined condition is met (S1200). When the predetermined condition is met (yes in S1200), the decoding device can determine whether MHP-intra-frame prediction is applied by parsing the MHP-intra-frame flag and perform the remaining operations (S1130 to S1190).

[0471] When the predetermined conditions are not met (no in S1200), the decoding device can determine whether the MHP-merging mode is applied by parsing the MHP-merging information without needing to determine whether MHP-intra-frame prediction is applied (S1160).

[0472] The predetermined conditions used to determine whether to signal / parse MHP-intraframe information will be described in detail below.

[0473] - Predetermined conditions can be related to the regular prediction mode of the current block. Predetermined conditions can be satisfied when the regular prediction mode does not include a specific prediction mode. For example, MHP-intra-frame prediction may not be applied when the regular prediction mode is at least one of AMVP, CIIP, or GPM modes. In other words, when the regular prediction mode is at least one of the above modes, the predetermined conditions are not met, and MHP-intra-frame information may not be signaled / resolved. That is, in this case, intra-frame prediction may not be used for the generation or derivation of additional prediction blocks.

[0474] - Predetermined conditions can relate to whether refinement is applied to the current block. Here, refinement applied to the current block can refer to tools or techniques such as DMVR for refining motion information or BDOF for refining sample values ​​in the block. As an example, when refinement is applied to the current block based on techniques such as DMVR or BDOF, MHP-intra-prediction may not be applied. In other words, when refinement is applied to the current block, the predetermined conditions are not met, and MHP-intra-prediction may not be signaled / resolved. That is, in this case, intra-prediction may not be used for the generation or derivation of additional prediction blocks.

[0475] - Predetermined conditions can be related to the block size. For example, the condition can be determined based on the block size or the block's aspect ratio. Specifically, when the block size (width × height) is less than a predetermined size, the predetermined condition is met and MHP-intra-frame prediction is allowed. As an example, MHP-intra-frame prediction is allowed when the block size is less than 1024. In other words, when the current block size is less than the predetermined size, MHP-intra-frame information can be signaled / parsed, and when the size is greater than or equal to the predetermined size, MHP-intra-frame information can be left unsigned / unparsed.

[0476] - Predefined conditions can be associated with quantization parameters (QP). For example, when the quantization parameter is less than a threshold, the predetermined condition is met and MHP-intra-frame prediction is allowed. As an example, the threshold can be set to a value greater than 32, and MHP-intra-frame prediction is allowed when the quantization parameter is 32. In other words, when the quantization parameter is less than the threshold, MHP-intra-frame information can be signaled / parsed, and when the quantization parameter is greater than or equal to the threshold, MHP-intra-frame information can be left unsigned / unparsed.

[0477] - Predefined conditions can be related to the temporal layer. For example, MHP-intra-frame prediction can be allowed when the temporal layer index is less than a threshold. As an example, the threshold can be set to a value greater than 4, and MHP-intra-frame prediction can be allowed when the temporal layer index is 4. In other words, when the temporal layer index is less than the threshold, MHP-intra-frame information can be signaled / parsed, and when the temporal layer index is greater than or equal to the threshold, MHP-intra-frame information can be left unsigned / unparsed.

[0478] The above conditions are merely examples applicable to one implementation method; other conditions may also be applied. Furthermore, one of the above conditions may be applied, or two or more conditions may be applied together.

[0479] although FIG. 27 The decoding method is described above, but operations with the same intent can also be applied to the encoding method performed by the encoding device (200). Therefore, in addition to the above references FIG. 26In addition to the operation or encoding method of the described encoding device (200), the encoding device may signal to MHP-intraframe information when the above conditions are met.

[0480] When decoding or encoding is performed according to the above description, the prediction modes used to generate the additional prediction blocks can have combinations as described in Table 10 below. These combinations are derived under the assumptions that MaxNum is 2 and 3, respectively. In the following text, “INTRA (intra-frame)” is used as the term indicating MHP-intra-frame prediction, and “INTER (inter-frame)” is used as the term indicating either MHP-inter-frame prediction or MHP-merging mode.

[0481] [Table 10]

[0482] According to one implementation, the number of additional prediction blocks generated by intra-frame prediction can be included as a condition for determining whether to signal / parse MHP-intra-frame information. As an example, only one additional prediction block may be allowed to be generated by intra-frame prediction. In the embodiments described below, for ease of description, the additional prediction block to which intra-frame prediction is applied will be referred to as an intra-additional prediction block, an MHP-intra-frame block, or an MHP-intra-frame prediction block.

[0483] FIG. 28 This is a flowchart illustrating another example of the operation of obtaining additional prediction mode information according to a decoding method performed by a decoding device according to one embodiment. FIG. 28 In the example, the number of intra-frame additional prediction blocks can be determined as a condition for deciding whether to signal / parse MHP-intra-frame information.

[0484] Reference FIG. 28 As an example of determining the aforementioned predetermined condition (S1200), the number of intra-frame additional prediction blocks can be determined (S1210), and as an example, the number of intra-frame additional prediction blocks can be limited to one. Therefore, when the number of currently generated intra-frame additional prediction blocks is 0 (yes in S1210), the predetermined condition is met, and the decoding device can parse the MHP-intra-frame information to determine whether MHP-intra-frame prediction is applied, and perform the remaining operations (S1130 to S1190).

[0485] When the number of currently generated intra-frame additional prediction blocks is not 0 (no in S1210), that is, when an intra-frame additional prediction block has been generated, the decoding device can determine whether the MHP-merging mode is applied by parsing the MHP-merging information (S1160) without needing to determine whether MHP-intra-frame prediction is applied, and perform the remaining operations (S1170 to S1190).

[0486] althoughFIG. 28 The decoding method is described above, but operations with the same intent can also be applied to the encoding method performed by the encoding device (200). Therefore, as mentioned above... FIG. 26 and FIG. 27 In the operation or encoding method of the described encoding device (200), the number of intra-frame additional prediction blocks can be determined as a predetermined condition, and MHP-intra-frame information can only be generated or signaled until the number of intra-frame additional prediction blocks is less than the predetermined number.

[0487] Referring again to Table 10 above, when MaxNum is 2 and the combination is {INTER, INTER}, the MHP-intra-frame flag is still signaled / resolved twice, even though the intra-frame mode is not applied to the generation of the additional prediction block. Furthermore, when MaxNum is 3 and the combination is {INTER, INTER, INTER}, the MHP-intra-frame flag is still signaled / resolved three times, even though the intra-frame mode is not applied to the generation of the additional prediction block.

[0488] However, when the number of intra-frame additional prediction blocks is limited to one, signaling / parsing overhead can be reduced and efficient encoding / decoding can be achieved. In this case, the prediction modes used to generate the additional prediction blocks can have combinations as described in Table 11 below.

[0489] [Table 11]

[0490] In the example above, since only one intra-frame append prediction block is allowed, combinations of two or more "INTRA" are not permitted. Comparing Table 11 with Table 10 above, when MaxNum is 2, the combination {INTRA, INTRA} will not occur. Furthermore, when MaxNum is 3, the combinations {INTRA, INTRA}, {INTRA, INTRA, INTER}, {INTRA, INTER, INTRA}, {INTER, INTRA, INTRA}, and INTRA, INTRA, INTRA} will not occur.

[0491] FIG. 29 This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information by a decoding method performed by a decoding device according to one embodiment. FIG. 29 In the example, the predetermined number of additional prediction blocks can be determined as a condition for determining whether to signal / parse MHP-intraframe information.

[0492] Reference FIG. 29The information indicating the number of additional prediction blocks can be parsed (S1221). To this end, the encoding device (200) can signal the number of additional prediction blocks applied to the current block and send this information to the decoding device (300). As an example, information indicating the number of additional prediction blocks can be signaled using syntax such as `mhp_num`. When the value of `mhp_num` is "0", it can indicate that no additional prediction blocks exist; when the value is "10", it can indicate that one additional prediction block may exist; and when the value is "11", it can indicate that two additional prediction blocks may exist. This binarization method is merely an example of how to signal the number of additional prediction blocks. Furthermore, various methods such as fixed-length codes, truncated Rice codes, truncated binary codes, and exponential Golomb codes can be used to signal the number of additional prediction blocks, and the meaning of `mhp_num` can be expanded and applied as the allowed number of additional prediction blocks increases.

[0493] As an example, MHP-intraframe information can be parsed only when the number of additional prediction blocks is one. Therefore, when the number of additional prediction blocks is greater than 0 (yes in S1222), it is determined whether the number of additional prediction blocks is 1 (S1223). FIG. 29 In the flowchart, the situation is limited to the case where the merge mode is applied to the current block, but this is just an example, and other conditions among the various conditions mentioned above can also be applied.

[0494] When the number of additional prediction blocks is 1 (yes in S1223), the MHP-intraframe flag can be parsed (S1130). When the number of additional prediction blocks is not 1, that is, when the number is greater than 1 (no in S1223), the MHP-merging flag can be parsed (S1160). The operations performed after parsing each flag are the same as described above.

[0495] although FIG. 26 The decoding method is described above, but operations with the same intent can also be applied to the encoding method performed by the encoding device (200). Therefore, as mentioned above... FIG. 28 and FIG. 29 In the operation or encoding method of the described encoding device (200), a signal can be sent to notify the number of additional prediction blocks applied to the current block. The method of signaling the number of additional prediction blocks is similar to that described above. FIG. 30 The decoding method is the same as described.

[0496] Furthermore, the encoding method can signal the MHP-intraframe information only when the number of additional prediction blocks is a specific number. As an example, the MHP-intraframe flag can be signaled only when the number of additional prediction blocks is one.

[0497] Referring again to Table 11 above, when the first additional prediction block is not an intra-predictive block, a signal is sent to notify / parse the MHP-intra-frame information until an intra-predictive block appears. Specifically, in the combinations {INTER, INTER} and {INTER, INTER, INTER}, even though no intra-predictive block exists, the MHP-intra-frame information is still signaled / parsed two and three times, respectively.

[0498] However, allowing MHP-intra-frame prediction only when the number of additional prediction blocks is a specific number (e.g., one) can reduce signaling / parsing overhead and achieve efficient encoding / decoding. In this case, the prediction modes used to generate the additional prediction blocks can have combinations as described in Table 12 below.

[0499] [Table 12]

[0500] In the example above, since MHP-intra-frame prediction is only allowed when the number of additional prediction blocks is one, only the combination {INTRA} including "INTRA" is possible.

[0501] In the above implementation, the MHP-merging information is signaled / parsed before the MHP-intra-frame information. However, this order can be changed to consider the applicability of the prediction mode. For example, it can be determined first whether the MHP-merging mode is applied. That is, the MHP-merging information can be signaled / parsed before the MHP-intra-frame information.

[0502] FIG. 30 This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information by a decoding method performed by a decoding device according to one embodiment. FIG. 30 In the example, the MHP-merging information is parsed before the MHP-intraframe information.

[0503] Reference FIG. 31 The operations of determining the current iteration count to perform the MaxNum iterations (S1300), parsing the multi-prediction block mode flag (S1310), and determining its value (S1320) are the same as described above.

[0504] When the value of the multi-prediction block mode flag (mhp_flag) is 1 (yes in S1320), the MHP-merging information is parsed first before the MHP-intra-frame information (S1330). When the value of the MHP-merging flag (mhp_mrg), which serves as an example of the MHP-merging information, is 1, it indicates that the merging mode is used for the generation of additional prediction blocks, and when the value of the MHP-merging flag (mhp_mrg) is 0, it indicates that the merging mode is not used for the generation of additional prediction blocks.

[0505] When the MHP-merge information indicates that the MHP-merge mode is applied, that is, when the value of the MHP-merge flag (mhp_mrg) is 1 (yes in S1340), the information required to apply the MHP-merge mode can be parsed, such as the merge index (merge_idx) and weight information (weight_idx) (S1350).

[0506] When the MHP-merging information indicates that the MHP-merging mode is not applied, that is, when the value of the MHP-merging flag (mhp_mrg) is 0 (no in S1340), the MHP-intra-frame information, such as the MHP-intra-frame flag (mhp_intra_flag), can be parsed (S1370). In this case, according to the implementation, it can be first determined whether a predetermined condition is met (S1360), and when the predetermined condition is met (yes in S1360), the MHP-intra-frame information can be parsed. The description of the predetermined condition has been given above.

[0507] When the MHP-intra-frame information indicates that the MHP-intra-frame mode is applied, that is, when the value of the MHP-intra-frame flag (mhp_intra_flag) is 1 (yes in S1380), the additional information required to apply the MHP-intra-frame mode is parsed (S1390).

[0508] When the MHP-intra-frame information indicates that the MHP-intra-frame mode is not applied, that is, when the value of the MHP-intra-frame flag (mhp_intra_flag) is 0 (no in S1380), it is determined that the MHP-AMVP is applied, and the information required to apply the MHP-AMVP can be parsed, such as the reference index (refIdx), the motion vector predictor index (mvp_idx), the motion vector difference information (mvd_data), and the weight index (weight_idx) (S1400).

[0509] although FIG. 31 The decoding method is described in the text, but operations with the same intent can also be applied to encoding methods performed by the encoding device. The encoding device (200) can generate and encode information indicating the presence of additional prediction blocks (e.g., a multi-prediction block mode flag (mhp_flag)), and when additional prediction blocks exist, it can generate and encode MHP-merging information. When the MHP-merging mode is applied, additional information required for applying the MHP-merging mode (e.g., merge index, weight index, etc.) can be generated and encoded.

[0510] When MHP-merging mode is not applied, MHP-intra-frame information can be generated and encoded based on whether predetermined conditions are met. When MHP-intra-frame mode is applied, additional information required for applying MHP-intra-frame mode can be generated and encoded. When MHP-intra-frame mode is not applied, additional information required for applying MHP-inter-frame mode (e.g., reference index, motion vector predictor index, motion vector difference information, etc.) can be generated and encoded.

[0511] Assuming that the MHP-merging mode is more applicable, as in the example above, when signaling / parsing MHP-merging information before MHP-intraframe information can reduce unnecessary signaling / parsing, lower signaling / parsing overhead, and achieve efficient encoding / decoding.

[0512] FIG. 31 This is a flowchart illustrating another example of the operation of obtaining multi-prediction block mode information by a decoding method performed by a decoding device according to one embodiment. In this example, MHP-merging information can be used to distinguish whether MHP-merging mode or MHP-intra-frame mode is applied without separately signaling / parsing MHP-intra-frame information.

[0513] Reference FIG. 27 The operations of determining the current iteration count to perform the MaxNum iterations (S1500), parsing the multi-prediction block pattern flag (S1510), and determining its value (S1520) are the same as described above.

[0514] When the value of the multi-prediction block mode flag is 1 (yes in S1520), MHP-merging information, such as the MHP-merging flag (mhp_mrg), is parsed (S1530). In this example, when the value of the MHP-merging flag is 1, it indicates that the merging mode is applied to the generation of additional prediction blocks, and when the value of the MHP-merging flag is 0, it indicates that the intra-frame mode is applied to the generation of additional prediction blocks.

[0515] Therefore, when the value of the MHP-merge flag (mhp_mrg) is 1 (yes in S1540), the additional information required to apply the MHP-merge mode (e.g., merge index, weight index, etc.) can be parsed (S1550).

[0516] When the value of the MHP-merging flag (mhp_mrg) is 0 (no in S1540), it corresponds to the situation where the MHP-intra-frame mode is applied. Therefore, the additional information required for applying the MHP-intra-frame mode can be parsed (S1560).

[0517] Meanwhile, according to another example, it can be determined whether the MHP-merge mode is applied only if predetermined conditions are met. In this case, the above reference can be applied. FIG. 31 At least one of the predefined conditions described, and when the corresponding condition is met, the MHP-merging information can be parsed.

[0518] Alternatively, additional information required for applying the MHP-intra-frame mode can be parsed only when predetermined conditions are met. In this case, additional prediction blocks may not be generated if the MHP-merging flag is 0 but the predetermined conditions are not met.

[0519] although FIG. 27 The decoding method is described in the text, but operations with the same intent can also be applied to encoding methods performed by the encoding device. The encoding device (200) can generate and encode information indicating the presence of additional prediction blocks (e.g., a multi-prediction block mode flag (mhp_flag)), and when additional prediction blocks exist, it can generate and encode MHP-merging information. When the MHP-merging mode is applied, it can generate and encode additional information required for applying the MHP-merging mode (e.g., merge index, weight index, etc.). When the MHP-merging mode is not applied, it can generate and encode additional information required for applying the MHP-intra-frame mode.

[0520] Furthermore, as mentioned above, MHP-merging information can be generated and encoded only when predetermined conditions are met. In this case, the above reference can be applied. Implementation 2 At least one of the predefined conditions described, and when the corresponding condition is met, MHP-merging information can be generated and encoded.

[0521] Alternatively, additional information required for applying the MHP-intra-frame mode can be generated and encoded only when predetermined conditions are met. In this case, additional prediction blocks may not be generated if the MHP-merging flag is 0 but the predetermined conditions are not met.

[0522] According to this example, since the application of the MHP-intraframe mode can be determined using the MHP-merging information without separately signaling / parsing the MHP-intraframe information, signaling / parsing overhead can be reduced and efficient encoding / decoding can be achieved.

[0523] Furthermore, the above-described implementation can also be applied to the MHP-IBC prediction mode in the same manner. Therefore, the additional prediction mode used to generate additional prediction blocks may include at least one of the MHP-intra-frame prediction mode or the MHP-IBC prediction mode, and information about the MHP-IBC prediction mode can also be signaled / parsed in the same manner as described above.

[0524] FIG. 32

[0525] According to this embodiment, in the multi-prediction block mode, in addition to the signaling of the aforementioned relevant information, this information can also be inherited or derived from neighboring blocks. The relevant embodiments will be described in detail below.

[0526] The merging mode is a mode that borrows motion information directly from adjacent or non-adjacent neighboring blocks without signaling motion information. Borrowing motion information from neighboring blocks can be described as inheritance, transfer, or deduction. However, the scope of the disclosed embodiments is not limited by terminology, and even if terms other than inheritance, transfer, or deduction are used, the embodiment can fall within the scope of this embodiment as long as they correspond to the operations, functions, or meanings described in this embodiment.

[0527] According to one implementation, when a multi-prediction block pattern is applied to adjacent or non-adjacent neighbor blocks of the current block, information about the multi-prediction block pattern can be derived in the same or similar manner as other information. Here, the information about the multi-prediction block pattern may include information indicating whether the multi-prediction block pattern is applied (multi-prediction block pattern information), information about the regular prediction pattern, information about additional prediction patterns, etc.

[0528] Specifically, when a merge mode is applied to the current block, prediction information can be derived from neighboring blocks. This prediction information may include motion information of neighboring blocks. When a multi-prediction block mode is applied to a neighboring block used to obtain motion information, the prediction information may include information about the multi-prediction block mode of that neighboring block. In this case, the information about the multi-prediction block mode of that neighboring block can be inherited and used to generate additional prediction blocks for the current block. That is, a multi-prediction block mode can also be applied to the current block. Here, neighboring blocks can include both adjacent and non-adjacent blocks, and can include temporal and spatial neighboring blocks.

[0529] In this embodiment, whether the inheritance method described herein is applied can be signaled or deduced on a unit basis, such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), Picture Header (PH), Slice Header (SH), Code Tree Unit (CTU), or Code Unit (CU). The flag used to determine whether the inherited multi-prediction block mode is applied can be distinguished from the flag (mhp_flag) used to determine whether the multi-prediction block mode is signaled, and the inheritance method described below can be applied when the value of the corresponding flag is 1. Furthermore, even if the value of the corresponding flag is 1, the inheritance method described below may not be applied if a predetermined condition is not met. Additionally, even if the corresponding flag does not exist, the inheritance method described below may not be applied if a predetermined condition is not met.

[0530] Furthermore, both signaling and derivation methods can be applied to obtain information about multi-prediction block patterns. For example, when there are N additional prediction blocks and M additional prediction blocks (M≤N) obtained through derivation (or inheritance), information related to the NM additional prediction blocks can be obtained through signaling.

[0531] The conditions used to determine whether inheritance for multiple prediction blocks is applied can be defined as follows: - When DMVR is applied to a block, inheritance of multi-prediction blocks may not be applied. Furthermore, whether inheritance is applied can vary depending on the prediction mode of the multi-prediction block. For example, inheritance may not be applied in MHP-intra-frame mode.

[0532] - When BDOF is applied to a block, inheritance for multi-prediction blocks may not be applied. Furthermore, whether inheritance is applied can vary depending on the prediction mode of the multi-prediction block. For example, inheritance may not be applied in MHP-intra-frame mode.

[0533] - When LIC is applied to a block, inheritance for multi-prediction blocks may not be applied. Furthermore, whether inheritance is applied can vary depending on the prediction mode of the multi-prediction block. For example, inheritance may not be applied in MHP-intra-frame mode.

[0534] - When OBMC is applied to a block, inheritance of multi-prediction blocks may not be applied. Furthermore, whether inheritance is applied can vary depending on the prediction mode of the multi-prediction block. For example, inheritance may not be applied in MHP-intra-frame mode.

[0535] - When CIIP, GPM, or GPM-INTRA is applied to a block, inheritance for multi-prediction blocks may not be applied. Furthermore, whether inheritance is applied can vary depending on the prediction mode of the multi-prediction block. For example, inheritance may not be applied in MHP-intra-frame mode.

[0536] Multi-prediction block mode is a method that can improve encoding / decoding quality and efficiency by combining various modes or types used for the prediction of the current block. When multi-prediction blocks are selectively inherited by applying the conditions in the example above, inheritance can be disallowed if the combined effect obtained through inheritance is non-existent or minimal, thereby reducing decoder complexity and achieving efficient decoding.

[0537] Furthermore, when the multi-prediction block mode applied to a neighboring block includes an intra-frame mode, that is, when the multi-prediction block of a neighboring block includes an intra-attached prediction block, the following predetermined conditions can be used to determine whether the intra-frame mode (MHP-intra-frame mode) or the intra-attached prediction block is inherited: - The MHP-intra-frame mode applied to neighboring blocks may not be inherited. Specifically, when neighboring blocks included in the merge candidate list are encoded / decoded by applying the multi-prediction block mode and the MHP-intra-frame mode is applied, the MHP-intra-frame mode can be excluded and the remaining modes inherited during the derivation of the motion information of the current block. FIG. 32 This example illustrates how to perform inheritance from neighboring blocks that apply multi-prediction block mode while excluding MHP-intra-frame mode. Because... FIG. 32 All prediction modes illustrated are used for additional prediction blocks, so even though they are indicated as INTRA and INTER, they can actually refer to MHP-intra-frame and MHP-inter-frame. FIG. 32 Example (A) illustrates a situation where a neighboring block is encoded / decoded using three additional prediction blocks, and an intra-frame mode, an inter-frame mode, and an inter-frame mode are used to generate these three additional prediction blocks. In this case, when inheriting the multi-prediction block mode of the neighboring block, the intra-frame mode can be excluded and the remaining modes inherited. When the current block allows three additional prediction blocks, information about the prediction modes applied to the remaining additional prediction blocks can be obtained by signaling. In this case, the prediction mode obtained by signaling can be either an intra-frame mode or an inter-frame mode, and the signaling method described above can be applied. FIG. 32 Example (B) illustrates a case where there are neighboring blocks encoded / decoded using two additional prediction blocks, and an inter-frame mode is used to generate these two additional prediction blocks. In this case, both inter-frame modes can be inherited. When the current block allows three additional prediction blocks, information about the prediction modes applied to the remaining additional prediction blocks can be obtained through signaling. In this case, the prediction modes obtained through signaling can be either intra-frame modes or inter-frame modes, and the signaling method described above can be applied. FIG. 32 Example (C) illustrates a case where a neighboring block is encoded / decoded using an additional prediction block, and an intra-frame mode is used to generate that additional prediction block. In this case, the intra-frame mode is not inherited from the neighboring block, and when the current block allows three additional prediction blocks, all information about the prediction modes applied to these three additional prediction blocks can be obtained through signaling. In this case, the prediction mode obtained through signaling can be either an intra-frame mode or an inter-frame mode, and the signaling method described above can be applied. That is, according to... FIG. 33 For example, the intra-frame mode used for generating additional prediction blocks can only be applied by signaling and not by inheritance.

[0538] - Inheritance can also be performed without distinguishing between intra-frame and inter-frame modes. Since this method always performs inheritance without determining conditions during the inheritance process, it is computationally efficient. Furthermore, because intra-frame modes can be applied to multiple additional prediction blocks, it can operate efficiently on sequences where compression efficiency can be improved through a weighted combination of multiple intra-frame prediction blocks.

[0539] - The number of intra modes that can be inherited can be limited. Since both intra modes obtained through signaling and those obtained through inheritance may exist, there may be one or more intra modes among the intra modes applied to the additional prediction block. In this case, the number of intra modes that can be inherited can be limited. FIG. 33 An example is shown where the number of intra-frame modes inherited from a neighboring block that applies a multi-prediction block mode is limited to one. FIG. 33 Example (A) illustrates a situation where a neighboring block is encoded / decoded using three additional prediction blocks, and an intra-frame mode, an intra-frame mode, and an inter-frame mode are used to generate these three additional prediction blocks. In this case, when inheriting the multi-prediction block mode information of the neighboring block, the first intra-frame mode can be excluded, and the remaining intra-frame and inter-frame modes can be inherited. When the current block allows three additional prediction blocks, information about the prediction modes applied to the remaining additional prediction blocks can be obtained by signaling. For example, the prediction mode obtained by signaling can be either an intra-frame mode or an inter-frame mode, and the signaling method described above can be applied. FIG. 33 (B) also illustrates a case where there are neighboring blocks encoded / decoded using three additional prediction blocks, and the intra-frame mode, intra-frame mode, and inter-frame mode are used to generate these three additional prediction blocks. As another example, when inheriting the multi-prediction block mode information of a neighboring block, the last intra-frame mode can be excluded and the remaining intra-frame and inter-frame modes can be inherited. Implementation 3 This is merely an example applicable to one implementation. The number and order of the intra-frame modes that can be inherited may be modified differently from the example above.

[0540] Alternatively, the information inherited from neighboring blocks related to the multi-prediction block mode can be restricted to distinguishing the intra-frame mode, inter-frame mode, and merge mode applied to additional prediction blocks. Besides prediction mode information, the information related to the multi-prediction block mode can also include motion information, weight information, and other additional information. However, storing a large amount of information directly impacts the increase in internal memory capacity; therefore, the method can be modified by limiting the amount of information stored.

[0541] For example, the storage of information about MHP-intra-frame modes can be restricted. That is, prediction mode information used to distinguish MHP-intra-frame modes can be included in the inherited information, but information about specific detailed prediction modes can be excluded. In this case, a preset default mode can be used as the detailed prediction mode, or a detailed prediction mode derived from neighboring blocks can be used. Furthermore, when weight information is not included in the inherited information, default weight values ​​can be used, or weight information derived from neighboring blocks can be used. Additionally, even if supplementary information about MHP-intra-frame modes exists, it can be excluded from the inherited information, and a default value can be used, or the information can be derived from neighboring blocks.

[0542] As another example, the storage of information regarding MHP-inter-frame mode and MHP-merging mode can be restricted. That is, prediction mode information and motion information used to distinguish between MHP-inter-frame mode and MHP-merging mode can be included in the inherited information, but weight information may not be included. In this case, preset default weight values ​​can be used, or weight information derived from neighboring blocks can be used. Furthermore, even if additional information exists for MHP-inter-frame mode and MHP-merging mode, it may not be included in the inherited information, and default values ​​or information that can be derived from neighboring blocks can be used.

[0543] Furthermore, the above-described implementation can also be applied to the MHP-IBC prediction mode in the same manner. Therefore, the additional prediction mode used to generate additional prediction blocks may include at least one of the MHP-intra-frame prediction mode or the MHP-IBC prediction mode, and whether the inheritance of the MHP-IBC prediction mode is applied can also be determined in the same manner as described above.

[0544] FIG. 34

[0545] According to one embodiment of this disclosure, when the additional prediction mode includes an intra-frame mode, the relevant information can also be derived on the decoder side without signaling the relevant information.

[0546] FIG. 34 This is a flowchart illustrating an example of deriving information about the MHP-intra-frame mode on the decoder side in a decoding method performed by a decoding device according to an embodiment.

[0547] Here, numMHP can represent the number of additional prediction blocks that are signaled or inherited. In this example, since information about the MHP-intra-frame mode is not signaled or inherited, numMHP can represent the sum of the MHP-inter-frame mode and the MHP-merging mode. In other words, numMHP can represent the sum of the number of additional prediction blocks with the MHP-inter-frame mode applied and the number of additional prediction blocks with the MHP-merging mode applied.

[0548] Alternatively, numMHP can be limited to the number of additional prediction blocks that are signaled.

[0549] Reference FIG. 35 During the decoding process, regular prediction blocks can be generated (S1600), and when numMHP is greater than 0 (yes in S1610), a corresponding number of inter-frame additional prediction blocks (MHP-inter-frame blocks) can be generated (S1620).

[0550] When predetermined conditions are met (Yes in S1630), an intra-frame additional prediction block (MHP-intra-block) can be generated by applying the MHP-intra-frame mode (S1640). Then, according to predetermined rules and embodiments of this disclosure, the regular prediction block, the inter-frame additional prediction block generated by signaling and inheritance, and the intra-frame additional prediction block generated by derivation method are combined to generate a final prediction block (S1650). As an example, the final prediction block can be generated by performing a weighted summation on the regular prediction block and the additional prediction block.

[0551] The above method can also be extended and applied to MHP-IBC mode or MHP-inter-frame mode. When numMHP is greater than 0, additional prediction blocks can be generated based on MHP-IBC mode or MHP-intra-frame mode, and MHP-inter-frame mode and MHP-IBC mode can be derived when predetermined conditions are met. If the definition is changed so that numMHP refers to the number of MHP-inter-frame modes, then the derived mode can be one or more of the three modes, depending on the predefined allowed number for each mode. The same method can also be applied when numMHP refers to the number of MHP-intra-frame modes or MHP-IBC modes.

[0552] FIG. 35 This is a flowchart illustrating another example of deriving information about MHP-intra-frame modes on the decoder side in a decoding method performed by a decoding device according to one embodiment.

[0553] Reference FIG. 34During decoding, regular prediction blocks can be generated (S1700), and when numMHP is greater than 0 (yes in S1710), a corresponding number of inter-frame additional prediction blocks (MHP-inter-frame blocks) can be generated (S1720). numMHP can represent the number of additional prediction blocks that are signaled or inherited. In this example, since information about MHP-intra-frame modes is not signaled or inherited, numMHP can represent the sum of MHP-inter-frame modes and MHP-merging modes. Alternatively, numMHP can be limited to the number of additional prediction blocks that are signaled.

[0554] When no additional prediction block exists, i.e., when numMHP is 0 (no in S1710), an intra-frame additional prediction block (MHP-intra-frame block) can be generated if a predetermined condition is met (yes in S1730) (S1740). In other words, even if the current block does not have an additional prediction block that has been signaled or inherited, an additional prediction block can be generated by applying the MHP-intra-frame mode if the predetermined condition is met.

[0555] Then, according to predetermined rules and embodiments of this disclosure, the regular prediction block can be combined with either an inter-frame additional prediction block generated by signaling and inheritance (the block generated in step S1720) or an intra-frame additional prediction block generated by derivation (the block generated in step S1740) to generate a final prediction block (S1650). As an example, the final prediction block can be generated by performing a weighted summation on the regular prediction block and the additional prediction block.

[0556] The above method can also be extended and applied to MHP-IBC mode or MHP-inter-frame mode. Even when numMHP == 0, additional prediction blocks can be generated based on MHP-IBC mode or MHP-inter-frame mode if predetermined conditions are met. If the definition is changed so that numMHP refers to the number of MHP-inter-frame modes, the derived mode can be one or more of the three modes, depending on the predefined allowed number for each mode. The same method can also be applied when numMHP refers to the number of MHP-intra-frame modes or MHP-IBC modes.

[0557] The above FIG. 35 and FIG. 36 Examples can be combined with each other. Specifically, an MHP-intra-block can be generated when numMHP > 0 and a predetermined condition is met, and an MHP-intra-block can also be generated when numMHP == 0 and another predetermined condition is met.

[0558] Since combining the inter-frame pattern of the block most similar to the current block in the reference image with the predicted block of the intra-frame pattern derived from the neighboring samples of the current block not only improves compression efficiency but also removes discontinuities between the current block and neighboring samples, the similarity to neighboring samples can be regarded as a condition for determining whether the MHP-intra-frame pattern is applied (the aforementioned "predetermined condition"). Therefore, in the above embodiment, the condition for determining whether to derive the MHP-intra-frame pattern can be defined as follows.

[0559] FIG. 36 This is a diagram used to explain the error between the neighbor samples of the current block and the neighbor samples of the regular predicted block.

[0560] Reference Implementation 4 Intra-frame supplementary prediction blocks can be generated by applying MHP-intra-frame mode based on the errors (SA, SL) between the neighbor samples (cA, cL) of the current block and the neighbor samples (pA, pL) of the regular prediction block. In other words, the error between the neighbor samples of the current block and the prediction block can be used as a criterion for judging the discontinuity of the block. In this example, the error can be information indicating the difference or error between multiple pieces of information. In some cases, cost can also be used in a similar sense.

[0561] Specifically, the error SA between the top neighbor sample cA of the current block and the top neighbor sample pA of the regular prediction block can be compared with a threshold. Furthermore, the error SL between the left neighbor sample cL of the current block and the left neighbor sample pL of the regular prediction block can be compared with a threshold. Additionally, the comparison of top and left neighbor samples can be considered simultaneously. For ease of description, the upper left portion is not described, but samples at corresponding positions can be considered when calculating the error between samples using top, left, or top / left samples.

[0562] The threshold can be set according to Equations 16 and 17 below.

[0563] [Formula 16]

[0564] Threshold = Width Template height / λ

[0565] [Equation 17]

[0566] Template width Height / λ

[0567] Here, λ can be set to a value that satisfies λ ≥ 1.

[0568] Specifically, the error (S0) between samples (or sample values, hereinafter, samples and sample values ​​are considered to have the same meaning) A SL The cost or cost can be obtained through the following methods or a combination thereof.

[0569] - The error or cost can be calculated using the Sum of Absolute Differences (SAD). That is, it can be calculated according to Equation 18 below. In Equation 18, c and p represent the neighbor samples of the current block and the regular predicted block, respectively, and x represents the position of the neighbor sample. Furthermore, it can be applied to N samples in the top, left, or top / left regions.

[0570] [Formula 18]

[0571] - The error or cost can be calculated using the Mean Reduced-Sum of Absolute Differences (MR-SAD). That is, it can be calculated according to Equation 19 below. In Equation 19, c and p represent the neighbor samples of the current block and the regular predicted block, respectively, and x represents the position of the neighbor sample. Furthermore, it can be applied to N samples in the top, left, or top / left regions.

[0572] [Formula 19]

[0573] - The error or cost can be calculated by considering the sum of squared errors (SSE) between samples. That is, it can be calculated according to Equation 20 below. In Equation 20, c and p represent the neighbor samples of the current block and the regular prediction block, respectively, and x represents the position of the neighbor sample. In addition, it can be applied to N samples in the top, left, or top / left area.

[0574] [Formula 20]

[0575] - The error or cost can be calculated by considering the mean squared error (MSE) between samples. That is, it can be calculated according to Equation 21 below. In Equation 21, c and p represent the neighbor samples of the current block and the regular prediction block, respectively, and x represents the position of the neighbor sample. In addition, it can be applied to N samples in the top, left, or top / left area.

[0576] [Equation 21]

[0577] - The cost calculation method for each block can be variably determined based on the characteristics of the block. For example, for a block with LIC applied, MR-SAD can be considered to calculate the cost. As another example, for a block with BCW applied, MR-SAD can be considered to calculate the cost. As yet another example, for a block with DMVR and BDOF applied, SAD can be considered to calculate the cost.

[0578] The neighbor sample region can consider the top and left 1-pixel rows most adjacent to the current block, or it can consider multiple top and left 1-pixel rows. The size of the sample region can be a fixed region based on rules predetermined between the encoder and decoder, or it can be variably determined and used based on individual signaling notifications or conditions. Conditions used to determine the sample region size can include sample distribution, block size information, detailed prediction modes, etc.

[0579] The predetermined conditions used to determine whether MHP-intra-frame mode is applied can consider the sample distribution of regions adjacent to the current block and the sample distribution of regions adjacent to the prediction block. Specifically, the distribution of the top neighbor sample cA of the current block and the top neighbor sample pA of the regular prediction block can be considered, and the distribution of the left neighbor sample cL of the current block and the left neighbor sample pL of the regular prediction block can also be considered. Furthermore, the top neighbor samples and the left neighbor samples can be compared simultaneously. For ease of description, the upper left portion is not described, but samples at the corresponding positions can be considered when calculating the error between samples using top, left, or top / left samples. The distribution of each sample adjacent to the current block and the regular prediction block can consider variance and standard deviation, and MHP-intra-frame mode is allowed when the difference between the distributions of the blocks is greater than a threshold.

[0580] In addition, the predefined conditions may include block size information. That is, the conditions may include block size (width × height), block width, block height, and block ratio (width / height or height / width). For example, when the block size is less than 32, MHP-intra-frame mode may not be applied. Alternatively, when the block size is greater than 1024, MHP-intra-frame mode may not be applied. As another example, when the width or height is less than 8 or greater than 64, MHP-intra-frame mode may not be applied. Alternatively, the following restrictions may be applied based on the block ratio: - MHP-INTRA is not allowed if (width >= height && width / height > 4). Otherwise, if (width < height && height / width > 4), then MHP-INTRA is not allowed. Furthermore, the predetermined conditions may include the detailed prediction mode of the block. That is, whether the MHP-intra-frame mode is applied can be determined based on the detailed prediction mode of the regular prediction block used to generate the current block. For example, merging modes include various detailed prediction modes such as regular merging, sub-block merging, MMVD, CIIP, GPM, and GPM-INTRA. In particular, since CIIP and GPM-INTRA already contain intra-frame information in the prediction block, the MHP-intra-frame mode can be applied without additional application when these prediction methods are applied to the regular prediction block. Furthermore, the MHP-intra-frame mode may not be applied when using the AMVP mode, which has high motion accuracy due to MVD. Additionally, the MHP-intra-frame mode may not be applied when the block is refined using motion information such as DMVR and BDOF or when prediction samples are refined.

[0581] In this embodiment, applying MHP-intra-frame mode can mean deriving MHP-intra-frame mode in the decoding device.

[0582] Furthermore, the predetermined conditions may include the number of numMHP. Whether to derive an MHP-intra-frame mode can be determined based on the number of MHP-AMVP, MHP-merge, or MHP-intra-frame modes that are signaled or inherited. For example, when numMHP represents the number of signaled and inherited MHP-AMVP / MHP-merge modes, deriving an MHP-intra-frame mode is allowed when numMHP is 0. As another example, deriving an MHP-intra-frame mode is allowed when numMHP is greater than 0 and less than MaxNum. The derived MHP-intra-frame mode can be used in combination with both signaled and inherited MHP-intra-frame modes.

[0583] Furthermore, the above-described implementation can also be applied to the MHP-IBC prediction mode in the same manner. Therefore, the additional prediction mode used to generate additional prediction blocks may include at least one of the MHP-intra-frame prediction mode or the MHP-IBC prediction mode, and information about the MHP-IBC prediction mode can also be derived in the same manner as described above.

[0584] FIG. 37

[0585] This embodiment relates to a method for signaling or deriving information required to generate intra-predictive blocks when MHP-intra-frame mode is applied, that is, when intra-frame mode is used to generate intra-predictive blocks.

[0586] When the value of the MHP-intra_flag is 1, a signal can be sent to notify / parse the information required for applying the MHP-intra_mode, and this information can be used to generate intra-frame additional prediction blocks. In this disclosure, to distinguish it from the prediction modes used in the terms representing MHP-inter-frame mode (or MHP-merging mode, MHP-AMVP) and MHP-intra_mode, the detailed modes, prediction techniques, prediction tools, prediction methods, etc., applied to each prediction mode are collectively referred to as detailed prediction modes. However, the scope of the disclosed embodiments is not limited by such terminology, and even if other terms are used, they may fall within the scope of the disclosed embodiments as long as they have substantially the same meaning as the detailed prediction modes described in this disclosure.

[0587] Specifically, the detailed prediction modes used to generate intra-frame additional prediction blocks can be determined within a specific list, which can be an MPM (Most Probable Mode) list. Among the multiple candidates included in the MPM list, the candidate indicated by an index can be used as the detailed prediction mode, and this index can be defined as a syntax element such as mhp_mpm_idx.

[0588] FIG. 37 This is a flowchart illustrating an example of the operation of signaling / parsing information about MHP-intra-frame mode.

[0589] Reference FIG. 38 The MHP-intra flag (mhp_intra_flag) indicating whether the intra-frame mode is applied to the generation of the additional prediction block (in other words, whether the MHP-intra-frame mode is applied) is signaled / parsed (S1800). When the MHP-intra-frame mode is applied (yes in S1810), the detailed prediction mode information (e.g., mhp_mpm_idx) among the multiple candidates included in the MPM list that are used for the MHP-intra-frame mode can be signaled / parsed (S1820).

[0590] For ease of description, this flowchart only illustrates the signaling / parsing of the MHP MPM index, but additional signaling / parsing of other information required for performing MHP-intra-frame mode can also be added.

[0591] Alternatively, the prediction method applied to MHP-intra-frame mode can be identified as one of the other candidates not belonging to the MPM list. FIG. 38 This is a flowchart illustrating another example of the operation of signaling / parsing information about MHP-intra-frame modes.

[0592] Reference FIG. 39The signal notifies / parses the MHP-intra_flag (mhp_intra_flag) indicating whether the intra-frame mode is applied to the generation of the additional prediction block (in other words, whether the MHP-intra-frame mode is applied) (S1900).

[0593] When the MHP-intra-frame mode is applied, i.e., when the value of the MHP-intra-frame flag is 1 (yes in S1910), a signal can be sent to notify / parse the flag (mhp_mpm_flag) indicating whether the detailed prediction mode applied to the MHP-intra-frame mode exists in the MPM list (S1920). When the value of mhp_mpm_flag is 1, it can indicate that the detailed prediction mode applied to the MHP-intra-frame mode exists in the MPM list, and when the value of mhp_mpm_flag is 0, it can indicate that the detailed prediction mode applied to the MHP-intra-frame mode does not exist in the MPM list.

[0594] When the detailed prediction mode applied to the MHP-intra-frame mode exists in the MPM list, that is, when the value of mhp_mpm_flag is 1 (yes in S1930), information indicating a specific candidate in the MPM list (mhp_mpm_idx) can be signaled / parsed (S1940).

[0595] When the prediction method applied to MHP-intra-frame mode does not exist in the MPM list, that is, when the value of mhp_mpm_flag is 0 (no in S1930), information indicating that it does not belong to one of the other candidates in the MPM list can be signaled / parsed. For example, this information can be signaled / parsed as a syntax element such as mhp_mpm_remainder (S1950).

[0596] The MPM list used for MHP-intra-frame mode can be configured as a separate list and a separate detailed prediction mode distinct from the MPM list used for generating regular intra-frame modes. Alternatively, to reduce encoding / decoding complexity, the same MPM list can be used for both MHP-intra-frame mode and regular intra-frame mode.

[0597] Furthermore, compression efficiency can also be improved by signaling / parsing the detailed prediction mode with the highest probability among the prediction mode candidates before indicating information about specific candidates in the MPM list (e.g., mhp_mpm_idx). FIG. 39 This is a flowchart illustrating yet another example of the operation of signaling / parsing information about MHP-intra-frame modes.

[0598] Reference FIG. 40The signal notifies / parses the MHP-intra_flag (mhp_intra_flag) indicating whether the intra-frame mode is applied to the generation of the additional prediction block (in other words, whether the MHP-intra-frame mode is applied) (S2000).

[0599] When the MHP-intra-frame mode is applied, that is, when the value of the MHP-intra-frame flag is 1 (yes in S2010), a signal can be sent to notify / parse the flag (mhp_mpm_flag) indicating whether the detailed prediction mode applied to the MHP-intra-frame mode exists in the MPM list (S2020).

[0600] When a detailed prediction mode applied to MHP-intra-frame mode exists in the MPM list, i.e., when the value of mhp_mpm_flag is 1 (yes in S2030), information indicating whether a detailed prediction mode with a high selection probability (e.g., planar mode) is used (e.g., mhp_not_planar_flag) can be signaled / parsed first (S2040) to determine whether a detailed prediction mode with a high selection probability is used. In this case, the MPM list can be configured by excluding planar modes as candidates. In this example, considering that planar modes generally have a high selection probability for regular prediction modes, it is determined first whether planar modes are used; however, considering the characteristics of multi-prediction block modes, it is also possible to determine first whether another detailed prediction mode is used.

[0601] When the value of mhp_not_planar_flag is 1 (yes in S2050), it indicates that planar mode is not used, and can signal / parse information indicating a specific candidate in the MPM list (mhp_mpm_idx) (S2060).

[0602] When the detailed prediction mode applied to the MHP-intra-frame mode does not exist in the MPM list, that is, when the value of mhp_mpm_flag is 0 (no in S2030), information indicating that one of the other candidates not belonging to the MPM list can be signaled / parsed (e.g., mhp_mpm_remainder) (S2080).

[0603] When the value of mhp_not_planar_flag is not 1 (no in S2050), it indicates that planar mode is used and additional prediction blocks can be generated using planar mode (S2070).

[0604] The MPM list for MHP-intra-frame modes can include the following candidates and can be configured in a predefined order. Furthermore, when constructing the list, repeating modes can be checked to ensure they are not included: - Non-directional modes, such as planar mode and DC mode, etc. - Angle patterns (angle mode or orientation mode) with specific values -Dividing angles into finer or coarser patterns - Predictive modes of adjacent or non-adjacent blocks encoded / decoded via intra-frame modes and / or modified modes (e.g., mode ±1, mode ±2). - A history-based prediction pattern that includes prediction information for intra-frame blocks encoded / decoded prior to the current block. - Default mode (e.g., flat mode) -ISP mode -MIP mode - Intra-prediction mode derived using DIMD - Intra-prediction mode derived using the TIMD method - The intra-prediction mode and / or modified mode (e.g., mode ±1, mode ±2) stored (used) in the MHP of the current block (among multiple additional prediction blocks). - Intra-prediction modes and / or modified modes (e.g., mode ±1, mode ±2) stored in multiple prediction block modes included in adjacent or non-adjacent blocks encoded / decoded before the current block. - Intra-prediction mode and / or modified mode (e.g., mode ±1, mode ±2) when the prediction mode of the current block is CIIP mode. - Intra-prediction modes and / or modified modes (e.g., mode ±1, mode ±2) of adjacent or non-adjacent blocks encoded / decoded via CIIP modes. - Intra-prediction mode and / or modified mode (e.g., mode ±1, mode ±2) when the prediction mode of the current block is GPM-INTRA mode. - Intra-prediction modes and / or modified modes (e.g., mode ±1, mode ±2) of adjacent or non-adjacent blocks encoded / decoded via GPM-INTRA mode. The size of the MPM list can be determined by a value agreed upon between the encoder and decoder. For example, the size of the MPM list can be set to 6.

[0605] Furthermore, when constructing the MPM list and signaling to mhp_mpm_idx, the candidate order of the MPM list can be changed to reduce signaling bits. For example, template-based cost can be used, and the candidate list can be configured in ascending order of cost. When calculating the cost used to reconstruct the candidate list, SAD, MR-SAD, SSE, MSE, etc., can be used.

[0606] Angle patterns, serving as candidate detailed prediction modes, can be generated by non-uniformly dividing angles to create patterns with corresponding directions. MHP-intra-mode can use such angle patterns and can be selected from a different set of angles than the regular intra-mode, obtained through finer or coarser division. For example, while the regular intra-mode has 65 directional patterns, the MHP-intra-mode can have 129 finer-divided directional patterns or 35 coarser-divided directional patterns. As another example, when the regular intra-mode has 65 directional patterns, angles from other directions not supported in the corresponding direction can be used as candidates.

[0607] FIG. 40 Examples showing the positions of the current block's neighboring and non-neighboring blocks.

[0608] The aforementioned adjacent blocks can refer to FIG. 40 The blocks at positions 1 to 5 in the middle, and non-adjacent blocks can refer to Implementation 5 The blocks located at positions 6 to 23 in the middle.

[0609] MHP-intra-frame mode can also use a detailed prediction mode with fixed values ​​without signaling. The detailed prediction mode with fixed values ​​applied as MHP-intra-frame mode can be determined by the following methods. Alternatively, a combination of two or more of the following methods can be used to determine the detailed prediction mode with fixed values.

[0610] - Planar mode or DC mode can be used as a detailed prediction mode with fixed values. Specifically, planar mode can be selected when reference samples exist, and DC mode can be selected when reference samples do not exist or the number of reference samples is less than a certain number. For example, DC mode can be selected when the number of reference samples is 8. Alternatively, the width + height of the block can be considered to determine the number of reference samples. For example, DC mode can be selected when the width ≥ height and the number is less than height / 2, or when the width < height and the number is less than width / 2.

[0611] - When scanning blocks at predefined adjacent or non-adjacent locations based on the current block, the mode of the first available block can be used as a detailed prediction mode with a fixed value. In this case, availability can refer to a location encoded / decoded in an intra-frame mode and existing within the allowed CTU range. Intra-frame modes can include SGPM modes.

[0612] - In blocks encoded / decoded in intra-frame mode within predefined adjacent or non-adjacent positions of the current block, frequently occurring patterns in the detailed prediction mode can be used as fixed values. When patterns with the same frequency exist, a detailed prediction mode with a fixed value can be determined based on a predefined priority.

[0613] - A history-based detailed prediction pattern that includes prediction information for intra-frame blocks encoded / decoded before the current block can be used as a fixed value. For example, the pattern included in the 0th list of the history buffer can be a detailed prediction pattern with a fixed value.

[0614] - The default mode can be determined as a detailed prediction mode with a fixed value. This method can be used when the intra-frame detailed prediction mode cannot be obtained from adjacent / non-adjacent blocks and history-based storage buffers. In this case, the default detailed prediction mode can be determined as a planar mode.

[0615] -ISP mode

[0616] -MIP mode

[0617] - Intra-prediction mode derived using DIMD

[0618] - Intra-prediction mode derived using the TIMD method

[0619] - The intra-prediction mode and / or modified mode (e.g., mode ±1, mode ±2) stored (used) in the MHP of the current block (among multiple additional prediction blocks).

[0620] - Intra-prediction modes and / or modified modes (e.g., mode ±1, mode ±2) stored in multiple prediction block modes included in adjacent or non-adjacent blocks encoded / decoded before the current block.

[0621] - Intra-prediction mode and / or modified mode (e.g., mode ±1, mode ±2) when the prediction mode of the current block is CIIP mode.

[0622] - Intra-prediction modes and / or modified modes (e.g., mode ±1, mode ±2) of adjacent or non-adjacent blocks encoded / decoded via CIIP modes.

[0623] - Intra-prediction mode and / or modified mode (e.g., mode ±1, mode ±2) when the prediction mode of the current block is GPM-INTRA mode.

[0624] - Intra-prediction modes and / or modified modes (e.g., mode ±1, mode ±2) of adjacent or non-adjacent blocks encoded / decoded via GPM-INTRA mode.

[0625] When one or more intra-frame additional prediction blocks exist, different fixed detailed prediction modes can be applied to each intra-frame additional prediction block. For example, information about the fixed detailed prediction mode for the first intra-frame additional prediction block can be signaled, and information about the fixed detailed prediction mode applied to the next intra-frame additional prediction block can be obtained using the derivation method described above. The methods for determining the fixed detailed prediction modes for the first and second prediction blocks can also be applied in the reverse order described above, and both fixed detailed prediction modes can be obtained either by signaling or by derivation.

[0626] Furthermore, even if an intra-frame additional prediction block exists, i.e., even if an MHP-intra-frame mode exists, the corresponding mode can still use two or more fixed detail prediction modes. For example, an intra-frame prediction block can be generated using two or more fixed detail prediction modes selected through the derivation method described above, and then an intra-frame additional prediction block can be generated by applying a weighted sum. Subsequently, the final prediction block can be generated by a weighted sum with the regular prediction block, as described above.

[0627] Furthermore, the above-described implementation can also be applied to the MHP-IBC prediction mode in the same manner. Therefore, the additional prediction mode used to generate additional prediction blocks may include at least one of the MHP-intra-frame prediction mode or the MHP-IBC prediction mode, and information about the MHP-IBC prediction mode can also be signaled / parsed or derived in the same manner as described above.

[0628] FIG. 41

[0629] This embodiment relates to a method for signaling or obtaining information (e.g., block vector information, weight information, etc.) required to generate an MHP-IBC block when the IBC mode is used as one of the additional prediction modes, that is, when the additional prediction block includes an MHP-IBC block.

[0630] FIG. 41 This is a flowchart illustrating an example of the operation of signaling / parsing the information required to generate an MHP-IBC block.

[0631] Reference FIG. 40 The system can signal / parse MHP-IBC information (e.g., mhp_ibc_flag) indicating whether the MHP-IBC mode is applied to the current block (i.e., whether the IBC mode is applied to the generation of additional prediction blocks) (S2100). When the value of mhp_ibc_flag is 1, it indicates that the MHP-IBC mode is applied. That is, this indicates that an MHP-IBC block exists. When the value of mhp_ibc_flag is not 1 (e.g., when it is 0), it indicates that the MHP-IBC mode is not applied.

[0632] The block vector information used to generate MHP-IBC blocks can be derived using the merge method or the AMVP (ABVP) method. Therefore, when the value of mhp_ibc_flag is 1 (yes in S2110), a signal can be sent to notify / resolve whether the block vector information used to generate MHP-IBC blocks is obtained through the merge method (e.g., mhp_ibc_mrg) (S2120).

[0633] When the value of mhp_ibc_mrg is 1 (yes in S2130), it is possible to signal / parse information (e.g., merge_idx) and weight information (e.g., weight_idx) of a specific candidate that exists in the candidate list of the indicator block vector (S2140).

[0634] When the value of mhp_ibc_mrg is 0 (no in S2130), information of the indicator block vector predictor (e.g., bvp_idx) and information of the indicator block vector difference (e.g., bvd_data) can be signaled / parsed (S2150).

[0635] Furthermore, when one or more MHP-IBC prediction blocks exist, the block vectors of each prediction block can be different from each other. In this case, the candidate lists of block vectors for each MHP-IBC prediction block can exist independently. Alternatively, different block vectors can be indicated by using different indices (e.g., merge_idx) in the same candidate list. Alternatively, the following method can be applied: signaling the index (merge_idx) indicating the block vector of one block, and deriving the index of another block as merge_idx + 1 without signaling.

[0636] The Block Vector Predictor (BVP) candidate list can also be obtained from the neighboring / non-neighboring blocks of the current block. The neighboring / non-neighboring blocks of the current block can represent the above... Implementation 6 The blocks at positions 1 to 5 and positions 6 to 23. The positions and inspection order of neighboring blocks can be partially changed and applied.

[0637] FIG. 42

[0638] This embodiment relates to weight information required for generating the final prediction block using the corresponding prediction block when MHP-intra-frame mode is applied, i.e., when intra-frame mode is used to generate additional prediction blocks to generate intra-predictive blocks. As described above, weights can be applied to the regular prediction block and the additional prediction block in order to generate the final prediction block. This embodiment proposes a signaling notification and derivation method for such weight information. The method described in this embodiment can be applied not only to MHP-intra-frame mode but also to MHP-IBC mode.

[0639] Weight information applied to each additional prediction block can be obtained by signaling a weight index from the weight candidate list. When an MHP-intra-prediction block exists within the additional prediction block, the weight information applied to it can be distinguished from the weight information applied to the MHP-inter-prediction block. In other words, the weight information applied to the MHP-intra-mode can be distinguished from the weight information applied to the MHP-inter-mode. Therefore, the weight candidate list for the MHP-inter-mode and the weight candidate list for the MHP-intra-mode can exist independently. Furthermore, the size of the weight candidate lists can be defined as distinct values.

[0640] Furthermore, when an MHP-IBC prediction block exists within an additional prediction block, the weight information applied to it can be distinguished from the weight information applied to MHP-intra-prediction blocks and MHP-inter-prediction blocks. In other words, the weight information applied to the MHP-IBC mode can be distinguished from the weight information applied to the MHP-intra-prediction and MHP-inter-prediction modes. Therefore, the weight candidate lists for MHP-inter-prediction and MHP-intra-prediction modes, as well as the weight candidate list for the MHP-IBC mode, can exist independently.

[0641] These weights are values ​​defined for the additional prediction blocks. As an example, the weight applied to the additional prediction blocks can be defined as w, and the weight applied to the regular prediction blocks can be defined as 1-w. Suppose the weight w applied to the additional prediction blocks is 4 / 16, then the weight 1-w applied to the regular prediction blocks is 12 / 16.

[0642] FIG. 42 This is a flowchart illustrating another example of the operation of signaling / parsing weight information for MHP-intra-frame mode.

[0643] The method described in this example can be applied in combination with all or part of the methods described in the above embodiments. Therefore, the terms and other content already described above will not be repeated.

[0644] Reference FIG. 43The system sends a signal to notify / parse MHP-intra-frame information (e.g., mhp_intra_flag) indicating whether MHP-intra-frame mode is applied (S2200). When MHP-intra-frame information is applied, i.e., when the value of mhp_intra_flag is 1 (yes in S2210), it can send a signal to notify / parse the weight information (e.g., intra_weight_idx) applied to MHP-intra-frame mode or MHP-intra-frame prediction block (S2230). When intra_weight_idx, which serves as index information, is used as weight information, the weight value indicated by this index can be used to generate the final prediction block.

[0645] Meanwhile, when the MHP-IBC mode is applied, mhp_intra_flag and intra_weight_idx can be replaced with mhp_ibc_flag and ibc_weight_idx respectively, and can be applied together with the motion information used for the MHP-IBC mode.

[0646] In addition, the methods described below can also be applied to the MHP-IBC mode by appropriately replacing the syntax elements.

[0647] FIG. 43 This is a flowchart illustrating yet another example of the operation of signaling / parsing weight information for MHP-intra-frame mode.

[0648] Reference FIG. 44 When the predetermined conditions are met (yes in S2220), a signal can be sent to notify / parse the weight information (S2230). When the predetermined conditions are not met (no in S2220), the default weight information can be used without sending a signal.

[0649] The conditions for determining whether weight information has been signaled / parsed can be determined by applying at least one of the following methods.

[0650] This condition can be determined based on information such as the current block size (width × height), the block width, the block height, and the block ratio (width / height or height / width). For example, when the block size is less than 32, the default weight information can be used. Alternatively, when the block size is greater than 1024, the default weight information can also be used.

[0651] This condition can be determined based on the prediction mode of the current block. For example, when prediction modes such as MMVD, GPM (GPM-INTRA), or CIIP are applied to the current block, default weight information can be used without signaling the weight information. As another example, default weight information can be used when applying a sub-block-based prediction mode. Furthermore, default weight information can be used when applying an AMVP mode with high motion accuracy due to MVD. Additionally, default weight information can be used when the block is refined by motion information or prediction samples applied using methods such as DMVR and BDOF.

[0652] This condition can be determined based on the prediction direction of the current block. For example, whether to signal intra_weight_idx can be determined based on whether unidirectional or bidirectional prediction is applied. As an example, intra_weight_idx is signaled when unidirectional prediction is applied, and the default weight information can be used when bidirectional prediction is applied.

[0653] This condition can be determined based on the BCW (Bidirectional Prediction with CU-level Weights) index of the current block. As an example, when the BCW index does not indicate a specific value (equal weights), the intra_weight_idx can be signaled.

[0654] This condition can be determined based on the LIC (Local Illumination Compensation) flag information of the current block. As an example, when the LIC is applied, it can be signaled to intra_weight_idx.

[0655] This condition can be determined based on the OBMC (Overlapping Block Motion Compensation) flag information of the current block. As an example, when OBMC is applied, default weight information can be used.

[0656] This condition can be determined based on the number of additional prediction blocks. When multiple MHP-intra-prediction blocks exist, only one block can be signaled to intra_weight_idx, and the weights of the remaining blocks can use default values. Alternatively, when MHP-inter-prediction blocks exist, default weight information can be used.

[0657] The default weight information mentioned above can be predefined values. As an example, the default weight values ​​can be equal weights. Alternatively, pixel-based weight information within a block can also be defined as default values.

[0658] FIG. 44 This is a flowchart illustrating yet another example of the operation of signaling / parsing weight information for MHP-intra-frame mode.

[0659] exist FIG. 45In the example, the weight information can be signaled / parsed differently depending on whether a predetermined condition is met. Specifically, when the predetermined condition is met (yes in S2220), the first weight information can be signaled / parsed (S2231), and when the predetermined condition is not met (no in S2220), the second weight information can be signaled / parsed (S2232). As an example, the first weight information can be represented by the syntax element intra_weight_idx0, and the second weight information can be represented by the syntax element intra_weight_idx1.

[0660] That is, the first weight information and the second weight information can be distinguished and applied according to the conditions. The conditions for distinguishing the first weight information and the second weight information can be at least one of the above conditions.

[0661] The weight candidate list that can be indicated by the first weight information and the weight candidate list that can be indicated by the second weight information can be configured independently. Alternatively, the weight candidate list that can be indicated by the first weight information and the weight candidate list that can be indicated by the second weight information can be the same, and the first weight information and the second weight information can also be adjusted to indicate different candidates respectively.

[0662] The candidates for weight information can be configured as follows. However, the candidates for weight information described below are merely examples applicable to one implementation method, and the values, order, number, etc., of the candidates can be changed.

[0663] Table 13 below illustrates an example of using four indices to represent weight information. `intra_weight_idx0` and `intra_weight_idx1` can have weight values ​​with different ranges. When signaling a weight index, the value of `intra_weight_idx` in Table 13 can be referenced.

[0664] [Table 13]

[0665] Alternatively, the weight information can also be configured as shown in Table 14 below. In the example in Table 14, the weight information is represented using two indices. It can be seen that candidates are configured to include candidates with different weight values ​​based on regions within the block, and candidates with the same weight value for all regions within the block. `intra_weight_idx0` (`intra_weight_idx`) illustrates an example where the block is divided into four parts and different weights are applied to each region. The value of `intra_weight_idx` in Table 14 can be referenced when signaling a weight index.

[0666] [Table 14]

[0667] FIG. 45 An example is given where the current block is split when weights are applied to an MHP-intra-prediction block and different weights are applied to the corresponding split regions.

[0668] Reference Implementation 7 The current block can be divided into two or three parts vertically, horizontally, or diagonally, and both uniform and non-uniform division are possible. Furthermore, the current block can be divided into different numbers of regions, i.e., four or more, and uniform / non-uniform division, as well as position-based weights within the block, can be applied. That is, pixel-level weights can be applied.

[0669] The weights applied to the MHP-intra-prediction block are denoted by w, and when different weight information exists for each region, they are denoted as wN (N≥0). Therefore, the weights applied to two split regions can be denoted by w0 and w1, and the weights applied to three split regions can be denoted by w0, w1, and w2.

[0670] The weights can be determined based on the index information signaled, or they can be fixed values ​​without signaling the index information. Furthermore, the weights can be derived by considering the weights of neighboring blocks. The weight information of the additional predicted blocks applied to neighboring blocks can be used as is, or the weight information can be used based on the intra-frame prediction mode. Specifically, in the case of a mode predicting from a top reference sample, the weight of the intra-frame block can be set to decrease as its position increases from the top. Conversely, in the case of a mode predicting from a left reference sample, the weight of the intra-frame block can be set to decrease as its position increases from the left.

[0671] Furthermore, when neighboring blocks are encoded / decoded in intra-frame mode, the weight of the MHP-intra-prediction block can be set to be greater than when neighboring blocks are encoded / decoded in inter-frame or IBC mode. Alternatively, when neighboring blocks are encoded / decoded in intra-frame mode, samples closer to the neighboring block can be assigned a larger weight. That is, weight information can be applied to the entire current block, applied to each split region, or applied based on pixel-level location. Additionally, when weights are applied at the split region level or pixel-level, the weight for a specific region or location can be 0. That is, there may be specific regions and locations where multi-prediction block mode is not applied.

[0672] Furthermore, when neighboring blocks are encoded / decoded in IBC mode, the weight of the MHP-IBC prediction block can be set to be greater than when neighboring blocks are encoded / decoded in inter-frame or intra-frame mode.

[0673] FIG. 46

[0674] This embodiment relates to a method for deriving reference samples for generating MHP-intra-prediction blocks when MHP-intra-prediction blocks exist.

[0675] FIG. 46 This is a flowchart illustrating an example of a decoding method performed in MHP-intra-frame mode according to a decoding method of one embodiment.

[0676] Reference FIG. 47 The prediction mode to be applied to intra-frame prediction is determined (S2300), and a reference sample corresponding to the determined prediction mode is generated (S2310). A prediction block is generated according to the prediction mode using the generated reference sample (S2320). For the generated prediction block, sample refinement can be performed by boundary smoothing or PDPC (S2330).

[0677] In the embodiments described below, the process of generating the reference sample will be described in detail (S2310).

[0678] To generate a reference sample, it is first populated with reconstructed samples adjacent to the current block. If no reference sample exists, it can be populated from neighboring samples using a predefined method.

[0679] When predetermined conditions are met, filtering is performed on the reference sample. For example, a smoothing filter can be applied to correct the reference sample. Furthermore, when the prediction mode requires reference samples at fractional pixel locations, reference samples for each phase can be generated using an interpolation filter. At least one of various filters, such as a cubic interpolation filter and a Gaussian interpolation filter, can be used as the interpolation filter.

[0680] According to one implementation, the strength and type of filtering can be determined by using reference samples. FIG. 47 Examples illustrating the locations of reference samples used to generate MHP-intra-prediction blocks and samples within regular prediction blocks.

[0681] When the difference between the activity A0 of the reference sample used to generate the MHP-intra-prediction block and the activity A1 of the sample within the regular prediction block is greater than a threshold (Tn, n≥0), it can be determined that there is a large discontinuity between the neighboring samples and the regular prediction block. The calculation of sample activity and the determination of whether filtering is applied can be performed using at least one of the following methods: -A0 and A1 are calculated as the variance of the samples in each region, and whether the filter is applied can be determined based on whether |A0 – A1| > T0.

[0682] -A0 and A1 are calculated as the standard deviations of samples in each region, and whether filtering is applied can be determined based on whether |A0 – A1| > T1 holds true.

[0683] -A0 and A1 are calculated as the SAD between the reference sample of the MHP intra-prediction block and the sample in the regular prediction block, and whether filtering is applied can be determined based on whether SAD > T2 holds true.

[0684] -A0 and A1 are calculated as the SSE between the reference sample of the MHP intra-prediction block and the sample in the regular prediction block, and whether filtering is applied can be determined based on whether SSE > T3 holds true.

[0685] In this case, when the number of reference samples in the MHP-intra-prediction block differs from the number of samples in the regular prediction block, additional processes such as matching the number of samples or performing normalization by using the number of samples may be involved. For example... FIG. 48 As illustrated, the range of reference samples used to calculate sample activity can be variably determined based on the intra-frame prediction mode.

[0686] FIG. 49 This example illustrates how, to compare the activity levels of samples corresponding to the same location, the positions of samples in a regular prediction block are altered and applied to regions adjacent to the prediction block. The aforementioned methods for calculating sample activity and applying normalization based on sample size can be applied equivalently.

[0687] Based on the degree of discontinuity and prediction pattern defined by K thresholds (K>0, integers), the reference sample can be filtered in the following manner.

[0688] - Unfiltered reference samples can be used as is.

[0689] - The applied filter can be [1, 2, 1] / 4.

[0690] - The applied filter can be [2, 3, 6, 3, 2] / 16.

[0691] - The filter used can be a strong filter. FIG. 49 This example demonstrates how a strong filter is applied to the current block. Reference samples can be generated based on distance ratios using either top-left and top-right reference samples or top-left and bottom-left reference samples. Implementation 8 In the example, 63 can be changed and applied to ((width) m) -1), ((height) m) -1). (m > 1, an integer)

[0692] Furthermore, the above-described embodiments can be equally applied to the MHP-IBC prediction mode. Therefore, the additional prediction mode used to generate additional prediction blocks may include at least one of the MHP-intra-frame prediction mode or the MHP-IBC prediction mode, and the above method can also be applied when generating prediction blocks using the MHP-IBC prediction mode.

[0693] FIG. 46

[0694] This embodiment relates to a filtering method during the generation of intra-prediction blocks when an intra-prediction block exists as an additional prediction block, that is, when an MHP-intra-prediction block exists.

[0695] In intra-frame prediction modes, especially in directional prediction modes, there is a problem of discontinuities appearing in regions (or boundaries) that are not adjacent to the reference sample. To solve this problem, as mentioned above... FIG. 50 As described, techniques such as boundary smoothing or PDPC can be applied to the boundary portions of the generated intra-prediction blocks. That is, the sample refinement of the prediction blocks during the decoding process described above can improve some samples based on the generated prediction blocks by using at least one of boundary smoothing or PDPC, thereby generating corrected prediction blocks.

[0696] One implementation of the proposed method can be applied to the sample refinement process of a prediction block. Specifically, after generating a prediction block using the MHP-intra-frame mode, refinement with boundary filtering characteristics can be applied to the prediction block. For this purpose, information about the filters applied to the MHP-intra-frame prediction block (e.g., mhp_filter_flag or mhp_filter_idx) can be signaled / parsed.

[0697] FIG. 50 This is a flowchart illustrating an example of the operation of signaling / parsing filtered information for MHP-intra-frame mode.

[0698] Reference FIG. 51 The system signals / parses MHP-intra-frame information (e.g., mhp_intra_flag) (S2400), and when the MHP-intra-frame information indicates that an MHP-intra-frame mode is applied, such as when the value of mhp_intra_flag is 1 (Yes in S2410), it can signal / parse MHP-intra-frame filter information (S2430). The MHP-intra-frame filter information indicates information about the filters applied to the MHP-intra-frame prediction block and may include syntax elements such as mhp_filter_flag.

[0699] For example, when the value of mhp_filter_flag is 1, a pixel-position-based thinning process can be performed in the MHP-intra-prediction block using predefined filters. When the value of mhp_filter_flag is 0, simplified boundary filtering can be applied or no filtering can be applied.

[0700] The method described in this embodiment can be combined with the embodiments described above. For example, it can be signaled / parsed together with all or part of syntax elements such as prediction mode information for MHP-intra-frame modes, mhp_mpm_flag, mhp_not_planar_flag, mhp_mpm_idx, and mhp_mpm_remainder. Furthermore, it can be signaled / parsed together with weight information applied to MHP-intra-frame prediction blocks (e.g., intra_weight_idx, intra_weight_idx0, intra_weight_idx1). Additionally, the various embodiments described above can be applied together with the MHP-intra-frame filter information described in this embodiment.

[0701] FIG. 51 This is a flowchart illustrating another example of the operation of signaling / parsing filtered information for MHP-intra-frame mode.

[0702] according to FIG. 50 Examples can be found in FIG. 52 In the illustrated process, before signaling / parsing the MHP-intra-frame filter information (S2430), an additional process is performed to determine whether a predetermined condition is met (S2420). Therefore, in this example, when the predetermined condition is met (yes in S2420), the MHP-intra-frame filter information (e.g., mhp_filter_flag) can be signaled / parsed (S2430).

[0703] The conditions for determining whether MHP-intra-frame filter information has been signaled / parsed can be determined by applying at least one of the following methods.

[0704] This condition can be determined based on information such as the current block size (width × height), the block width, the block height, and the block ratio (width / height or height / width). For example, when the block size is less than 32, MHP-intra-frame filter information may not be signaled / parsed. Alternatively, when the block size is greater than 1024, MHP-intra-frame filter information may not be signaled / parsed.

[0705] This condition can be determined based on the prediction mode of the current block. For example, when prediction modes such as MMVD, GPM (GPM-INTRA), or CIIP are applied to the current block, MHP-intra-frame filter information may not be signaled / resolved. As another example, when a sub-block-based prediction mode is applied, MHP-intra-frame filter information may not be signaled / resolved. Furthermore, when an AMVP mode with high motion accuracy due to MVD is applied, MHP-intra-frame filter information may not be signaled / resolved. Additionally, when the block is a block refined by motion information refinement or prediction sample refinement using methods such as DMVR and BDOF, MHP-intra-frame filter information may not be signaled / resolved.

[0706] This condition can be determined based on the prediction direction of the current block. For example, whether to signal / resolve the `mhp_filter_flag` can be determined based on whether unidirectional or bidirectional prediction is applied. For instance, when unidirectional prediction is applied, the `mhp_filter_flag` can be signaled / resolved, and when bidirectional prediction is applied, the `mhp_filter_flag` can be left unsigned / unresolved.

[0707] This condition can be determined based on the BCW (Bidirectional Prediction with CU-level Weights) index of the current block. For example, when the BCW index does not indicate a specific value, the mhp_filter_flag can be signaled / resolved.

[0708] This condition can be determined based on the LIC (Local Illumination Compensation) flag information of the current block. For example, when the LIC is applied, the mhp_filter_flag can be signaled / resolved.

[0709] This condition can be determined based on the OBMC (Overlapping Block Motion Compensation) flag information of the current block. For example, when OBMC is applied, the mhp_filter_flag may not be signaled or parsed.

[0710] This condition can be determined based on the number of additional prediction blocks. When multiple MHP-intra-prediction blocks exist, the mhp_filter_flag may not be signaled / resolved. Alternatively, when MHP-inter-prediction blocks exist, the mhp_filter_flag may not be signaled / resolved.

[0711] The sample refinement method applied to MHP-intra-prediction blocks can be determined based on the information provided by the aforementioned signal notification, or it can be determined as a fixed value without signal notification. Furthermore, the method can be derived by considering the filter information of neighboring blocks. For example, the filter information of neighboring blocks can be used as is. Additionally, predefined filter information can be used based on the directionality of the intra-prediction mode. For example, when applying a vertical direction mode, the prediction block is generated using top reference samples, so the discontinuity with the left neighbor samples may be relatively increased. Therefore, the samples at that location can be refined to compensate for this discontinuity. As another example, when applying a horizontal direction mode, the prediction block is generated using left reference samples, so the discontinuity with the top neighbor samples may be increased. Therefore, the samples at that location can be refined to compensate for this discontinuity. In this case, the closer to the block boundary, the larger the range of refinement values ​​can be.

[0712] The sample refinement method can be applied to all samples in a block, or only to a portion of the region. Furthermore, the range of refinement values ​​can vary depending on the sample's location. For example, as mentioned above, the closer to the block boundary, the larger the range of refinement values ​​can be.

[0713] Furthermore, the above-described implementation can also be applied to the MHP-IBC prediction mode in the same manner. Therefore, the additional prediction mode used to generate additional prediction blocks may include at least one of the MHP-intra-frame prediction mode or the MHP-IBC prediction mode, and when the MHP-IBC prediction mode is applied, the filtering information can also be signaled / parsed in the same manner as described above.

[0714] Although the various embodiments are described separately for ease of explanation, combinations of two or more embodiments are possible, and the necessary modifications to the combinations of embodiments may also be included within the scope of the disclosed invention or the disclosed embodiments.

[0715] FIG. 52 Examples of content streaming systems to which embodiments of the present disclosure can be applied are shown.

[0716] refer to ​ The content streaming system using the embodiments of this disclosure may mainly include an encoding server, a streaming server, a web server, a media storage device, a user device, and a multimedia input device.

[0717] An encoding server generates a bitstream by compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data, and then sends it to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders generate bitstreams directly, the encoding server can be omitted.

[0718] Bitstreams can be generated by applying the encoding method or bitstream generation method of the embodiments of this disclosure, and the streaming server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0719] A streaming server sends multimedia data to a user's device via a web server based on the user's request, and the web server acts as a medium to notify the user what services are available. When a user requests a service from the web server, the web server delivers it to the streaming server, and the streaming server sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server, which in this case controls the commands / responses between each device in the content streaming system.

[0720] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store bitstreams for specific time periods.

[0721] Examples of user equipment may include mobile phones, smartphones, laptops, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays (HMDs), digital TVs, desktop computers, digital signage, etc.).

[0722] In a content streaming system, each server can be operated as a distributed server, and in this case, data received from each server can be distributed and processed.

[0723] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of this disclosure can be combined and implemented as a device, and the technical features of the device claims of this disclosure can be combined and implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims of this disclosure can be combined and implemented as a device, and the technical features of the method claims and the technical features of the device claims of this disclosure can be combined and implemented as a method.

[0724] Industrial applicability

[0725] The embodiments of this disclosure can be used to encode / decode images.

Claims

1. A decoding method, the decoding method comprising: Obtain information about common forecasting patterns; Generate a conventional prediction block based on the conventional prediction pattern; Obtain information about additional forecasting models; Generate additional prediction blocks based on the aforementioned additional prediction pattern; The final prediction block is generated based on the combination of the regular prediction block and the additional prediction block. The additional prediction mode includes at least one of intra-frame prediction mode or IBC prediction mode. The information regarding the additional prediction mode includes at least one of information indicating whether to apply the intra-frame prediction mode to generate the additional prediction block or information indicating whether to apply the IBC prediction mode to generate the additional prediction block.

2. The decoding method according to claim 1, wherein, Obtaining information about the additional prediction mode includes obtaining at least one of the following: information indicating whether to apply the intra-prediction mode or information indicating whether to apply the IBC prediction mode to generate the additional prediction block, before obtaining the information indicating whether to apply the inter-frame prediction mode to generate the additional prediction block.

3. The decoding method according to claim 1, wherein, Obtaining the information about the additional prediction mode includes: identifying whether a determination condition is met, and based on the determination condition being met, obtaining at least one of the information indicating whether to apply the intra-frame prediction mode or the information indicating whether to apply the IBC prediction mode to generate additional prediction blocks.

4. The decoding method according to claim 1, wherein, Obtaining information about the additional prediction mode includes: obtaining the information indicating whether to apply the inter-frame prediction mode to generate the additional prediction block before at least one of the information indicating whether to apply the intra-frame prediction mode to generate the additional prediction block or the information indicating whether to apply the IBC prediction mode to generate the additional prediction block.

5. The decoding method according to claim 1, wherein, The information regarding the additional prediction mode includes detailed prediction mode information regarding at least one of the intra-prediction mode or IBC prediction mode used to generate the additional prediction block. The process of obtaining the information about the additional prediction mode includes: generating the additional prediction block based on applying at least one of the intra prediction mode or the IBC prediction mode, and obtaining detailed prediction mode information about the intra prediction mode or the IBC prediction mode used to generate the additional prediction block.

6. The decoding method according to claim 1, wherein, The information regarding the additional prediction mode includes weight information for intra-predictive additional prediction blocks generated based on the intra-predictive mode or IBC-predictive additional prediction blocks generated based on the IBC prediction mode. Generating the final prediction block includes performing a weighted summation on the regular prediction block and the additional prediction block.

7. The decoding method according to claim 6, wherein, The weight information includes weight values ​​that are different from each other. Generating the final prediction block includes applying the weight value to the partitioned region obtained by dividing the intra-frame additional prediction block or the IBC additional prediction block.

8. An encoding method, the encoding method comprising: Based on the merge pattern being determined as the prediction pattern for the current block, information about the prediction is derived from neighboring blocks; as well as A predicted block for the current block is generated based on the information derived from the neighboring blocks. Generating the predicted block for the current block includes: applying the multi-prediction block pattern to the current block based on information derived from the neighboring blocks, including information about the multi-prediction block pattern.

9. The encoding method according to claim 8, wherein, Deducing information about the prediction from the neighboring blocks includes: identifying whether a certain condition is met, and based on the fact that the certain condition is met, deriving the information about the multi-prediction block pattern.

10. The encoding method according to claim 8, wherein, Deducing information about the prediction from the neighboring blocks includes: identifying whether a certain condition is met, and based on the condition being met, deriving information about the intra-prediction mode or IBC prediction mode from the information about the multi-prediction block mode.

11. An encoding method, the encoding method comprising: Generate information indicating whether to apply the multi-prediction block pattern to the current block; Based on applying the multi-prediction block pattern to the current block, information about the regular prediction pattern used to generate regular prediction blocks and information about the additional prediction pattern used to generate additional prediction blocks are generated. as well as The information indicating whether the multi-prediction block mode is applied to the current block, the information regarding the regular prediction mode used to generate the regular prediction block, and the information regarding the additional prediction mode used to generate the additional prediction block are encoded. The additional prediction mode includes at least one of intra-frame prediction mode or IBC prediction mode. The information regarding the additional prediction mode includes at least one of information indicating whether to apply the intra-frame prediction mode to generate the additional prediction block or information indicating whether to apply the IBC prediction mode to generate the additional prediction block.

12. An encoding method, the encoding method comprising: Based on the merge pattern being determined as the prediction pattern for the current block, information indicating the application of the merge pattern to the current block and a merge candidate list are generated; Select a candidate from the list of merged candidates and generate information indicating the selected candidate; The information indicating whether to apply the merge mode to the current block and the information indicating the selected candidate are encoded. The selected candidates correspond to neighboring blocks that have applied the multi-prediction block mode. The information about the multi-prediction block pattern applied to the selected candidate is derived as information about the prediction for the current block.

13. A computer-readable storage medium for storing a bit stream, the bit stream comprising: Indicates whether to apply the multi-prediction block mode to the information of the current block; Information about the regular prediction patterns used to generate regular prediction blocks; as well as Information regarding the additional prediction patterns used to generate additional prediction blocks. The additional prediction mode includes at least one of intra-frame prediction mode or IBC prediction mode. The information regarding the additional prediction mode includes at least one of information indicating whether to apply the intra-frame prediction mode to generate the additional prediction block or information indicating whether to apply the IBC prediction mode to generate the additional prediction block.

14. A computer-readable storage medium for storing a bit stream, the bit stream comprising: Indicates the information to be applied to the current block using the merge pattern; Information indicating the candidate selected from the merge candidate list; as well as Information regarding predictions for the selected candidates, The selected candidates correspond to neighboring blocks that have applied the multi-prediction block mode. The information about the multi-prediction block pattern applied to the selected candidate is derived as information about the prediction for the current block.

15. A method for transmitting image information, the method comprising: Generate the image information including a bitstream, wherein the bitstream includes information indicating whether a multi-prediction block mode is applied to the current block, information about the regular prediction mode used to generate regular prediction blocks, and information about the additional prediction mode used to generate additional prediction blocks; and Send the image information including the bitstream. The additional prediction mode includes at least one of intra-frame prediction mode or IBC prediction mode. The information regarding the additional prediction mode includes at least one of information indicating whether to apply the intra-frame prediction mode to generate the additional prediction block or information indicating whether to apply the IBC prediction mode to generate the additional prediction block.

16. A method for sending image information, the method comprising: Generate the image information including a bitstream, wherein the bitstream includes information indicating the application of a merging mode to the current block, information indicating the candidate selected from the merging candidate list, and information about the prediction for the selected candidate; and Send the image information including the bitstream. The selected candidates correspond to neighboring blocks that have applied the multi-prediction block mode. The information about the multi-prediction block pattern applied to the selected candidate is derived as information about the prediction for the current block.

Images