Image encoding / decoding method and device, and recording medium on which bitstream is stored

Abstract

An image encoding / decoding method and a device, a recording medium on which a bitstream is stored, and a transmission method are provided. The image decoding method may comprise the steps of: configuring a candidate list including a plurality of candidates for the current block; calculating a template matching cost for each candidate on the basis of the difference between a template of the current block and a template corresponding to a candidate block designated by each candidate of the candidate list; rearranging the order of the candidate list on the basis of the template matching cost, and selecting one or more candidates from the rearranged candidate list; and predicting the current block on the basis of the selected one or more candidates.

Description

Video encoding / decoding method, device, and recording medium storing a bitstream

[0001] The present disclosure relates to a video encoding / decoding method, an apparatus, and a recording medium storing a bitstream. Specifically, the present disclosure relates to template matching-based video encoding and decoding.

[0002] Recently, the demand for high-resolution, high-quality video, such as UHD (Ultra High Definition) video, has been increasing across various application fields. As video data becomes higher in resolution and quality, the relative volume of data increases compared to conventional video data; consequently, transmission and storage costs increase when video data is transmitted using existing wired or wireless broadband lines or stored using existing storage media. To address these issues arising from the increase in video data resolution and quality, high-efficiency video encoding and decoding technologies for video with higher resolution and quality are required.

[0003] Intra-frame prediction is a method that predicts samples of a block by referencing samples of the same picture. In the case of picture I, where there is no picture to reference, the block is encoded / decoded solely through intra-frame prediction; therefore, there is a strong demand to improve the accuracy of intra-frame prediction. Accordingly, various intra-frame prediction methods are being discussed to enhance the accuracy of intra-frame prediction.

[0004] The present disclosure aims to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] In addition, the present disclosure aims to provide a recording medium storing a bitstream generated by an image decoding method or device according to the present disclosure.

[0006] A video decoding method according to one embodiment of the present disclosure may be characterized by comprising: a step of forming a candidate list including a plurality of candidates for a current block; a step of calculating a template matching cost for each candidate based on the difference between a template of the current block and a template corresponding to a candidate block designated by each candidate of the candidate list; a step of rearranging the order of the candidate list based on the template matching cost and selecting one or more candidates from the rearranged candidate list; and a step of predicting the current block based on the one or more selected candidates.

[0007] According to one embodiment, the template of the current block may be composed of a restoration sample adjacent to the upper or left side of the current block.

[0008] According to one embodiment, template search may be stopped based on the template matching cost.

[0009] According to one embodiment, the cessation of the template search may be determined when the minimum value of the template matching cost calculated for the candidates in the candidate list exceeds a predefined threshold.

[0010] According to one embodiment, the cost function for determining the template matching cost includes a term regarding distortion, and the cost function may be defined to further include at least one of a term regarding a bit rate or a term regarding complexity.

[0011] According to one embodiment, the term regarding the bit rate may be defined by at least one of 1) a bit size calculated by CABAC, 2) a lookup table defining the bit rate and template matching cost, and 3) a value of a differential motion vector.

[0012] According to one embodiment, the term regarding complexity may be defined by at least one of 1) the number of operations required to calculate the template matching cost, 2) the memory size required for additional information stored for template matching, 3) a lookup table defining the complexity and the template matching cost, and 4) the distance between the current block and the candidate template.

[0013] According to one embodiment, the candidate list may be characterized as being an AMVP (advanced motion vector prediction) candidate list, a merge candidate list, a GPM (geometric partitioning mode) candidate list, an SGPM (spatial geometric partitioning mode) candidate list, an IBC (intra block copy) candidate list, or an intra template matching prediction candidate list.

[0014] According to one embodiment, the AMVP candidate list is composed of up to 10 candidates by sequentially adding spatial MVP (motion vector prediction) candidates, temporal MVP candidates, non-adjacent MVP candidates, and HMVP (historical motion vector prediction) candidates without duplication, and the order of the AMVP candidates can be reordered based on the template matching cost.

[0015] According to one embodiment, the merge candidate list is one of a general merge candidate list, a template matching-based merge candidate list, and an affine merge candidate list, and for each merge candidate in the merge candidate list, a template matching cost between the template of the current block and the template of the corresponding merge candidate is calculated, and the order of the merge candidates may be reordered according to the ascending order of the template matching costs.

[0016] According to one embodiment, the GPM candidate list includes a partition mode candidate for a method of partitioning the current block into two partition regions, and based on information related to the motion vector of the current block, the template matching cost of each partition mode candidate of the GPM candidate list is calculated, and the order of the partition mode candidates can be rearranged according to the ascending order of the template matching costs.

[0017] According to one embodiment, the SGPM candidate list includes SGPM candidates composed of a combination of a split mode index, a first intra-frame prediction mode index, and a second intra-frame prediction mode index, and the template matching cost of each SGPM candidate in the SGPM candidate list is calculated, and the candidate order of the SGPM candidates can be reordered based on the calculated template matching cost.

[0018] According to one embodiment, the template matching cost for each IBC candidate in the IBC candidate list is calculated, and the cost function for calculating the template matching cost of the IBC candidate is defined such that the closer the distance between the current block and the IBC prediction block, the smaller the template matching cost becomes, and the candidate order of the IBC candidate can be reordered based on the template matching cost calculated by the cost function.

[0019] According to one embodiment, a template matching cost is calculated for each of the in-screen template matching prediction candidates in the in-screen template matching prediction candidate list, and the reference search area for in-screen template matching is divided into a plurality of sub-areas, and the template matching cost is determined based on at least one of the area index of the sub-area, the size of the sub-area, and the distance between the current block and the sub-area, and the candidate order of the in-screen template matching prediction candidates can be reordered based on the template matching cost.

[0020] According to one embodiment, a template matching cost for each IBC candidate in the IBC candidate list is calculated, the template matching cost is calculated or adjusted according to the block vector type of the IBC candidate, and the candidate order of the IBC candidates can be reordered based on the template matching cost calculated for each IBC candidate.

[0021] A video encoding method according to one embodiment of the present disclosure may include: a step of forming a candidate list including a plurality of candidates for a current block; a step of calculating a template matching cost for each candidate based on the difference between a template of the current block and a template corresponding to a candidate block designated by each candidate in the candidate list; a step of rearranging the order of the candidate list based on the template matching cost and selecting one or more candidates from the rearranged candidate list; and a step of predicting the current block based on the one or more selected candidates.

[0022] A non-transient computer-readable recording medium according to one embodiment of the present disclosure can store a bitstream generated by an image encoding method according to various embodiments of the present disclosure.

[0023] A bitstream transmission method according to one embodiment of the present disclosure includes the step of transmitting the bitstream, and can transmit a bitstream generated by an image encoding method according to various embodiments of the present disclosure.

[0024] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0025] According to the present disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency may be provided.

[0026] In addition, according to the present disclosure, a method for improving the accuracy of intra-prediction based on template matching may be provided.

[0027] In addition, according to the present disclosure, a cost function for calculating the template matching cost is provided, so that the rearrangement of candidates within the candidate list is performed efficiently, thereby improving prediction efficiency.

[0028] In addition, according to the present disclosure, as various in-frame prediction modes using templates are provided, the prediction efficiency of in-frame prediction can be improved.

[0029] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0030] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present disclosure applies.

[0031] FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present disclosure is applied.

[0032] FIG. 3 is a schematic diagram illustrating a video coding system to which the present disclosure can be applied.

[0033] Figure 4 shows an example of template matching.

[0034] Figure 5 illustrates a template matching-based prediction process for in-screen prediction.

[0035] Figure 6 shows the template region and reference region of TIMD.

[0036] Figure 7 illustrates multiple restored sample lines referenced in the in-frame prediction.

[0037] FIG. 8 illustrates an example of an input / output structure of a filter used in an EIP.

[0038] FIG. 9 illustrates examples of reference regions for deriving EIP coefficients around the current block.

[0039] Figure 10 illustrates an embodiment of an SGPM candidate.

[0040] FIG. 11 illustrates an example of an SGPM template.

[0041] FIG. 12 is a flowchart illustrating an image decoding method according to one embodiment of the present disclosure.

[0042] FIG. 13 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0043] A video decoding method according to one embodiment of the present disclosure may be characterized by comprising: a step of forming a candidate list including a plurality of candidates for a current block; a step of calculating a template matching cost for each candidate based on the difference between a template of the current block and a template corresponding to a candidate block designated by each candidate of the candidate list; a step of rearranging the order of the candidate list based on the template matching cost and selecting one or more candidates from the rearranged candidate list; and a step of predicting the current block based on the one or more selected candidates.

[0044] The present disclosure is subject to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the present disclosure. Similar reference numerals in the drawings refer to the same or similar functions across various aspects. The shapes and sizes of elements in the drawings may be provided illustratively for clearer explanation. The detailed description of the exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present disclosure in relation to one embodiment. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of the exemplary embodiments is limited only by the appended claims, together with all equivalents to those claimed therein, provided they are properly described.

[0045] In this disclosure, terms such as first, second, etc. may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of this disclosure, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0046] The components shown in the embodiments of the present disclosure are depicted independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for convenience of explanation; however, at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are included within the scope of the rights of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0047] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit this disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. Additionally, some components of this disclosure may not be essential components performing an essential function in this disclosure, but may be optional components merely for enhancing performance. This disclosure may be implemented by including only the components essential to embody the essence of this disclosure, excluding components used merely for performance enhancement, and a structure including only the essential components, excluding optional components used merely for performance enhancement, is also included within the scope of this disclosure.

[0048] In the embodiments, the term "at least one" may mean one of a number of 1 or more, such as 1, 2, 3, and 4. In the embodiments, the term "a plurality of" may mean one of a number of 2 or more, such as 2, 3, and 4.

[0049] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In describing the embodiments of this specification, if it is determined that a detailed description of related known configurations or functions may obscure the gist of this specification, such detailed description is omitted, and the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0050] Glossary

[0051] In the following, “image” may refer to a single picture constituting a video, or it may refer to the video itself. For example, “encoding and / or decoding of an image” may mean “encoding and / or decoding of an image”, and may also mean “encoding and / or decoding of one of the images constituting the video”.

[0052] In the following, "video" and "video" may be used interchangeably with the same meaning. Additionally, the target image may be an image to be encoded and / or an image to be decoded. Furthermore, the target image may be an input image fed into an encoding device and an input image fed into a decoding device. Here, the target image may have the same meaning as the current image.

[0053] In the following, the terms encoder and image encoding device may be used interchangeably.

[0054] In the following, the decoder and the image decoder may be used interchangeably with each other.

[0055] In the following, "image," "picture," "frame," and "screen" may be used interchangeably with the same meaning.

[0056] In the following, “target block” may be an encoding target block that is the target of encoding and / or a decoding target block that is the target of decoding. Additionally, the target block may be a current block that is the target of current encoding and / or decoding. For example, “target block” and “current block” may be used interchangeably.

[0057] In the following description, "block" and "unit" may be used interchangeably. Additionally, to distinguish it from a block, "unit" may refer to a block containing a luminance (Luma) component block and a corresponding chroma (Chroma) component block. For example, a Coding Tree Unit (CTU) may consist of a single luminance component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated with it.

[0058] In the following, “sample,” “pixel,” and “pixel” may be used interchangeably with the same meaning. Here, a sample may represent a basic unit constituting a block.

[0059] In the following, “inter” and “inter-screen” may be used interchangeably with the same meaning.

[0060] In the following, “intra” and “in-screen” may be used interchangeably with the same meaning.

[0061]

[0062] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present disclosure applies.

[0063] The encoding device (100) may be an encoder, a video encoding device, or an image encoding device. The video may include one or more images. The encoding device (100) may sequentially encode one or more images.

[0064] Referring to FIG. 1, the encoding device (100) may include an image segmentation unit (110), an intra prediction unit (120), a motion prediction unit (121), a motion compensation unit (122), a switch (115), a subtractor (113), a converter (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse converter (170), an adder (117), a filter unit (180), and a reference picture buffer (190).

[0065] Additionally, the encoding device (100) can generate a bitstream containing encoded information through encoding of an input image and can output the generated bitstream. The generated bitstream can be stored on a computer-readable recording medium or streamed via a wired / wireless transmission medium.

[0066] The video segmentation unit (110) can divide the input video into various forms to increase the efficiency of video encoding / decoding. That is, the input video consists of multiple pictures, and a single picture can be processed by hierarchically dividing it for compression efficiency, parallel processing, etc. For example, a single picture can be divided into one or more tiles or slices and then divided again into multiple CTUs (Coding Tree Units). Alternatively, a single picture can first be divided into multiple sub-pictures defined as groups of rectangular slices, and each sub-picture can be divided into the said tiles / slices. Here, the sub-pictures can be utilized to support the function of partially and independently encoding / decoding and transmitting the picture. Since multiple sub-pictures can each be restored individually, they have the advantage of being easy to edit in applications that configure multi-channel inputs into a single picture. In addition, the tiles can be divided horizontally to create bricks. Here, a brick can be utilized as the basic unit of parallel processing within a picture. Additionally, a single CTU can be recursively partitioned into a Quadtree (QT), and the terminal node of the partition can be defined as a Coding Unit (CU). The CU can be divided into a Prediction Unit (PU) and a Transform Unit (TU) to perform prediction and partitioning. Meanwhile, the CU can be utilized as the prediction unit and / or the transformation unit itself. Here, for flexible partitioning, each CTU can be recursively partitioned into a Multi-Type Tree (MTT) as well as a Quadtree (QT). The partitioning of the CTU into a Multi-Type Tree can begin at the terminal node of the QT, and the MTT can be composed of a Binary Tree (BT) and a Triple Tree (TT).For example, the MTT structure can be classified into vertical binary splitting mode (SPLIT_BT_VER), horizontal binary splitting mode (SPLIT_BT_HOR), vertical ternary splitting mode (SPLIT_TT_VER), and horizontal ternary splitting mode (SPLIT_TT_HOR). Additionally, when splitting, the minimum block size (MinQTSize) of the quad tree for the luminance block can be set to 16x16, the maximum block size (MaxBtSize) of the binary tree to 128x128, and the maximum block size (MaxTtSize) of the triple tree to 64x64. Furthermore, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the triple tree can be set to 4x4, and the maximum depth (MaxMttDepth) of the multi-type tree can be set to 4. Additionally, to increase the encoding efficiency of the I slice, a dual tree can be applied that uses different CTU splitting structures for the luminance and chrominance components. On the other hand, in P and B slices, the luminance and color difference CTBs (Coding Tree Blocks) within the CTU can be divided into a single tree that shares a coding tree structure.

[0067] The encoding device (100) may perform encoding on an input image in an intra mode and / or inter mode. Alternatively, the encoding device (100) may perform encoding on an input image in a third mode other than the intra mode and inter mode (e.g., IBC mode, Palette mode, etc.). However, if the third mode has functional characteristics similar to the intra mode or inter mode, it may be classified as an intra mode or inter mode for convenience of explanation. In this disclosure, the third mode will be classified and described separately only when a specific description of the third mode is required.

[0068] When intra mode is used as the prediction mode, the switch (115) can be switched to intra, and when inter mode is used as the prediction mode, the switch (115) can be switched to inter. Here, intra mode may mean an intra-frame prediction mode, and inter mode may mean an inter-frame prediction mode. The encoding device (100) can generate a prediction block for an input block of an input image. Additionally, after the prediction block is generated, the encoding device (100) can encode a residual block using the residual of the input block and the prediction block. The input image may be referred to as the current image that is the subject of current encoding. The input block may be referred to as the current block that is the subject of current encoding or the encoding target block.

[0069] When the prediction mode is an intra mode, the intra prediction unit (120) may use a sample of a block that has already been encoded / decoded around the current block as a reference sample. The intra prediction unit (120) may perform spatial prediction for the current block using the reference sample and generate prediction samples for the input block through spatial prediction. Here, intra prediction may mean intra-frame prediction.

[0070] In the intra prediction method, non-directional prediction modes such as DC mode and Planar mode, and directional prediction modes (e.g., 65 directions) may be applied. Here, the intra prediction method can be expressed as an intra prediction mode or an intra-frame prediction mode.

[0071] When the prediction mode is an inter mode, the motion prediction unit (121) can search for the region that best matches the input block from the reference image during the motion prediction process and derive a motion vector using the searched region. At this time, the search region can be used as the region. The reference image can be stored in the reference picture buffer (190). Here, the reference image can be stored in the reference picture buffer (190) when encoding / decoding of the reference image is processed.

[0072] The motion compensation unit (122) can generate a prediction block for the current block by performing motion compensation using a motion vector. Here, inter-prediction may mean inter-frame prediction or motion compensation.

[0073] The motion prediction unit (121) and motion compensation unit (122) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform inter-frame prediction or motion compensation, based on the encoding unit, it can determine whether the motion prediction and motion compensation method of the prediction unit included in the corresponding encoding unit is a Skip Mode, Merge Mode, Advanced Motion Vector Prediction (AMVP) Mode, or Intra Block Copy (IBC) Mode, and can perform inter-frame prediction or motion compensation according to each mode.

[0074] In addition, based on the above-mentioned inter-frame prediction method, the AFFINE mode of sub-PU-based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and GPM (Geometric Partitioning Mode) mode of PU-based prediction may be applied. Furthermore, to improve the performance of each mode, HMVP (History based MVP), PAMVP (Pairwise Average MVP), CIIP (Combined Intra / Inter Prediction), AMVR (Adaptive Motion Vector Resolution), BDOF (Bi-Directional Optical-Flow), BCW (Bi-predictive with CU Weights), LIC (Local Illumination Compensation), TM (Template Matching), OBMC (Overlapped Block Motion Compensation), etc. may be applied.

[0075] Among these, AFFINE mode is a technology used in both AMVP and MERGE modes and also offers high encoding efficiency. Conventional video coding standards have the disadvantage of failing to properly compensate for real-world movements, such as zoom in / out and rotation, because they perform Motion Compensation (MC) by considering only the translation of blocks. To address this, a 4-parameter affine motion model using two control point motion vectors (CPMV) and a 6-parameter affine motion model using three control point motion vectors can be applied to inter-prediction. Here, CPMV is a vector representing one of the affine motion models of the top-left, top-right, or bottom-left corners of the current block.

[0076] The subtractor (113) can generate a residual block using the difference between the input block and the prediction block. The residual block may also be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or both transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal in block units.

[0077] The transformation unit (130) can generate a transform coefficient by performing a transform on the remaining block and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the remaining block. When a transform skip mode is applied, the transformation unit (130) may skip the transform on the remaining block.

[0078] A quantized level can be generated by applying quantization to a conversion coefficient or a residual signal. In the following embodiments, the quantized level may also be referred to as a conversion coefficient.

[0079] For example, a 4x4 luminance residual block generated through intra prediction can be transformed using a Discrete Sine Transform (DST)-based basis vector, while the remaining residual blocks can be transformed using a Discrete Cosine Transform (DCT)-based basis vector. Additionally, the transformation blocks for a single block can be divided into a quad tree form using Residual Quad Tree (RQT) technology, and after performing transformation and quantization on each transformation block divided by RQT, a coded block flag (cbf) can be transmitted to increase coding efficiency in the case where all coefficients become zero.

[0080] As another alternative, the Multiple Transform Selection (MTS) technique can be applied to perform transformations using multiple transformation bases selectively. In other words, instead of dividing a CU into TUs via RQT, a function similar to TU division can be performed using the Sub-block Transform (SBT) technique. Specifically, SBT is applied only to inter-frame prediction blocks and, unlike RQT, divides the current block into ½ or ¼ sizes in the vertical or horizontal direction, and then performs a transformation on only one of the blocks. For example, if divided vertically, a transformation can be performed on the leftmost or rightmost block, and if divided horizontally, a transformation can be performed on the topmost or bottommost block.

[0081] In addition, Low Frequency Non-Separable Transform (LFNST), a secondary transform technique that further transforms the residual signal converted to the frequency domain through DCT or DST, can also be applied. LFNST performs additional transformation on the 4x4 or 8x8 low-frequency region in the upper left corner, thereby allowing the residual coefficients to be concentrated in the upper left corner.

[0082] The quantization unit (140) can generate a quantized level by quantizing a transformation coefficient or residual signal according to a quantization parameter (QP, Quantization parameter) and can output the generated quantized level. At this time, the quantization unit (140) can quantize the transformation coefficient using a quantization matrix.

[0083] For example, a quantizer using QP values ​​from 0 to 51 can be used. Alternatively, if the image size is larger and higher coding efficiency is required, QP values ​​from 0 to 63 can be used. Additionally, a Dependent Quantization (DQ) method using two quantizers instead of a single one can be applied. DQ performs quantization using two quantizers (e.g., Q0, Q1), but can be applied so that the quantizer to be used for the next transform coefficient is selected based on the current state through a state transition model, even without signaling information regarding the use of a specific quantizer.

[0084] The entropy encoding unit (150) can generate a bitstream and output a bitstream by performing entropy encoding according to a probability distribution on values ​​calculated by the quantization unit (140) or coding parameter values ​​calculated during the encoding process. The entropy encoding unit (150) can perform entropy encoding on information regarding a sample of an image and information for decoding an image. For example, information for decoding an image may include syntax elements, etc.

[0085] When entropy coding is applied, a small number of bits are allocated to symbols with a high probability of occurrence and a large number of bits are allocated to symbols with a low probability of occurrence, thereby representing the symbols and reducing the size of the bit sequence for the symbols to be encoded. The entropy coding unit (150) may use encoding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) for entropy coding. For example, the entropy coding unit (150) may perform entropy coding using a Variable Length Coding (VLC) table. In addition, the entropy encoding unit (150) may perform arithmetic encoding using the derived binarization method, probability model, and context model after deriving a binarization method of the target symbol and a probability model of the target symbol / bin.

[0086] In this regard, when applying CABAC, in order to reduce the size of the probability table stored in the decoder, the table probability update method may be changed to a table update method using a simple formula. In addition, two different probability models may be used to obtain more accurate symbol probability values.

[0087] The entropy encoding unit (150) can convert a 2-dimensional block form coefficient into a 1-dimensional vector form through a transform coefficient scanning method to encode a transform coefficient level (quantized level).

[0088] Coding parameters may include information (flags, indexes, etc.) that is encoded in the encoding device (100) and signaled to the decoding device (200), such as syntax elements, as well as information derived during the encoding process or decoding process, and may refer to information required when encoding or decoding images.

[0089] Here, signaling a flag or index may mean that in an encoder, the corresponding flag or index is entropy encoded and included in a bitstream, and in a decoder, the corresponding flag or index is entropy decoded from the bitstream.

[0090] The encoded current image can be used as a reference image for other images processed later. Accordingly, the encoding device (100) can restore or decode the encoded current image again, and can store the restored or decoded image as a reference image in the reference picture buffer (190).

[0091] The quantized level can be dequantized in the dequantization unit (160) and inverse transformed in the inverse transform unit (170). The dequantized and / or inverse transformed coefficients can be added to the prediction block through the adder (117). A reconstructed block can be generated by adding the dequantized and / or inverse transformed coefficients and the prediction block. Here, the dequantized and / or inverse transformed coefficients refer to coefficients for which at least one of dequantization and inverse transformation has been performed, and may refer to the reconstructed residual block. The dequantization unit (160) and the inverse transform unit (170) can be performed as the reverse process of the quantization unit (140) and the transformation unit (130).

[0092] The restoration block may pass through a filter section (180). The filter section (180) may apply a deblocking filter, Sample Adaptive Offset (SAO), Adaptive Loop Filter (ALF), Bilateral filter (BIF), LMCS (Luma Mapping with Chroma Scaling), etc., to the restoration sample, restoration block, or restoration image as a whole or part of the filtering technique. The filter section (180) may also be referred to as an in-loop filter. In this case, the term in-loop filter is also used as a name that excludes LMCS.

[0093] Deblocking filters can remove block distortion occurring at the boundaries between blocks. To determine whether to perform deblocking, the decision to apply the filter to the current block can be made based on samples contained in a few columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering intensity.

[0094] To compensate for encoding errors using a sample adaptive offset, an appropriate offset value can be added to the sample value. The sample adaptive offset can correct the offset from the original image on a sample-by-sample basis for the deblocked image. One method may be to divide the samples included in the image into a certain number of regions, determine the region to be offset, and apply the offset to that region, or to apply the offset by considering the edge information of each sample.

[0095] A bilateral filter (BIF) can also correct the offset from the original image on a sample-by-sample basis for the deblocked image.

[0096] An adaptive loop filter can perform filtering based on a comparison of the reconstructed image and the original image. After dividing the samples included in the image into predetermined groups, a filter to be applied to each group can be determined, thereby performing filtering differently for each group. Information regarding whether to apply an adaptive loop filter can be signaled per coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may vary depending on each block.

[0097] In LMCS (Luma Mapping with Chroma Scaling), Luma mapping (LM) refers to remapping luminance values ​​through a piece-wise linear model, and Chroma scaling (CS) refers to a technique that scales the residual values ​​of the chrominance component according to the average luminance value of the predicted signal. In particular, LMCS can be utilized as an HDR correction technique that reflects the characteristics of HDR (High Dynamic Range) video.

[0098] The restored block or restored image that has passed through the filter unit (180) can be stored in the reference picture buffer (190). The restored block that has passed through the filter unit (180) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (180). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0099] FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present disclosure is applied.

[0100] The decoding device (200) may be a decoder, a video decoding device, or an image decoding device.

[0101] Referring to FIG. 2, the decoding device (200) may include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), a motion compensation unit (250), an adder (201), a switch (203), a filter unit (260), and a reference picture buffer (270).

[0102] The decoding device (200) can receive a bitstream output from the encoding device (100). The decoding device (200) can receive a bitstream stored in a computer-readable recording medium or a bitstream stream streamed through a wired / wireless transmission medium. The decoding device (200) can perform decoding on the bitstream in intra mode or inter mode. Additionally, the decoding device (200) can generate a restored image or a decoded image through decoding and can output the restored image or the decoded image.

[0103] If the prediction mode used for decoding is intra mode, the switch (203) can be switched to intra. If the prediction mode used for decoding is inter mode, the switch (203) can be switched to inter.

[0104] The decoding device (200) can decode the input bitstream to obtain a reconstructed residual block and generate a prediction block. Once the reconstructed residual block and the prediction block are obtained, the decoding device (200) can generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may be referred to as the current block.

[0105] The entropy decoding unit (210) can generate symbols by performing entropy decoding according to the probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the inverse process of the entropy encoding method described above.

[0106] The entropy decoding unit (210) can convert a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a conversion coefficient scanning method to decode a conversion coefficient level (quantized level).

[0107] The quantized level can be dequantized in the dequantization unit (220) and inversely transformed in the inverse transformation unit (230). The quantized level can be generated as a restored residual block as a result of performing dequantization and / or inverse transformation. At this time, the dequantization unit (220) can apply a quantization matrix to the quantized level. The dequantization unit (220) and the inverse transformation unit (230) applied to the decoding device can apply the same technology as the dequantization unit (160) and the inverse transformation unit (170) applied to the aforementioned encoding device.

[0108] When an intra mode is used, the intra prediction unit (240) can generate a prediction block by performing a spatial prediction on the current block using sample values ​​of already decoded blocks around the block to be decoded. The intra prediction unit (240) applied to the decoding device can apply the same technology as the intra prediction unit (120) applied to the aforementioned encoding device.

[0109] When an inter mode is used, the motion compensation unit (250) can generate a prediction block by performing motion compensation on the current block using a motion vector and a reference image stored in the reference picture buffer (270). The motion compensation unit (250) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform motion compensation, it can be determined whether the motion compensation method of the prediction unit included in the corresponding encoding unit is a skip mode, merge mode, AMVP mode, or current picture reference mode based on the encoding unit, and motion compensation can be performed according to each mode. The motion compensation unit (250) applied to the decoder can apply the same technology as the motion compensation unit (122) applied to the aforementioned encoding unit.

[0110] The adder (201) can generate a restored block by adding the restored residual block and the prediction block. The filter unit (260) can apply at least one of the following to the restored block or the restored image: an inverse-LMCS, a deblocking filter, a sample adaptive offset, and an adaptive loop filter. The filter unit (260) applied to the decoder can apply the same filtering technology as the filter unit (180) applied to the aforementioned encoding device.

[0111] The filter unit (260) can output a restored image. The restored block or the restored image can be stored in a reference picture buffer (270) and used for inter-frame prediction. The restored block that has passed through the filter unit (260) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (260). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0112] FIG. 3 is a schematic diagram illustrating a video coding system to which the present disclosure can be applied.

[0113] A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and / or image information or data to the decoding device (20) via a digital storage medium or network in the form of a file or streaming.

[0114] An encoding device (10) according to one embodiment may include a video source generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to one embodiment may include a receiving unit (21), a decoding unit (22), and a rendering unit (23). The encoding unit (12) may be called a video / image encoding unit, and the decoding unit (22) may be called a video / image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The receiving unit (21) may be included in the decoding unit (22). The rendering unit (23) may include a display unit, and the display unit may be composed of a separate device or an external component.

[0115] The video source generation unit (11) can acquire video / image through a process of capturing, synthesizing, or generating video / image. The video source generation unit (11) may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / image, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and can generate video / image (electronically). For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0116] The encoding unit (12) can encode the input video / image. The encoding unit (12) can perform a series of procedures such as prediction, conversion, and quantization for compression and encoding efficiency. The encoding unit (12) can output the encoded data (encoded video / image information) in the form of a bitstream. The detailed configuration of the encoding unit (12) can also be configured in the same way as the encoding device (100) of FIG. 1 described above.

[0117] The transmission unit (13) can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit (21) of the decoding device (20) via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit (21) can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

[0118] The decoding unit (22) can decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12). The detailed configuration of the decoding unit (22) can also be configured to be identical to the decoding device (200) of FIG. 2 described above.

[0119] The rendering unit (23) can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0120]

[0121] Hereinafter, with reference to FIGS. 4 to 11, an intra prediction method based on template matching (TM) is described in detail.

[0122] Figure 4 shows an example of template matching.

[0123] FIG. 4 illustrates the concept that a prediction block (440) having a template similar to the template of the current block (430) is selected in the search area (420) within the restoration area (410) by template matching.

[0124] The restoration area (410) contains samples that have already been restored. The current block (430) may be placed at a location adjacent to the restoration area (410), and a current template (431) consisting of upper and left adjacent samples of the current block (430) is set.

[0125] The search area (420) represents the range within the restoration area (410) where candidate blocks are searched for the prediction of the current block (430). Multiple candidate blocks may exist within the search area (420), and a prediction block (440) may be selected from among them.

[0126] A similar template (441) consisting of upper and left adjacent samples of the prediction block (440) may be set in the prediction block (440). A template matching cost is calculated based on the difference between the current template (431) and the similar template (441), and a candidate block with the minimum template matching cost may be selected as the prediction block (440).

[0127] As illustrated in FIG. 4, the encoding device and the decoder search for candidate blocks in a reference search area. For each candidate block, a cost function is applied to calculate the difference between the current CU template and the candidate block template, and the candidate block with the minimum cost is determined as the optimal prediction block. The template matching cost function can primarily be calculated using the sum of absolute differences (SAD). Additionally, the cost function can be calculated using the sum of squared errors (SSE).

[0128] Template matching can be applied to the following functions in addition to generating prediction blocks. Among multiple reference block candidates, the prediction block can be improved by utilizing the comparison results between the current template and candidate templates. Furthermore, in advanced motion vector prediction (AMVP) and merge prediction, candidates can be reordered based on template matching costs to reconstruct the candidate index.

[0129] When encoding the current block, performing template matching on multiple recovery reference blocks can increase the amount of computation. Therefore, the encoding device and the decoding device perform template matching by defining a reference search region that limits the search range of the reference blocks.

[0130] When using templates, the encoder and decoder predict the current block using an in-frame reference block that has a template most similar to the template of the current block. During the process of encoding the current block, the encoder determines the reference block using the similarity between the reference block and the template of the current block. Then, the encoder determines the predicted signal of the current block using the determined reference block. Similarity can be measured based on SAD or SSE.

[0131] When performing template matching on multiple restored reference blocks, the computational complexity of image encoding and decoding increases. Therefore, the encoding device and the decoding device define a reference region that limits the range in which reference blocks can be selected, and perform template matching within the reference region.

[0132] Figure 5 illustrates a template matching-based prediction process for in-screen prediction.

[0133] FIG. 5 illustrates an example of a reference search area (520) set within a restoration area based on the current block (510) in an in-screen prediction using template matching.

[0134] The current block (510) has a block width (W) (511) and a block height (H) (512). The reference search area (520) refers to the range in which candidate blocks are searched around the current block (510), and the reference search area (520) may be defined by being divided into multiple sub-regions. For example, the reference search area (520) may be defined to include R1 (521), R2 (522), R3 (523), R4 (524), R5 (525), and R6 (526).

[0135] In FIG. 5, the size of the reference search area (520) can be determined based on the size of the current block (510) (block width (511), block height (512)). For example, the horizontal search range (530) can be set to 2*max(5*block width, 64), and the vertical search range (531) can be set to max(5*block height, 64).

[0136] The order in which candidate blocks are evaluated within the reference search area (520) can be defined by area. For example, candidate blocks may be searched in the order of R4 (524) → R5 (525) → R6 (526) → R1 (521) → R2 (522) → R3 (523).

[0137] When the encoding device and the decoding device encode the current block (510) using template prediction, they determine whether the size of the current block (510) satisfies the minimum size condition and the maximum size condition for applying the template matching-based prediction mode. And if the minimum size condition and the maximum size condition are satisfied, flag information indicating whether the current block (510) was encoded in the template matching-based prediction mode may be generated.

[0138] The encoding device encodes the current block (510) by performing template matching prediction based on flag information. Then, the decoding device parses flag information from the bitstream and restores the current block by performing template matching prediction based on flag information.

[0139] Candidate reordering based on template matching techniques improves encoding efficiency by adaptively reordering the indices of motion vector candidates.

[0140] Candidate reordering can be applied to i) general MERGE mode, ii) TM MERGE mode, iii) AFFINE MERGE mode, and iv) AMVP mode. After the candidate list is constructed, a template matching cost is calculated for each candidate included in the candidate list. The template matching cost is measured based on the difference between the template sample of the current block and the reference sample corresponding to each candidate. For example, the template matching cost can be calculated based on the sum of absolute differences (SAD). The encoder and decoder reorder the candidate list according to the ascending order of the template matching costs.

[0141] Below, DIMD (Decoder side intra mode derivation) technology is explained.

[0142] In DIMD technology, a template region is constructed using a restored sample adjacent to the current block. The template region can be composed of three lines each from the restored left line and the restored upper line of the current block.

[0143] By applying a Sobel filter to the template area, the horizontal and vertical gradients can be calculated. Based on the calculated gradients, orientation and magnitude information are extracted.

[0144] A histogram of oriented gradient (HoG) is generated using the extracted angle and magnitude information. Five angles with relatively high cumulative magnitude values ​​are selected from the HoG, and five in-frame prediction modes corresponding to each of the five selected angles are determined.

[0145] Prediction blocks can be generated based on each of the five determined in-frame prediction modes. Additionally, (i) a prediction block based on a planar mode or (ii) a prediction block based on a BV (block vector) can be additionally generated. At least some of the prediction blocks can be weighted and combined to form a final prediction block.

[0146] Since the intra-frame prediction mode determined by DIMD is completed before the creation of the MPM (most probable mode) list, the intra-frame prediction mode determined by DIMD can be utilized during the MPM list creation process.

[0147] Below, TIMD (Template intra mode derivation) technology is explained.

[0148] Figure 6 shows the template region and reference region of TIMD.

[0149] FIG. 6 illustrates an example of a template (630) set around the current block (610) in a template matching-based prediction and a template reference area (640) set for reference of the template.

[0150] The current block (610) has a block width (W) (611) and a block height (H) (612). An L-shaped template (630) can be formed by the left and upper adjacent restoration samples of the current block (610). The template (630) can be configured to include, for example, a left template length (L1) (613) extending in the left direction of the current block (610) and an upper template length (L2) (614) extending in the upper direction of the current block (610).

[0151] A template reference area (640) may be set around the current block (610) to obtain a reference sample corresponding to the template (630) or to search for candidate templates for performing template matching. The horizontal range (641) of the template reference area (640) may be set to 2*(block width+L1)+1, and the vertical range (642) of the template reference area (640) may be set to 2*(block height+L2)+1.

[0152] In FIG. 6, the template (630) is shown as a hatched area, and the template reference area (640) is shown as an highlighted area in the shape of an outer border.

[0153] In TIMD technology, for templates (630) around the current block (610), prediction is performed by applying an in-frame prediction mode included in the MPM list. The sum of absolute transformed differences (SATD) cost between the predicted value for the template (630) and the restored template value is calculated.

[0154] Based on the calculated SATD cost, the mode with the minimum SATD cost is selected as costMode1, and the mode with the second lowest SATD cost is selected as costMode2. Additionally, among DC mode or planar mode, the non-directional mode with the smaller SATD cost may be selected as costMode3.

[0155] The inclusion of the non-directed mode (costMode3) may be limited to cases where all of the following conditions are satisfied.

[0156] - When the third undirected mode (costMode3) is a different mode from costMode1 and costMode2

[0157] - If costMode3 < 1.5 × costMode1

[0158] If both of the above two conditions are true, three in-screen prediction modes (costMode1, costMode2, costMode3) can be combined using the weights defined in Equation 1.

[0159] [Mathematical Formula 1]

[0160] If at least one of the two conditions above is false, two in-frame prediction modes (costMode1, costMode2) are used. Whether the two modes are combined by weight is determined according to Equation 2.

[0161] [Mathematical Formula 2]

[0162] costMode2 < 2 * costMode1

[0163] If Equation 2 is satisfied, two modes are used in a weighted combination. If Equation 2 is not satisfied, only one of the two modes with the lower SATD cost can be used.

[0164] When two modes are combined by weight, the weights can be set so that a higher weight is assigned to the mode with the lower SATD cost. For example, the weights can be defined by Equation 3.

[0165] [Mathematical Formula 3]

[0166] weight1 = costMode2 / (costMode1+ costMode2)

[0167] weight2 = 1 - weight1

[0168] The size of the template (630) is determined based on the size of the current block (610). If the block width (W) (611) or block height (H) (612) of the current block (610) is greater than 8, the template size L1 or L2 may be set to 4 as shown in FIG. 6. If the block width (W) (611) and block height (H) (612) of the current block (610) are 8 or less, the template size L1 or L2 may be set to 2.

[0169] The Extended Multiple Reference Line (MRL) technology is described below.

[0170] Figure 7 illustrates multiple restored sample lines referenced in the in-frame prediction.

[0171] According to FIG. 7, restoration sample lines are shown as a plurality of reference lines (720 to 727) based on the current block (710). On the upper and left sides of the current block (710), a 0th reference line (720) adjacent to the current block (710) may be set, and a 1st reference line (721), a 2nd reference line (722), a 3rd reference line (723), a 4th reference line (724), a 5th reference line (725), a 6th reference line (726), and a 7th reference line (727) may be sequentially set at a position further spaced from the 0th reference line (720).

[0172] Each reference line (720 to 727) may provide a different set of reference samples depending on the degree of separation from the upper or left direction of the current block (710) and may be used as a reference for in-frame prediction. In FIG. 7, the dot () indicates that additional reference lines (728) that are further separated than the 7th reference line (727) may be included.

[0173] According to one embodiment, a 0th reference line (720), a 1st reference line (721), and a 3rd reference line (723) may be used as reference lines for in-frame prediction. According to an embodiment, the number of reference lines for in-frame prediction may be expanded. For example, a 0th reference line (720), a 1st reference line (721), a 3rd reference line (723), a 5th reference line (725), a 7th reference line (727), and a 12th line (not shown) may be used as reference lines for in-frame prediction.

[0174] The following describes the EIP (extrapolated intra prediction) technology.

[0175] FIG. 8 illustrates an example of an input / output structure of a filter used in an EIP.

[0176] Figure 8 illustrates different types of EIP filter types (810, 820, 830). In each EIP filter type (810, 820, 830), samples shown in gray can be used as EIP input samples (840), and samples shown in white can be generated as EIP output samples (850).

[0177] In the first EIP filter type (810), a configuration is shown in which a block-shaped output sample (850) is generated based on a plurality of input samples (840) arranged in a block shape.

[0178] In the second EIP filter type (820), a configuration is shown in which an output sample (850) is generated in a horizontal direction based on an input sample (840) arranged in a horizontal direction.

[0179] In the third EIP filter type (830), a configuration is shown in which an output sample (850) is generated in a vertical direction based on an input sample (840) arranged in a vertical direction.

[0180] In FIG. 8, the arrangement of input samples (840) and output samples (850) can be used as an example of a filter structure (tap structure) applied in the process of deriving EIP coefficients and / or generating a prediction signal.

[0181] FIG. 9 illustrates examples of reference regions for deriving EIP coefficients around the current block (910).

[0182] FIG. 9 may include a first embodiment (900a), a second embodiment (900b), and a third embodiment (900c). In each embodiment, a reference area (920) may be set on the left and upper sides of the current block (910). The area of ​​the reference area (920) may be defined by a left reference length (921) (leftSize) and an upper reference length (922) (aboveSize), along with the size of the current block (910). Additionally, a filter block (930) for selecting a reference sample may be placed within the reference area (920), and the size of the filter block (930) may be defined by a filter width (931) (fWidth) and a filter height (932) (fHeight).

[0183] In the first embodiment (900a), a configuration is shown in which a reference area (920) is spaced apart from the left and upper sides of the current block (910), respectively, and a filter block (930) is placed in the reference area (920).

[0184] In the second embodiment (900b), a reference area (920) is configured to include reference samples placed near the upper side of the current block (910), and a configuration is shown in which a filter block (930) is placed in the reference area (920).

[0185] In the third embodiment (900c), a reference area (920) is configured to include reference samples placed near the left side of the current block (910), and a configuration is shown in which a filter block (930) is placed in the reference area (920).

[0186] As illustrated in FIG. 9, the arrangement shape and length (921, 922) of the reference area (920) may be set differently depending on the location of the current block (910), the range of available restoration samples, or the applied prediction mode.

[0187] In the EIP technique, the three types of filters shown in Fig. 8 and the reference sample area around the current block shown in Fig. 9 are used to derive EIP coefficients, and the current block is predicted based on the derived EIP coefficients.

[0188] EIP can consist of three stages.

[0189] First, the EIP coefficients are derived using the reference sample region around the current block and the EIP filter type. Then, a prediction signal is recursively generated within the current block. Subsequently, the gradient of the predicted block is analyzed to derive the mode of the predicted block.

[0190] The derived mode is utilized and can be used as reference information when a transform kernel is selected, for example, MTS (multiple transform selection) or LFNST (low-frequency non-separable transform).

[0191] The Spatial Geometric partitioning mode (SGPM) technique is described below.

[0192] SGPM is a technology that utilizes a geometric partitioning concept similar to GPM (geometric partitioning mode) but operates as an intra-frame prediction mode. That is, in SGPM, the current block is divided into two regions, and an intra-frame prediction mode is applied to each of the divided regions to perform prediction.

[0193] Figure 10 illustrates an embodiment of an SGPM candidate.

[0194] FIG. 10 illustrates an example of a candidate in which a current block (1010) is divided into two regions by a partitioning mode (1020) in a spatial geometric partitioning mode (SGPM), and a different intra-frame prediction mode is applied to each divided region, and an example of a signaling method for said candidate.

[0195] At the top of FIG. 10, a configuration is shown in which the current block (1010) is divided into a first region (1021) and a second region (1022) by a division mode (1020). A first in-frame prediction mode (1030) may be applied to the first region (1021), and a second in-frame prediction mode (1031) may be applied to the second region (1022). The division mode (1020) may be defined, for example, by information indicating the direction and / or location of the division boundary.

[0196] For SGPM, a candidate list can be constructed that includes i) a partition mode representing a region partition and ii) two intra-frame prediction modes. To form candidate combinations as illustrated in FIG. 10, index information (partition_mode_idx) representing one of a total of 26 region partition modes and two intra-frame prediction mode indices (intra_pred_mode0_idx, intra_pred_mode1_idx) may be used. Here, the 26 region partition modes may consist of multiple partition types that define the shape of the geometric partition boundary (e.g., the direction and / or location of the boundary) that divides the current block into two regions differently, for example. The length of the candidate list is set to 16, and the index of the selected candidate can be signaled.

[0197] In the bottom left of Fig. 10, a configuration is shown in which whether SGPM is applied is indicated by cu_sgpm_flag, and when cu_sgpm_flag is true, partition_mode_index, intra_pred_mode0_idx, and intra_pred_mode1_idx are each individually signaled. That is, the partition mode and two intra-frame prediction modes are determined by three syntax elements.

[0198] In the bottom right of Fig. 10, a configuration is illustrated in which, when cu_sgpm_flag is true, sgpm_candidate_index is signaled, and sgpm_candidate_index is mapped to a candidate combination stored in the candidate list to determine partition_mode_index, intra_pred_mode0_idx, and intra_pred_mode1_idx. That is, it is determined by a single syntax element representing a combination of partition mode and two intra-frame prediction modes in the candidate list.

[0199] FIG. 11 illustrates an example of an SGPM template.

[0200] In FIG. 11, a current block (1110) is shown, and a template (1120) can be established from restoration samples adjacent to the upper and left sides of the current block (1110). The template (1120) may be configured to include, for example, an upper template (1121) extending in the upper direction of the current block (1110) and a left template (1122) extending in the left direction of the current block (1110).

[0201] The size of the upper template (1121) can be defined as the upper template size (1131), and the size of the left template (1122) can be defined as the left template size (1132). In FIG. 11, the upper template size (1131) and the left template size (1132) may be set to be the same or different from each other.

[0202] In the rearrangement of SGPM candidates, the cost between the candidate template corresponding to the template (1120) and the template (1120) of the current block (1110) is calculated, and the candidate order can be reconfigured based on the calculated cost. Here, the cost can be calculated, for example, as SAD (sum of absolute differences) or SATD (sum of absolute transformed differences).

[0203] The candidate order of the above candidate list for SGPM is reordered in order of lowest SAD cost calculated in the template area, as shown in Fig. 11.

[0204] Below, an improved TM cost function is proposed that can be used in the process of improving prediction blocks, reordering candidate indices, or deriving parameters using template matching technology.

[0205] According to one embodiment, in the TM technique, only distortions such as SAD between the prediction block and the template of the current block are used as costs, so it was difficult to derive optimal parameters or optimal prediction blocks. In the present invention, a method is proposed in which terms regarding bit rate or complexity are explicitly or implicitly included in the TM cost function in addition to the distortion function (e.g., SAD).

[0206] Below, embodiments of a template cost function for improving encoding efficiency are described.

[0207] Mathematical Equation 4 represents an example of an improved TM cost function (TMCost).

[0208] [Mathematical Formula 4]

[0209] TMCost = TMDistortion + λ × Rate

[0210] In mathematical equation 4, TMDistortion can be, for example, SAD. Rate represents the bit size required in the process of encoding the current block. λ is a real number greater than 0 and can be, for example, a Lagrangian constant (Lagrangian multiplier).

[0211] The Rate term in Equation 4 may be defined in such a way that the bit size required to encode or decode the corresponding block is calculated or estimated. For example, the Rate term may be defined by at least one of the following methods.

[0212] A. The bit size calculated by CABAC (context-adaptive binary arithmetic coding) can be used.

[0213] B. Values ​​obtained by a lookup table may be used. For example, a table may be used for content-adaptive variable length coding (VLC) of a symbol, and the binarization result of the symbol may be referenced to estimate the rate.

[0214] C. The rate can be estimated based on the size of the symbol to be encoded. For example, the rate can be estimated based on the magnitude of the motion vector difference (MVD) obtained by subtracting the motion vector prediction from the motion vector. Additionally, the absolute value of the MVD can be used, or the rate can be estimated by counting the number of bins after the MVD is binarized.

[0215] D. Symbols to be encoded are converted into indices, and a Rate can be estimated based on the index values. For example, if the magnitude and direction of the MVD are composed of a predefined discrete set, as in MMVD (merge with motion vector difference), the Rate can be estimated based on the magnitude of the index.

[0216] E. The bit rate may be implicitly reflected in the process of calculating TMDistortion. For example, when SATD is used instead of SAD, the bit rate may be implicitly included as the transformation coefficients of the residual signal after prediction are indirectly estimated. In this case, TMCost may be substantially the same as the TMDistortion calculated by SATD.

[0217] According to Equation 4, the candidate with the minimum TMCost among the candidates can be selected. In addition, whether template matching is applied can be controlled according to the following conditions.

[0218] According to one embodiment, if the minimum value of TMCost calculated for the candidates exceeds a predefined threshold, template matching may not be applied. For example, when TM is used for candidate reordering, if the minimum value of TMCost calculated for the candidates exceeds the threshold, the reordering process may not be performed. As another example, when TM is used for prediction block refinement, if the TMCost calculated for refinement exceeds the threshold, the refinement process may not be performed.

[0219] Template matching is applied only when the output of the Rate function is not zero, and may not be applied when the output of the Rate function is zero. For example, if MV is a zero-motion vector and the Rate function is defined based on the magnitude of the motion vector, the output of the Rate function may be zero. In this case, template matching may not be applied.

[0220] Examples of template cost functions for improving complexity are described below.

[0221] In addition to the Rate function, the TM Cost function (TMCost) may include a Complexity function representing complexity, as shown in Equation 5. When a Complexity function is included, the amount of computation required for template matching can be reduced.

[0222] [Mathematical Formula 5]

[0223] TMCost = TMDistortion + λ × Complexity

[0224] In mathematical equation 5, TMDistortion can be, for example, SAD. Complexity represents the amount of computation required in the process of encoding or decoding the current block. λ is a real number greater than 0 and can be, for example, a Lagrangian constant.

[0225] The Complexity term of mathematical formula 5 can be defined by at least one of the following methods.

[0226] A. Complexity can be defined based on the number of addition operations and / or multiplication operations required to calculate the template cost.

[0227] B. Complexity can be defined based on the size of memory access required to calculate the template cost. Here, the size of memory access may include the size of memory required to store additional information for performing template matching.

[0228] C. Complexity can be defined based on an area index that indicates the area required to calculate the template cost. For example, an area index representing the areas R1 to R6 of FIG. 7 may be used. The area index may be used directly as a complexity value, or the area index may be converted into a complexity value through a predetermined table.

[0229] D. Complexity can be defined based on the distance between the template of the reference block and the template of the current block. The distance may be used directly as the complexity value, or the distance may be converted into a complexity value by a predetermined function. Here, the predetermined function may be defined such that the complexity value increases as the distance increases.

[0230] E. At least one of the values ​​defined in the Rate function can be used in the definition of Complexity.

[0231] According to Equation 5, a template corresponding to the candidate with the minimum TMCost among the candidates can be selected. Additionally, whether to perform template search can be controlled according to the following conditions.

[0232] According to one embodiment, if the TMCost calculated for the candidates exceeds a predefined threshold, the template search may be stopped. For example, during the process of evaluating candidates according to the search order, if the minimum TMCost calculated so far exceeds the threshold, the template search may be stopped.

[0233] According to one embodiment, template search is performed only when the output of the Complexity function is not 0, and template search may be stopped when the output of the Complexity function is 0.

[0234] All of the embodiments of the above template cost function can be applied to encoding tools using the templates introduced in this disclosure.

[0235] Examples of rearranging the candidate list of AMVP are described below.

[0236] The MVP candidate list for AMVP (advanced motion vector prediction) can consist of up to 10 candidates with duplicate candidates removed. For example, the MVP candidate list can be composed in the order of i) spatially neighboring candidates, ii) temporally collocated candidates, iii) spatially non-neighboring candidates, and iv) history-based motion vector prediction (HMVP) candidates.

[0237] According to one embodiment, the motion vectors of blocks spatially adjacent to the current block can be added to the MVP candidate list as MVP candidates.

[0238] According to one embodiment, the motion vector of a collocated block located at the same position as the current block within a reference picture may be added to the MVP candidate list as an MVP candidate. For example, temporal MVP candidates may be added in the order of the bottom-right (BR) position and the center (C) position of the collocated block.

[0239] According to one embodiment, up to one temporal candidate may be added to each of reference picture list 0 (RPL0) and reference picture list 1 (RPL1).

[0240] According to one embodiment, in RPL0, in addition to candidates corresponding to temporally adjacent blocks, locations of temporally non-adjacent neighboring blocks may be determined, and among the determined locations, one candidate with the lowest template matching cost (TM cost) may be added to the MVP candidate list.

[0241] According to one embodiment, in RPL1, a candidate corresponding to the BR position may be added first, and a candidate corresponding to the C position may be added later. Or the opposite.

[0242] According to one embodiment, a motion vector of a block spatially non-adjacent to the current block can be added to the MVP candidate list as an MVP candidate.

[0243] According to one embodiment, HMVP candidates may be added to the MVP candidate list. For example, up to four HMVP candidates may be added.

[0244] The MVP candidate list configured in the above manner can be reordered according to the ascending order of template matching costs, after calculating the template matching cost for each candidate using template matching. After reordering, the candidate located at index 0 can be selected as the final MVP candidate. The index (mvp_idx) of the MVP candidate used in AMVP can be determined based on template matching.

[0245] In AMVP, signaling regarding MVD (motion vector difference) and motion vector-related information may be required. However, if only SAD-based template matching costs are used, it is difficult to reflect the bit amount required for the signaling in the cost. Therefore, the bit size regarding MVD, reference picture list (RPL0 / RPL1), reference picture index, etc. is reflected in the template matching cost function, and the MVP candidate list can be reordered based on the reflected template matching cost.

[0246] For example, zero motion vectors often have a relatively small bit size but large distortion, so it may be difficult to select zero motion vectors based solely on distortion-based template matching costs. In this embodiment, to enable zero motion vectors to be selected as final MVP candidates, a bit size term may be reflected in the template matching cost during the search process for reordering.

[0247] The selection of the final MVP candidates is not limited to selecting a single candidate located at index 0. For example, two candidates located at index 0 and index 1 may be selected as the final two MVP candidates, and mvp_idx may be transmitted. Alternatively, the final selection method may be determined based on additional template matching.

[0248] In addition, the template matching cost function used in each stage of candidate list construction / reordering / selection may be applied differently depending on the stage. For example, a SAD-based template matching cost function may be applied in the first stage, while a template matching cost function with an added Rate term may be applied in the second stage.

[0249] An example of GPM (geometric partitioning mode) reordering is described below.

[0250] The existing GPM can be extended to enable motion vector refinement for uni-directional motion vectors (uni-directional MVs). When the above extension mode is used, the transmission of the motion vector difference (MVD) can be determined for each partition region of the GPM block.

[0251] In the above process, template matching based reordering for GPM split modes may be applied. In template matching based reordering, the template matching cost (TM cost) for each GPM split mode is calculated based on information related to the motion vector of the current GPM block. Subsequently, the GPM split modes are reordered according to the ascending order of their template matching costs.

[0252] The cost of template matching can be calculated, for example, according to the cost function of Equation 4.

[0253] · [Equation 4 (Example)]

[0254] TMCost = TMDistortion + λ × Rate

[0255] Here, TMDistortion may represent distortion between the template of the current GPM block and the candidate prediction (or candidate split) template determined by each GPM split mode, and Rate may be defined including whether MVD is transmitted per split region and / or the bit size required for split mode signaling.

[0256] An example of GPM (geometric partitioning mode) reordering is described below.

[0257] According to one embodiment, the GPM can be extended to enable motion vector refinement for unidirectional motion vectors (unidirectional MVs). When the extension mode is used, the transmission of an MVD can be determined for each divided area of ​​the GPM block.

[0258] In the above process, template matching based reordering for GPM split mode may be applied. In template matching based reordering, a template matching cost (TM cost) for each GPM split mode is calculated based on information related to the motion vector of the current GPM block. Subsequently, the GPM split modes are reordered according to the ascending order of the template matching costs. For example, the template matching cost can be calculated according to the cost function of Equation 4 introduced above.

[0259] Below, an example of a method for improving the complexity of IntraTMP (intra template matching prediction) is described.

[0260] To reduce the complexity of the template matching process, region indexes or region characteristic values ​​for the reference search regions R1 to R6 in FIG. 7 may be included in the template matching cost function. For example, the template matching cost (TMCost) may be defined to include a complexity term along with a distortion term, and the complexity term may be calculated based on indices or measurements for R1 to R6. The value output by the complexity function may be defined, for example, according to at least one of the following methods.

[0261] According to one embodiment, the complexity value may be output such that it decreases as the distance between the current block and each region R1 to R6 becomes closer. The complexity value based on distance may be defined by a predetermined function and may be set to increase as the distance increases. According to the above definition, the template region corresponding to the region closer to the current block may be selected relatively more frequently. Accordingly, the size of the peak memory required during the template search process may be reduced.

[0262] According to one embodiment, the size of each region R1 to R6 is measured, and the complexity value may be output such that it decreases as the region size decreases. The complexity value according to the region size may be defined by a predetermined function, and may be set so that the complexity value increases as the region size increases. According to the above definition, regions requiring relatively less computation during the template search process may be selected more frequently. Accordingly, the amount of computation during the template search process may be reduced.

[0263] The method for rearranging the candidate list of an IBC (intra block copy) is described below.

[0264] Template-based reordering technology can be applied during the process of constructing the candidate list used in IBC. In template-based reordering, the cost for each candidate is calculated, and the candidates can be reordered in ascending order of cost.

[0265] According to one embodiment, the cost can be defined such that the closer the distance between the current block and the IBC prediction block, the smaller the cost value becomes. For example, the cost value based on distance can be defined by a predetermined function and can be set so that the cost value increases as the distance increases. According to the above definition, a candidate corresponding to a template region close to the current block can be selected relatively more frequently. Accordingly, the size of the peak memory required during the template search process can be reduced. Additionally, a template region that makes the size of the IBC block vector relatively small can be selected more frequently.

[0266] According to one embodiment, in the process of constructing a candidate list used in IBC, the cost can be defined such that the smaller the block vector difference (BVD), the smaller the cost value. Here, BVD can be defined, for example, as the difference between the IBC block vector and the block vector prediction. According to the above definition, the amount of bits required for signaling the BVD required for IBC prediction can be reduced.

[0267] According to one embodiment, an IBC candidate may include not only an IBC block vector representing the displacement between the current block and the predicted IBC block, but also the type of the IBC block vector. Accordingly, the cost for each IBC candidate may be adjusted or calculated based on the type of the IBC block vector. Thus, by considering the type of the IBC block vector in the cost for each IBC candidate, an IBC candidate with a high probability of selection may be guided to have a higher priority in the IBC candidate list.

[0268] The type of the above IBC block vector may include a flip type indicating whether to determine the prediction block by applying a vertical flip or a horizontal flip to the reference block.

[0269] In calculating the cost for each IBC candidate as described above, not only the size of the block vector but also the type of the block vector may be considered. Therefore, by considering the size and type of the IBC block vector in the cost for each IBC candidate, IBC candidates with a high probability of selection can be guided to have a higher priority in the IBC candidate list.

[0270]

[0271] FIG. 12 is a flowchart illustrating an image decoding method according to one embodiment of the present disclosure. The image decoding method of FIG. 12 can be performed by an image decoding device.

[0272] In step 1202, a candidate list containing multiple candidates for the current block may be constructed.

[0273] According to one embodiment, the candidate list may be an AMVP (advanced motion vector prediction) candidate list, a merge candidate list, a GPM (geometric partitioning mode) candidate list, an SGPM (spatial geometric partitioning mode) candidate list, an IBC (intra block copy) candidate list, or an intra template matching prediction candidate list.

[0274] In step 1204, the template matching cost for each candidate can be calculated based on the difference between the template of the current block and the template corresponding to the candidate block specified by each candidate in the candidate list.

[0275] According to one embodiment, the template of the current block may be composed of a restoration sample adjacent to the upper or left side of the current block.

[0276] According to one embodiment, the cost function for determining the template matching cost includes a term regarding distortion, and the cost function may be defined to further include at least one of a term regarding bit rate or a term regarding complexity.

[0277] According to one embodiment, the term regarding the bit rate may be defined by at least one of 1) a bit size calculated by CABAC, 2) a lookup table defining the bit rate and template matching cost, and 3) a value of a differential motion vector.

[0278] According to one embodiment, the term regarding complexity may be defined by at least one of 1) the number of operations required to calculate the template matching cost, 2) the memory size required for additional information stored for template matching, 3) a lookup table defining the complexity and the template matching cost, and 4) the distance between the current block and the candidate template.

[0279] In step 1206, the order of the candidate list is reordered based on the template matching cost, and one or more candidates may be selected from the reordered candidate list.

[0280] According to one embodiment, template search may be stopped based on the template matching cost. The stopping of the template search may be determined when the minimum value of the template matching cost calculated for the candidates in the candidate list exceeds a predefined threshold.

[0281] According to one embodiment, in AMVP mode, the AMVP candidate list is composed of up to 10 candidates by sequentially adding spatial MVP candidates, temporal MVP candidates, non-adjacent MVP candidates, and HMVP candidates without duplication, and the order of AMVP candidates can be reordered based on template matching costs.

[0282] According to one embodiment, in merge mode, the merge candidate list is one of a general merge candidate list, a template matching-based merge candidate list, and an affine merge candidate list, and for each merge candidate in the merge candidate list, a template matching cost between the template of the current block and the template of the corresponding merge candidate is calculated, and the order of the merge candidates may be reordered according to the ascending order of the template matching costs.

[0283] According to one embodiment, in GPM mode, the GPM candidate list includes a partition mode candidate for a method of partitioning the current block into two partition regions, and based on information related to the motion vector of the current block, the template matching cost of each partition mode candidate in the GPM candidate list is calculated, and the order of the partition mode candidates can be reordered according to the ascending order of the template matching costs.

[0284] According to one embodiment, in SGPM mode, the SGPM candidate list includes SGPM candidates composed of a combination of a split mode index, a first intra-frame prediction mode index, and a second intra-frame prediction mode index, and the template matching cost of each SGPM candidate in the SGPM candidate list is calculated, and the candidate order of the SGPM candidates can be reordered based on the calculated template matching cost.

[0285] According to one embodiment, in IBC mode, the template matching cost of each IBC candidate in the IBC candidate list is calculated, and the cost function for calculating the template matching cost of the IBC candidate is defined such that the closer the distance between the current block and the IBC prediction block, the smaller the template matching cost becomes, and the candidate order of the IBC candidate can be reordered based on the template matching cost calculated by the cost function.

[0286] According to one embodiment, a template matching cost for each IBC candidate in the IBC candidate list is calculated, the template matching cost is calculated or adjusted according to the block vector type of the IBC candidate, and the candidate order of the IBC candidates can be reordered based on the template matching cost calculated for each IBC candidate.

[0287] According to one embodiment, in an in-screen template matching prediction mode, a template matching cost is calculated for each in-screen template matching prediction candidate in an in-screen template matching prediction candidate list, a reference search area for in-screen template matching is divided into a plurality of sub-areas, and the template matching cost is determined based on at least one of the area index of the sub-areas, the size of the sub-areas, and the distance between the current block and the sub-areas, and the candidate order of the in-screen template matching prediction candidates can be reordered based on the template matching cost.

[0288] In step 1208, the current block can be predicted based on one or more selected candidates.

[0289] Meanwhile, the prediction method applied in the image decoding method of FIG. 12 can be performed in the same way in the image encoding method. A bitstream can be generated by encoding the current block according to the image encoding method using the prediction method using the template of the present disclosure. The bitstream can be stored in a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0290] FIG. 13 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0291] As illustrated in FIG. 13, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0292] The encoding server described above compresses content input from multimedia input devices, such as smartphones, cameras, and CCTVs, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and CCTVs, generate the bitstream directly, the encoding server may be omitted.

[0293] The bitstream above may be generated by a video encoding method and / or video encoding device to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0294] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0295] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0296] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0297] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0298]

[0299] The above embodiments may be performed in the same or a corresponding way in the encoding device and the decoding device. Additionally, an image may be encoded / decoded using at least one of the above embodiments or a combination of at least one.

[0300] The order in which the above embodiments are applied may differ between the encoding device and the decoder. Alternatively, the order in which the above embodiments are applied may be the same between the encoding device and the decoder.

[0301] The above embodiments may be performed for each of the luminance and chrominance signals. Alternatively, the above embodiments for the luminance and chrominance signals may be performed in the same way.

[0302] In the above embodiments, methods are described based on flowcharts as a series of steps or units; however, the present disclosure is not limited to the order of steps, and some steps may occur in a different order or simultaneously with other steps as described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, other steps may be included, or one or more steps of the flowcharts may be omitted without affecting the scope of the present disclosure.

[0303] The above embodiments may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present disclosure, or they may be those known and available to those skilled in the art of computer software.

[0304] The bitstream generated by the encoding method according to the above embodiment may be stored in a non-transient computer-readable recording medium. Additionally, the bitstream stored in the non-transient computer-readable recording medium may be decoded by the decoding method according to the above embodiment.

[0305] Herein, examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

[0306] Although the present disclosure has been described above with specific details such as specific components, limited embodiments, and drawings, this is provided only to aid in a more comprehensive understanding of the present disclosure and is not limited to the above embodiments, and a person skilled in the art to which the present disclosure belongs can make various modifications and variations from this description.

[0307] Accordingly, the scope of the present disclosure is not limited to the embodiments described above, and all things equivalent or equivalently modified to the claims set forth below, as well as the claims set forth below, shall be considered to be within the scope of the scope of the present disclosure.

[0308] The present disclosure may be used in an apparatus for encoding / decoding images and a recording medium storing a bitstream.

Claims

1. A step of constructing a candidate list including multiple candidates for the current block; A step of calculating a template matching cost for each candidate based on the difference between the template of the current block and the template corresponding to the candidate block designated by each candidate in the candidate list; A step of reordering the order of the candidate list based on the above template matching cost, and selecting one or more candidates from the reordered candidate list; and An image decoding method comprising the step of predicting the current block based on one or more selected candidates.

2. In Paragraph 1, An image decoding method in which the template of the current block above consists of a restoration sample adjacent to the upper or left side of the current block.

3. In Paragraph 1, A video decoding method in which template search is stopped based on the above-mentioned template matching cost.

4. In Paragraph 3, An image decoding method in which the cessation of the above-mentioned template search is determined by the case where the minimum value of the template matching cost calculated for the candidates of the above-mentioned candidate list exceeds a predefined threshold value.

5. In Paragraph 1, A method for decoding images, wherein the cost function for determining the above template matching cost includes a term regarding distortion, and the cost function is further defined to include at least one of a term regarding a bit rate or a term regarding complexity.

6. In Paragraph 5, An image decoding method characterized in that the term regarding the bit rate is defined by at least one of 1) a bit size calculated by CABAC (context-adaptive binary arithmetic coding), 2) a lookup table defining the bit rate and template matching cost, and 3) the value of a differential motion vector.

7. In Paragraph 5, An image decoding method characterized in that the above-mentioned term regarding complexity is defined by at least one of 1) the number of operations required to calculate the template matching cost, 2) the memory size required for additional information stored for template matching, 3) a lookup table defining the complexity and the template matching cost, and 4) the distance between the current block and the candidate template.

8. In Paragraph 1, An image decoding method characterized in that the above candidate list is an AMVP (advanced motion vector prediction) candidate list, a merge candidate list, a GPM (geometric partitioning mode) candidate list, an SGPM (spatial geometric partitioning mode) candidate list, an IBC (intra block copy) candidate list, or an intra template matching prediction candidate list.

9. In Paragraph 8, The above AMVP candidate list consists of up to 10 candidates, with spatial MVP (motion vector prediction) candidates, temporal MVP candidates, non-adjacent MVP candidates, and HMVP (historical motion vector prediction) candidates added sequentially without duplication. A video decoding method in which the order of the AMVP candidates is rearranged based on the above template matching cost.

10. In Paragraph 8, The above merge candidate list is one of a general merge candidate list, a template matching-based merge candidate list, and an affine merge candidate list, and For each merge candidate in the above merge candidate list, a template matching cost is calculated between the template of the current block and the template of the corresponding merge candidate, and A video decoding method in which the order of the merge candidates is rearranged according to the ascending order of the template matching costs.

11. In Paragraph 8, The above GPM candidate list includes partition mode candidates for a method of dividing the above current block into two partition regions, and Based on information related to the motion vector of the current block, the template matching cost for each partition mode candidate of the GPM candidate list is calculated, and An image decoding method in which the order of the split mode candidates is rearranged according to the ascending order of the above template matching costs.

12. In Paragraph 8, The above SGPM candidate list includes SGPM candidates composed of a combination of a partitioning mode index, a first intra-frame prediction mode index, and a second intra-frame prediction mode index, and The template matching cost for each SGPM candidate in the above SGPM candidate list is calculated, and An image decoding method in which the candidate order of the above SGPM candidates is rearranged based on the calculated template matching cost.

13. In Paragraph 8, The template matching cost for each IBC candidate in the above IBC candidate list is calculated, and The cost function for calculating the template matching cost of the above IBC candidate is defined such that the template matching cost becomes smaller as the distance between the current block and the IBC prediction block decreases, and An image decoding method in which the candidate order of the above IBC candidates is rearranged based on the template matching cost calculated by the above cost function.

14. In Paragraph 8, The template matching cost for each IBC candidate in the above IBC candidate list is calculated, and The above template matching cost is calculated or adjusted according to the block vector type of the above IBC candidate, and A video decoding method in which the candidate order of the above IBC candidates is rearranged based on the template matching cost calculated for each IBC candidate.

15. In Paragraph 8, The template matching cost for each of the in-screen template matching prediction candidates in the above-mentioned in-screen template matching prediction candidate list is calculated, and The reference search area for on-screen template matching is divided into multiple sub-areas, and The above template matching cost is determined based on at least one of the area index of the said partial area, the size of the said partial area, and the distance between the current block and the said partial area, and A video decoding method in which the candidate order of the template matching prediction candidates within the above screen is rearranged based on the above template matching cost.

16. A step of constructing a candidate list including multiple candidates for the current block; A step of calculating a template matching cost for each candidate based on the difference between the template of the current block and the template corresponding to the candidate block designated by each candidate in the candidate list; A step of rearranging the order of the candidate list based on the above template matching cost, and selecting one or more candidates from the rearranged candidate list; and A video encoding method comprising the step of predicting the current block based on one or more selected candidates.

17. A bitstream transmission method generated by a video encoding method, The above transmission method includes the step of generating a bitstream by encoding an image according to an image encoding method and the step of transmitting the generated bitstream. The above image encoding method is, Template matching using a template is applied within a restoration area containing restored samples for prediction of the current block, and A step of constructing a candidate list including a plurality of candidates for the current block; A step of calculating a template matching cost for each candidate based on the difference between the template of the current block and the template corresponding to the candidate block designated by each candidate in the candidate list; A step of rearranging the order of the candidate list based on the above template matching cost, and selecting one or more candidates from the rearranged candidate list; and A transmission method comprising the step of predicting the current block based on one or more selected candidates.

Images