Intra predictive video encoding / decoding method, bitstream transmission method, and recording medium storing bitstream.

Abstract

This disclosure proposes an intra-prediction-based video coding / decoding method, apparatus, and recording medium for storing bitstreams. The video decoding method may include steps of generating a first prediction block based on intra-template matching prediction, generating a second prediction block based on a specific intra-prediction mode, and generating a final prediction block based on the first and second prediction blocks.

Description

[Technical Field]

[0001] This disclosure relates to an intra-predictive video encoding / decoding method, a method for transmitting a bitstream, and a recording medium storing a bitstream, and more specifically, to a video encoding / decoding method, a method for transmitting a bitstream, and a recording medium storing a bitstream that utilizes multiple intra-predictive blocks including an intraTMP prediction block. [Background technology]

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, has been increasing in various fields. As image data becomes higher resolution and higher quality, the amount of information or bits transmitted increases relatively compared to conventional image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs.

[0003] This necessitates highly efficient image compression technology to effectively transmit, store, and reproduce high-resolution, high-quality image information. [Overview of the project] [Problems that the invention aims to solve]

[0004] This disclosure aims to provide a video encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] Furthermore, this disclosure aims to provide a video encoding / decoding method and apparatus that utilizes multiple in-screen prediction blocks to improve coding quality.

[0006] Furthermore, this disclosure aims to provide a video encoding / decoding method and apparatus for generating a final prediction block using multiple intra-prediction blocks.

[0007] Furthermore, an object of the present disclosure is to provide a non - transient computer - readable recording medium for storing a bitstream generated by a video encoding method or apparatus according to the present disclosure.

[0008] Furthermore, an object of the present disclosure is to provide a non - transient computer - readable recording medium for storing a bitstream received by a video decoding apparatus according to the present disclosure, decoded, and used for video restoration.

[0009] Furthermore, an object of the present disclosure is to provide a method for transmitting a bitstream generated by a video encoding method or apparatus according to the present disclosure.

[0010] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those with ordinary knowledge in the technical field to which the present disclosure belongs from the following description.

Means for Solving the Problems

[0011] A video decoding method according to an aspect of the present disclosure may include generating a first prediction block based on intra - template matching prediction, generating a second prediction block based on a specific intra - prediction mode, and generating a final prediction block based on the first prediction block and the second prediction block.

[0012] On the other hand, according to an embodiment of the present disclosure, the specific intra - prediction mode may be an intra - prediction mode induced by TIMD or DIMD.

[0013] On the other hand, according to an embodiment of the present disclosure, the second prediction block may be a plurality of prediction blocks generated based on a plurality of intra - prediction modes induced by TIMD or DIMD.

[0014] On the one hand, according to an embodiment of the present disclosure, the specific intra prediction mode can be derived based on the template form of the intra template matching prediction.

[0015] On the one hand, according to an embodiment of the present disclosure, the template form can be obtained from a bitstream.

[0016] On the one hand, according to an embodiment of the present disclosure, the first prediction block can be generated based on a prediction mode selected from the intra template matching prediction mode list.

[0017] On the one hand, according to an embodiment of the present disclosure, the selected prediction mode can be selected based on prediction mode index information.

[0018] On the one hand, according to an embodiment of the present disclosure, the intra template matching prediction mode list can include at least one or more of a default mode, a filtering mode, a sub-pixel search mode, an intra fusion mode, an error-based fusion mode, or a filter-based fusion mode.

[0019] On the one hand, according to an embodiment of the present disclosure, the intra template matching prediction mode list can include prediction modes generated by at least one or more combinations of a default mode, a filtering mode, a sub-pixel search mode, an intra fusion mode, an error-based fusion mode, or a filter-based fusion mode.

[0020] On the other hand, according to one embodiment of the present disclosure, the error-based fusion mode may be based on an error calculated using SAD (Sum of difference), SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), or MR-SATD (Mean-removed sum of transformed difference).

[0021] On the other hand, according to one embodiment of the present disclosure, the first prediction block may be a prediction block formed by combining a plurality of intra-template matching prediction blocks.

[0022] A video encoding method relating to one aspect of this disclosure may include the steps of generating a first prediction block based on intra-template matching prediction, generating a second prediction block based on a specific intra-prediction mode, and generating a final prediction block based on the first and second prediction blocks.

[0023] Computer-readable recording media relating to yet another aspect of this disclosure can store bitstreams generated by the video encoding method or video encoding apparatus of this disclosure.

[0024] A transmission method relating to yet another aspect of the present disclosure may transmit a bitstream generated by a video encoding device or video encoding method of the present disclosure.

[0025] The features briefly summarized above are merely illustrative aspects of the detailed description of the disclosure described below and do not limit the scope of the disclosure. [Effects of the Invention]

[0026] According to this disclosure, a video encoding / decoding method and apparatus with improved encoding / decoding efficiency may be provided.

[0027] Furthermore, this disclosure may provide a video encoding / decoding method and apparatus for generating a final in-screen prediction block using multiple in-screen prediction blocks.

[0028] Furthermore, this disclosure may provide a video encoding / decoding method and apparatus that utilize intraTMP (intra template matching prediction) prediction blocks.

[0029] Furthermore, this disclosure may provide video encoding / decoding methods and apparatus that offer improved prediction efficiency, coding, and image quality.

[0030] Furthermore, this disclosure may provide a non-temporary computer-readable recording medium for storing a bitstream generated by a video encoding method or apparatus relating to this disclosure.

[0031] Furthermore, this disclosure may provide a non-temporary computer-readable recording medium for storing a bitstream that is received and decoded by the video decoding device relating to this disclosure and used for restoring the video.

[0032] Furthermore, this disclosure may provide a method for transmitting a bitstream generated by a video encoding method or apparatus relating to this disclosure.

[0033] The effects that can be obtained from this disclosure are not limited to those mentioned above, and any other effects not mentioned above can be clearly understood by a person with ordinary skill in the art to which this disclosure pertains from the following description. [Brief explanation of the drawing]

[0034] [Figure 1] This figure schematically illustrates a video coding system to which the embodiments described herein can be applied. [Figure 2] This figure schematically shows an image encoding device to which the embodiments of this disclosure can be applied. [Figure 3] This figure schematically shows an image decoding apparatus to which the embodiments of this disclosure can be applied. [Figure 4] This flowchart shows an example of an intra-predictive mode signaling method in an encoding device. [Figure 5] This flowchart shows an example of a method for determining the intra-prediction mode in a decoding device. [Figure 6] This diagram shows an example of a surrounding block used for MPM list guidance. [Figure 7] This is a diagram illustrating the DIMD HoG configuration method. [Figure 8] This is a diagram illustrating how to configure the prediction block in DIMD mode. [Figure 9] This is a diagram illustrating the intra-template matching search domain used. [Figure 10] This is a diagram illustrating a video coding method based on intraTMP mode according to one embodiment of the present disclosure. [Figure 11] These are drawings illustrating the form of a template applicable to one embodiment of the present disclosure. [Figure 12] These are drawings illustrating a template search method and an error calculation method that may be applied to one embodiment of the present disclosure. [Figure 13] This is a diagram illustrating a half-pixel search method that may be applied to one embodiment of the present disclosure. [Figure 14] This is a diagram illustrating a TMP filtering process that may be applied to one embodiment of the present disclosure. [Figure 15] These are drawings illustrating multiple reference template regions and current template regions according to one embodiment of the present disclosure. [Figure 16] This is a diagram illustrating the upper template area of ​​intraTMP according to one embodiment of the present disclosure. [Figure 17] This is a diagram illustrating the left template area of ​​intraTMP according to one embodiment of the present disclosure. [Figure 18] These are drawings illustrating an embodiment of a video decoding method and / or video encoding method according to the present disclosure. [Figure 19] This figure illustrates a content streaming system to which the embodiments described herein can be applied. [Modes for carrying out the invention]

[0035] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings, so that they can be easily implemented by a person with ordinary skill in the art to which the present disclosure pertains. However, the present disclosure can be implemented in a variety of different forms and is not limited to the embodiments described herein.

[0036] In describing embodiments of this disclosure, if it is determined that a specific description of a known configuration or function would obscure the gist of this disclosure, such detailed description will be omitted. In the drawings, parts unrelated to the description of this disclosure will be omitted, and similar parts will be denoted by the same reference numerals.

[0037] In this disclosure, when one component is described as being “connected,” “joined,” or “linked” to another component, this can include not only direct connections but also indirect connections where another component exists between them. Furthermore, when one component is described as “containing” or “having” another component, this means, unless otherwise stated to the contrary, that it may include another component rather than excluding it.

[0038] In this disclosure, terms such as "first," "second," etc., are used solely for the purpose of distinguishing one component from another, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of this disclosure, a first component in one embodiment may be called a second component in another embodiment, and similarly, a second component in one embodiment may be called a first component in another embodiment.

[0039] In this disclosure, components that are distinguished from each other are used to clearly describe their respective characteristics and do not necessarily mean that the components are separate. In other words, multiple components may be integrated to constitute a single hardware or software unit, or a single component may be distributed to constitute multiple hardware or software units. Therefore, such integrated or distributed embodiments are also included in the scope of this disclosure, without needing to be specifically mentioned.

[0040] In this disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments consisting of a subset of the components described in one embodiment are also included in the scope of this disclosure. Furthermore, embodiments that include additional components in addition to the components described in various embodiments are also included in the scope of this disclosure.

[0041] This disclosure relates to the encoding and decoding of images, and the terms used in this disclosure may have their ordinary meanings in the art to which this disclosure pertains, unless they are newly defined in this disclosure.

[0042] In this disclosure, "picture" generally means a unit representing any one image within a specific time period, and "slice / tile" is an encoding unit that constitutes part of a picture, and a single picture can consist of one or more slices / tiles. Furthermore, a slice / tile may contain one or more CTUs (coding tree units).

[0043] In this disclosure, “pixel” or “pel” may mean the smallest unit that constitutes a picture (or image). The term “sample” may also be used as a counterpart to pixel. A sample may generally represent a pixel or a pixel value, or it may represent only the pixel / pixel value of the luma component, or only the pixel / pixel value of the chroma component.

[0044] In this disclosure, “unit” can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. A unit may be used interchangeably with terms such as “sample array,” “block,” or “area,” as it may be used. Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0045] In this disclosure, “current block” can mean any one of the following: “current coding block,” “current coding unit,” “block to encode,” “block to decode,” or “block to process.” If prediction is performed, “current block” can mean “current prediction block” or “block to predict.” If transformation (inverse transformation) / quantization (inverse quantization) is performed, “current block” can mean “current transformation block” or “block to transform.” If filtering is performed, “current block” can mean “block to filter.”

[0046] Furthermore, in this disclosure, "current block" may mean the block containing all of the rumor component blocks and chroma component blocks, or the "rumor block of the current block," unless there is an explicit mention of a chroma block. The rumor component block of the current block may be expressed with an explicit mention of a rumor component block, such as "rumor block" or "current rumor block." Similarly, the chroma component block of the current block may be expressed with an explicit mention of a chroma component block, such as "chroma block" or "current chroma block."

[0047] In this disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Also, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0048] In this disclosure, “or” may be interpreted as “and / or.” For example, “A or B” may mean 1) “A” only, 2) “B” only, or 3) “A and B.” Alternatively, in this disclosure, “or” may mean “additionally or alternatively.”

[0049] Overview of the video coding system

[0050] Figure 1 is a schematic diagram showing a video coding system to which the embodiments of this disclosure can be applied.

[0051] A video coding system according to one embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 can transmit encoded video and / or image information or data to the decoding device 20 via a digital storage medium or network in file or streaming format.

[0052] An encoding device 10 according to one embodiment may include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. A decoding device 20 according to one embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be called a video / image encoding unit, and the decoding unit 22 may be called a video / image decoding unit. The transmission unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may also include a display unit, which may be configured as a separate device or external component.

[0053] The video source generation unit 11 can acquire video / images through processes such as video / image capture, synthesis, or generation. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. The video / image generation device may include, for example, a computer, tablet, and smartphone, and may generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by a process in which the relevant data is generated.

[0054] The encoding unit 12 can encode the input video / image. The encoding unit 12 can perform a series of steps such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit 12 can output the encoded data (encoded video / image information) in bitstream format.

[0055] The transmission unit 13 can acquire encoded video / image information or data output in bitstream format and transmit it in file or streaming format to the receiving unit 21 of the decoding device 20 or other external object via a digital storage medium or network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, and SSD. The transmission unit 13 can include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The transmission unit 13 can be provided as a transmission device separate from the encoding device 12, in which case the transmission device can include at least one processor that acquires encoded video / image information or data output in bitstream format, and a transmission unit that transmits it in file or stream format. The receiving unit 21 can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit 22.

[0056] The decoding unit 22 can decode the video / image by performing a series of steps such as inverse quantization, inverse transform, and prediction, corresponding to the operation of the encoding unit 12.

[0057] The rendering unit 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0058] Overview of Image Encoding Devices

[0059] Figure 2 is a schematic diagram showing an image encoding device to which the embodiments of this disclosure can be applied.

[0060] As shown in Figure 2, the image coding device 100 may include an image splitting unit 110, a subtraction unit 115, a transformation unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transformation unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185, and an entropy coding unit 190. The inter-prediction unit 180 and the intra-prediction unit 185 can together be called the "prediction unit". The transformation unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transformation unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115.

[0061] All or at least some of the multiple components constituting the image encoding device 100 can be implemented by a single hardware component (e.g., an encoder or processor) depending on the embodiment. Furthermore, the memory 170 may include a DPB (decoded picture buffer) and can be implemented by a digital storage medium.

[0062] The image splitting unit 110 can split an input image (or picture, frame) input to the image encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). Coding units can be obtained by recursively splitting a coding tree unit (CTU) or the largest coding unit (LCU) using a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure and / or a ternary-tree structure. For the splitting of coding units, a quad-tree structure may be applied first, followed by a binary-tree structure and / or a ternary-tree structure. Based on the final coding unit that cannot be further split, the coding procedure according to this disclosure can be performed. The largest coding unit can be used as the final coding unit, or a lower-depth coding unit obtained by dividing the largest coding unit can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and / or restoration, as described later. As another example, the processing units of the coding procedure may be prediction units (PU) or transformation units (TU). The prediction unit and the transformation unit may be divided or partitioned from the final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0063] The prediction unit (inter-prediction unit 180 or intra-prediction unit 185) can make predictions for the block to be processed (current block) and generate a predicted block that includes prediction samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block or on a CU basis. The prediction unit can generate various information regarding the prediction of the current block and transmit it to the entropy coding unit 190. The prediction information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0064] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) or at a distance from the current block, according to the intra-prediction mode and / or intra-prediction technique. The intra-prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0065] The interprediction unit 180 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different from each other. The temporal neighboring block may be called a collocated reference block, collocated CU (colCU), etc. The reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the interpretation unit 180 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of the surrounding block is used as the motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference can represent the difference between the motion vector of the current block and the motion vector predictor.

[0066] The prediction unit can generate a prediction signal based on various prediction methods and / or techniques described later. For example, the prediction unit can apply intra-prediction or inter-prediction to predict the current block, and can also apply intra-prediction and inter-prediction simultaneously. A prediction method that applies intra-prediction and inter-prediction simultaneously to predict the current block can be called CIIP (combined inter and intra prediction). The prediction unit can also perform intra-block copy (IBC) to predict the current block. Intra-block copy can be used for content image / video coding such as in games, for example, in SCC (screen content coding). IBC is a method of predicting the current block using a reference block that has already been restored in the current picture at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but it can be performed similarly to inter-prediction in that it derives the reference block within the current picture. In other words, IBC can use at least one of the interpretation techniques described in this disclosure.

[0067] The predicted signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. The subtraction unit 115 can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal output from the prediction unit (predicted block, predicted sample array) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit 120.

[0068] The transformation unit 120 can generate transformation coefficients by applying transformation techniques to the residual signal. For example, the transformation technique may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph, where the relationship information between pixels is represented by a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. The transformation process can be applied to pixel blocks of the same size and square shape, or to non-square, variable-sized blocks. The quantization unit 130 can quantize the conversion coefficients and transmit them to the entropy coding unit 190. The entropy coding unit 190 can encode the quantized signal (information about the quantized conversion coefficients) and output it in bitstream format. The information about the quantized conversion coefficients can be called residual information. The quantization unit 130 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector format based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector format of the quantized conversion coefficients.

[0069] The entropy coding unit 190 can perform various coding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy coding unit 190 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together or separately. The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream format in units of NAL (network abstraction layer) units. The video / image information may further include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. The signaling information, transmitted information and / or syntax elements referred to in this disclosure may be encoded via the encoding procedure described above and included in the bitstream.

[0070] The bitstream can be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, and SSD. A transmission unit (not shown) for transmitting the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) for storing it may be provided as internal / external elements of the image encoding device 100, or the transmission unit may be provided as a component of the entropy encoding unit 190.

[0071] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 140 and the inverse transformation unit 150, a residual signal (residual block or residual sample) can be reconstructed.

[0072] The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or the intra-prediction unit 185. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 155 may be called the reconstruction unit or the reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0073] The filtering unit 160 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 160 can generate various filtering-related information, as will be described later in the explanation of each filtering method, and transmit it to the entropy coding unit 190. The filtering-related information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0074] The corrected restored picture transmitted to memory 170 can be used as a reference picture in the interpretation unit 180. When interpretation is applied via this, the image encoding device 100 can avoid prediction mismatches between the image encoding device 100 and the image decoding device, and can also improve encoding efficiency.

[0075] The DPB in memory 170 can store the modified restored picture for use as a reference picture in the inter-prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 180 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 170 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 185.

[0076] Overview of the image decoding device

[0077] Figure 3 is a schematic diagram showing an image decoding apparatus to which the embodiments of this disclosure can be applied.

[0078] As shown in Figure 3, the image decoding device 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an additive unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 can together be called the "prediction unit". The inverse quantization unit 220 and the inverse transform unit 230 can be included in the residual processing unit.

[0079] All or at least some of the multiple components constituting the image decoding device 200 can be implemented by a single hardware component (e.g., a decoder or processor) according to the embodiment. Furthermore, the memory 170 may include a DPB and can be implemented by a digital storage medium.

[0080] An image decoding device 200, having received a bitstream containing video / image information, can restore the image by executing a process corresponding to the process performed in the image encoding device 100 in Figure 2. For example, the image decoding device 200 can perform decoding using the processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. The restored image signal decoded and output via the image decoding device 200 can then be reproduced via a playback device (not shown).

[0081] The image decoding device 200 can receive the signal output from the image encoding device 2 in bitstream format. The received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The image decoding device may further use the parameter set information and / or the general constraint information to decode the image. The signaling information, received information, and / or syntax elements referred to in this disclosure can be obtained from the bitstream by decoding via the decoding procedure. For example, the entropy decoding unit 210 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding blocks and the blocks to be decoded, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 210, information related to prediction is provided to the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and the residual values ​​that have undergone entropy decoding in the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. In addition, of the information decoded by the entropy decoding unit 210, information related to filtering can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives signals output from the image coding device may be further provided as an internal / external element of the image decoding device 200, or the receiving unit may be provided as a component of the entropy decoding unit 210.

[0082] On the other hand, the image decoding device according to this disclosure may be called a video / image / picture decoding device. The image decoding device may also include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder may include at least one of an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265.

[0083] The inverse quantization unit 220 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients in a two-dimensional block format. In this case, the rearrangement can be performed based on the coefficient scan order performed by the image encoding device. The inverse quantization unit 220 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0084] The inverse conversion unit 230 can inversely convert the conversion coefficients to obtain residual signals (residual blocks, residual sample arrays).

[0085] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode (prediction technique).

[0086] As described in the explanation of the prediction unit of the image coding device 100, the prediction unit can generate prediction signals based on various prediction methods (techniques) described later.

[0087] The intra-prediction unit 265 can predict the current block by referring to the samples in the current picture. The description of the intra-prediction unit 185 can also be applied to the intra-prediction unit 265.

[0088] The interprediction unit 260 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on a reference picture. In this case, to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in block, sub-block, or sample units based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 260 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes (techniques), and the prediction information may include information indicating the mode (technique) of interprediction for the current block.

[0089] The adder 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 260 and / or intra-prediction unit 265). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The description of the adder 155 also applies to the adder 235. The adder 235 is sometimes called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or for inter-prediction of the next picture via filtering, as described later.

[0090] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 250, specifically in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0091] The restored picture stored (modified) in the DPB of memory 250 can be used as a reference picture in the inter-prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 250 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 265.

[0092] In this specification, the embodiments described for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the image coding device 100 can be applied similarly or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the image decoding device 200, respectively.

[0093] Intra Prediction Mode / Type Determination

[0094] When intraprediction is applied, the intraprediction mode applied to the current block may be determined by utilizing the intraprediction modes of the surrounding blocks. For example, the decoding device may select one of the mpm candidates in an mpm (most probable mode) list derived based on the intraprediction modes of the surrounding blocks (e.g., the left and / or upper surrounding blocks) of the current block, and additional candidate modes, based on the received mpm index, or it may select one of the remaining intraprediction modes not included in the mpm candidates (and planner modes), based on the remaining intraprediction mode information. The mpm list may or may not include planner modes as candidates. For example, if the mpm list includes planner modes as candidates, the mpm list may have 6 candidates, and if the mpm list does not include planner modes as candidates, the mpm list may have 3 candidates. If the aforementioned mpm list does not include planner mode as a candidate, a not-planar flag (e.g., intra_luma_not_planar_flag) may be signaled to indicate whether the current intra-prediction mode of the block is not planner mode. For example, the mpm flag may be signaled first, and the mpm index and not-planar flag may be signaled if the value of the mpm flag is 1. Also, the mpm index may be signaled if the value of the not-planar flag is 1. Here, the reason why the aforementioned mpm list is configured not to include planner mode as a candidate is not because planner mode is not mpm, but because planner mode is always considered as mpm, so the flag (not-planar flag) is signaled first to check whether it is planner mode or not.

[0095] For example, whether the intra-prediction mode currently applied to a block is among the MPM candidates (and planner modes) or in the remaining mode can be indicated by the mpm flag (e.g., intra_luma_mpm_flag). A value of 1 for the mpm flag indicates that the intra-prediction mode for the current block is among the MPM candidates (and planner modes), and a value of 0 for the mpm flag indicates that the intra-prediction mode for the current block is not among the MPM candidates (and planner modes). A value of 0 for the not planar flag (e.g., intra_luma_not_planar_flag) indicates that the intra-prediction mode for the current block is planner mode, and a value of 1 for the not planar flag indicates that the intra-prediction mode for the current block is not planner mode. The aforementioned mpm index may be signaled in the form of mpm_idx or intra_luma_mpm_idx syntax elements, and the remaining intra prediction mode information may be signaled in the form of rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax elements. For example, the remaining intra prediction mode information can indicate one of the remaining intra prediction modes from the total intra prediction modes that are not included in the mpm candidates (and planner modes) by indexing them in order of prediction mode number. The intra prediction mode may be an intra prediction mode for a luma component (sample). The intra prediction mode information may include at least one of the following: the mpm flag (e.g., intra_luma_mpm_flag), the not planar flag (e.g., intra_luma_not_planar_flag), the mpm index (e.g., mpm_idx or intra_luma_mpm_idx), or the retaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder).In this document, the MPM list may be referred to by various terms such as the MPM candidate list or candModeList. When an MIP is applied to the current block, a separate mpm flag (e.g., intra_mip_mpm_flag), mpm index (e.g., intra_mip_mpm_idx), and remaining intra prediction mode information (e.g., intra_mip_mpm_remainder) for the MIP may be signaled, while the not planar flag is not signaled.

[0096] The intra-predictive mode signaling procedure in the encoding device and the intra-predictive mode determination procedure in the decoding device can be performed, for example, as follows:

[0097] Figure 4 is a flowchart illustrating an example of an intra-predictive mode signaling method in an encoding device.

[0098] Referring to Figure 4, the encoding device configures an MPM list for the current block (S400). The MPM list may include candidate intra-prediction modes (MPM candidates, etc.) that are likely to be applied to the current block. The MPM list may also include intra-prediction modes for surrounding blocks, and may further include specific intra-prediction modes by a predetermined method. A specific method for configuring the MPM list will be described later.

[0099] The encoding device determines the intra-prediction mode for the current block (S410). The encoding device can perform predictions based on a variety of intra-prediction modes and determine the optimal intra-prediction mode based on rate-distortion optimization (RDO) derived therefrom. In this case, the encoding device may determine the optimal intra-prediction mode using only the MPM candidates and planner modes configured in the MPM list, or it may determine the optimal intra-prediction mode using not only the MPM candidates and planner modes configured in the MPM list but also the remaining intra-prediction modes. Specifically, for example, if the intra-prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP) and not a normal intra-prediction type, the encoding device can determine the optimal intra-prediction mode by considering only the MPM candidates and planner modes as intra-prediction mode candidates for the current block. That is, in this case, the intra-prediction mode for the current block can be determined only from among the MPM candidates and planner modes, and in this case, the mpm flag may not be encoded / signaled. In this case, the decoding device can infer that the mpm flag is 1 without receiving separate signaling for the mpm flag.

[0100] On the other hand, if the intra prediction mode of the current block is generally one of the MPM candidates in the MPM list rather than the planner mode, the encoding device generates an mpm index (mpm idx) that points to one of the MPM candidates.

[0101] If the intra-prediction mode for the current block is not present in the MPM list, then retaining intra-prediction mode information is generated that points to the same mode as the intra-prediction mode for the current block from among the remaining intra-prediction modes not included in the MPM list (and planner modes).

[0102] The encoding device can encode intra-prediction mode information and output it in bitstream format. The intra-prediction mode information may include the aforementioned mpm flag, not planar flag, mpm index, and / or remaining intra-prediction mode information. Generally, the mpm index and remaining intra-prediction mode information are in an alternative relationship and are not signaled simultaneously when indicating the intra-prediction mode for a single block. That is, either an mpm flag value of 1 and the not planar flag or mpm index are signaled together, or an mpm flag value of 0 and the remaining intra-prediction mode information are signaled together. However, as mentioned above, if a specific intra-prediction type is currently applied to the block, the mpm flag may not be signaled, and only the not planar flag and / or mpm index may be signaled. That is, in this case, the intra-prediction mode information may include only the not planar flag and / or mpm index.

[0103] The decoding device can determine the intra-prediction mode in response to the intra-prediction mode information determined and signaled by the encoding device.

[0104] Figure 5 is a flowchart showing an example of a method for determining the intra-prediction mode in a decoding device.

[0105] Referring to Figure 5, the decoding device obtains intra-prediction mode information from the bitstream (S500). As described above, the intra-prediction mode information may include at least one of the following: mpm flag, not planar flag, mpm index, and remaining intra-prediction mode.

[0106] The decoding device constitutes an MPM list (S510). The MPM list is configured identically to the MPM list configured by the encoding device. That is, the MPM list may include intra-prediction modes for surrounding blocks, and may further include specific intra-prediction modes by a predetermined method. A specific method for configuring the MPM list will be described later.

[0107] Although S510 is illustrated as being performed after S500, this is merely an example, and S510 may be performed before or at the same time as S500.

[0108] The decoding device determines the intra-prediction mode of the current block based on the MPM list and the intra-prediction mode information (S520). For example, if the value of the mpm flag is 1, the decoding device may derive the planner mode as the intra-prediction mode of the current block (not based on the planar flag) or the candidate pointed to by the mpm index among the MPM candidates in the MPM list as the intra-prediction mode of the current block. As another example, if the value of the mpm flag is 0, the decoding device may derive the intra-prediction mode pointed to by the remaining intra-prediction mode information among the remaining intra-prediction modes not included in the MPM list and planner mode as the intra-prediction mode of the current block. On the other hand, as yet another example, if the intra-prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP), the decoding device may derive the candidate pointed to by the mpm index in the planner mode or the mpm index in the MPM list as the intra-prediction mode of the current block without checking the mpm flag.

[0109] For example, the not planar flag may be signaled when the MRL is not currently subject to a block (i.e., intra_luma_ref_idx == 0), and the not planar flag may be omitted when the MRL is currently subject to a block (i.e., intra_luma_ref_idx != 0). If the not planar flag is omitted, its value may be estimated by the decoding device to be 1.

[0110] On the other hand, the intra-prediction mode can include two directional intra-prediction modes and 65 directional intra-prediction modes. The non-directional intra-prediction mode can include a planar intra-prediction mode and a DC intra-prediction mode, and the directional intra-prediction mode can include intra-prediction modes 2 through 66. The extended directional intra-prediction mode can be applied to blocks of all sizes and can be applied to either the lumern component or the chromern component.

[0111] On the other hand, the intra-prediction mode may further include a CCLM (cross-component linear model) mode for chroma samples, in addition to the intra-prediction mode described above. The CCLM mode can be divided into LT_CCLM, L_CCLM, and T_CCLM depending on whether the left sample, the upper sample, or both are considered for LM parameter derivation, and can be applied only to chroma components.

[0112] The intra-prediction mode can be indexed, for example, as shown in Table 1 below.

[0113] [Table 1]

[0114] On the other hand, the intra prediction type (or additional intra prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, ISP, and MIP. The intra prediction type may be indicated based on intra prediction type information, and the intra prediction type information may be embodied in various forms. For example, the intra prediction type information may include intra prediction type index information that indicates one of the intra prediction types. As another example, the intra prediction type information may include reference sample line information (e.g., intra_luma_ref_idx) indicating whether the MRL is applied to the current block and, if so, which reference sample line is used; ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether the ISP is applied to the current block; ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of the subpartition when the ISP is applied; and flag information indicating whether PDCP is applied or flag information indicating whether LIP is applied. Furthermore, the intra prediction type information may include an MIP flag (or possibly called intra_mip_flag) indicating whether MIP is applied to the current block. On the other hand, as mentioned above, if MIP is applied to the current block (e.g., if the value of intra_mip_flag is 1), an MPM list for the MIP may be configured separately, and the MPM flag that may be included in the intra prediction mode information for the MIP may be called intra_mip_mpm_flag, the MPM index may be called intra_mip_mpm_idx, and the remaining intra prediction mode information may be called intra_mip_mpm_remainder.

[0115] Furthermore, various prediction modes can be used for MIP, and the intra-prediction modes for MIP can be used to derive the matrix and offset for MIP. As previously mentioned, the matrix may be called the (MIP)weighted matrix, and the offset may be called the (MIP)offset vector or (MIP)bias vector. The number of intra-prediction modes for MIP can be set differently based on the size of the block. For example, i) if the height and width of the block (e.g., CB or TB) are both 4, then 35 intra-prediction modes (i.e., intra-prediction modes 0-34) may be available; ii) if both the height and width of the block are 8 or less, then 19 intra-prediction modes (i.e., intra-prediction modes 0-18) may be available; and iii) in other cases, then 11 intra-prediction modes (i.e., intra-prediction modes 0-10) may be available. For example, if the current block height and width are both 4, it is called block size type 0; if the current block height and width are both 8 or less, it is called block size type 1; and all other cases are called block size type 2, then the number of intra-prediction modes for MIP may be organized as shown in the following table. However, this is merely an example, and the number of block size types and available intra-prediction modes may change. In this document, intra-prediction modes for MIP may be called MIP intra-prediction modes, MIP prediction modes, or MIP modes.

[0116] [Table 2]

[0117] On the other hand, the enhanced compression model (ECM) introduced a secondary MPM list. The existing primary MPM (PMPM) list consists of 6 entries, and the secondary MPM (SMPM) list contains 16 entries. A general MPM list with 22 entries is first configured, then the first 6 entries from the general MPM list are included in the PMPM list, and the remaining entries are included in the SMPM list. The first entry in the general MPM list is the planner mode, and the remaining entries consist of intra-modes for left (L), upper (A), left-lower (BL), right-upper (AR), and left-upper (AL) peripheral blocks, as shown in Figure 6, directional modes with an offset added to the first two available directional modes of the peripheral blocks, and the default mode. When the CU block is vertical, the order of the peripheral blocks can be upper (A), left (L), left-lower (BL), right-upper (AR), and left-upper (AL). Alternatively, the order is left side (L), top side (A), left-bottom side (BL), right-top side (AR), and left-top side (AL).

[0118] The PMPM flag is parsed, and if its value is 1, the PMPM index is parsed to determine which entry in the PMPM list is selected. Alternatively, the SPMPM flag is parsed to determine whether to parse the SPMPM index or the remaining mode.

[0119] Derivation of peripheral reference samples

[0120] When intraprediction is applied to the current block, peripheral reference samples to be used for intraprediction of the current block can be derived. The peripheral reference samples of the current block may include a total of 2 x nH samples adjacent to the left boundary and bottom-left of the current block of size nW x nH, a total of 2 x nW samples adjacent to the top boundary and top-right of the current block, and one sample adjacent to the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples in multiple columns and left peripheral samples in multiple rows. Furthermore, the peripheral reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW x nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0121] On the other hand, when MRL (Multiple Reference Line), which will be discussed later, is applied, the reference sample can be located on lines 1 through 3, rather than on line 0 adjacent to the current block on the left / above side. In this case, the number of peripheral reference samples may increase further. The specific regions and numbers of peripheral reference samples will be described later.

[0122] On the other hand, if the ISP described later is applied, the aforementioned peripheral reference samples can be derived on a subpartition basis.

[0123] Currently, some of the surrounding reference samples in a block may not have been decoded yet or may not be available. In this case, the decoder can construct the surrounding reference samples to use for prediction by interpolating the available samples.

[0124] Currently, some of the surrounding reference samples in a block may not yet be decoded or available. In this case, the decoder can construct the surrounding reference samples to use for prediction through extrapolation of available samples. Starting from the bottom-left corner and reaching the top-right corner reference sample, the decoder can construct the sample by updating the available samples with the latest sample (last available sample) and substituting or padding pixels that have not yet been decoded or are unavailable with the last available sample.

[0125] Derivation of Intra Prediction Mode / Type-Based Prediction Samples

[0126] The prediction unit of the encoding / decoding device can derive reference samples from the surrounding reference samples of the current block using the intra-prediction mode of the current block, and can generate predicted samples of the current block based on the reference samples.

[0127] For example, (i) a predicted sample may be derived based on the average or interpolation of neighboring reference samples of the current block, or (ii) the predicted sample may be derived based on reference samples among the neighboring reference samples of the current block that are located in a specific (predicted) direction relative to the predicted sample. Case (i) may be called a non-directional mode or non-angular mode, and case (ii) may be called a directional mode or angular mode. Alternatively, the predicted sample may be generated by interpolating the first neighbor sample with the second neighbor sample located in the opposite direction to the prediction direction of the intra-prediction mode of the current block, based on the predicted sample of the current block. In the above case, this may be called linear interpolation intra-prediction (LIP). Alternatively, a temporary predicted sample of the current block may be derived based on filtered neighboring reference samples, and the predicted sample of the current block may be derived by a weighted sum of the existing neighboring reference samples, i.e., at least one reference sample derived by the intra-prediction mode from the unfiltered neighboring reference samples, and the temporary predicted sample. In the aforementioned case, it may be called PDPC (Position-dependent intraprediction). Alternatively, intra-predictive coding can be performed by selecting the reference sample line with the highest prediction accuracy from among the surrounding multiple reference sample lines of the current block, using the reference sample located in the prediction direction on that line to derive the predicted sample, and then instructing (signaling) the decoding device with the reference sample line used. In the aforementioned case, it may be called multi-reference line intra-prediction (MRL) or MRL-based intra-prediction. Furthermore, the current block can be divided into vertical or horizontal subpartitions, and intra-prediction can be performed based on the same intra-prediction mode, but surrounding reference samples can be derived and used on a subpartition-by-subpartition basis.In other words, in this case, the intra-prediction mode for the current block is applied identically to the subpartition, but the intra-prediction performance can be improved in some cases by deriving and utilizing surrounding reference samples on a subpartition-by-subpartition basis. Such a prediction method may be called intra sub-partitions (ISP) or ISP-based intra-prediction. The specific details will be described later. Furthermore, if the prediction direction based on the prediction sample points between surrounding reference samples, that is, if the prediction direction points to a fractional sample position, the value of the prediction sample may be derived through interpolation of multiple reference samples located around the relevant prediction direction (around the relevant fractional sample position).

[0128] The intra-prediction method described above may be called an intra-prediction type, distinct from the intra-prediction mode. The intra-prediction type may be referred to by various terms such as intra-prediction technique or additional intra-prediction mode. For example, the intra-prediction type (or additional intra-prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, and ISP. Information regarding the intra-prediction type may be encoded by an encoding device and included in a bitstream to signal to a decoding device. Information regarding the intra-prediction type may be embodied in various forms, such as flag information indicating whether each intra-prediction type is applied or index information indicating one of the various intra-prediction types.

[0129] The MPM list used to derive the aforementioned intra-prediction mode may be configured differently depending on the intra-prediction type. Alternatively, the MPM list may be configured in common regardless of the intra-prediction type.

[0130] Derivation of peripheral reference samples

[0131] When intraprediction is applied to the current block, peripheral reference samples to be used for intraprediction of the current block can be derived. The peripheral reference samples of the current block may include a total of 2 x nH samples adjacent to the left boundary and bottom-left of the current block of size nW x nH, a total of 2 x nW samples adjacent to the top boundary and top-right of the current block, and one sample adjacent to the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples in multiple columns and left peripheral samples in multiple rows. Furthermore, the peripheral reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW x nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0132] On the other hand, when MRL (Multiple Reference Line) is applied, the reference sample can be located on lines 1 through 3, rather than line 0 adjacent to the current block on the left / above side. In this case, the number of peripheral reference samples can increase further. The specific regions and numbers of peripheral reference samples will be described later.

[0133] On the other hand, when ISP (Intra Sub-Partitions) is applied, the aforementioned peripheral reference samples can be derived on a sub-partition basis.

[0134] DIMD(Decoder side intra mode derivation)

[0135] DIMD mode allows the use of intra-prediction mode information by inducing it with an encoder and decoder without directly transmitting it. First, the horizontal and vertical gradients can be obtained from the second neighboring sample column and row, and then a Histogram of gradients (HoG) can be constructed.

[0136] Figure 7 is a diagram illustrating the DIMD HoG configuration method, and Figure 8 is a diagram illustrating the prediction block configuration method in DIMD mode. The HoG can be configured as shown in Figure 7. The HoG can currently be obtained by applying a Sobel filter using the L-shaped row and column of 3 pixels around the block. If the block boundaries are located in different CTUs, they are not used for texture analysis.

[0137] Subsequently, as shown in Figure 7, the two intra modes with the largest histogram amplitudes are selected, and then the prediction blocks predicted using these modes and the planar mode are blended to construct the final prediction block. The weights can be derived from the histogram amplitude. In addition, a DIMD flag is transmitted on a block-by-block basis to confirm whether DIMD is being used.

[0138] In DIMD, the intra-prediction can be derived as a weighted average between the planner and two induced directions. For this purpose, two angular modes are selected from the HoG (Histogram of Gradient) calculated from the surrounding pixels of the current block. Once the two modes are selected, their predictors (prediction blocks) and planner predictors are normally calculated, and the corresponding weighted average can be used as the final predictor (final prediction block) of the current block. At this time, the corresponding amplitudes in the HoG are used for each of the two modes to determine the weights.

[0139] Since the induced intra-mode will be included in the primary list of the intra-MPM, the DIMD process can be performed before the MPM list is constructed. The initially induced intra-mode of a DIMD block is saved with the block and can be used to construct the MPM list of surrounding blocks.

[0140] The DIMD chroma mode can use a DIMD induction method to derive the chroma intra-prediction mode of the current block based on pre-recovered peripheral Y, Cb, and Cr samples in the surrounding rows and columns. Specifically, to build the HoG, horizontal and vertical gradients can be calculated for each collocated pre-recovered chroma sample of the current chroma block, as well as for the pre-recovered Cb and Cr samples. Subsequently, the chroma intra-prediction of the current chroma block can be performed using the intra-prediction mode with the largest histogram amplitude value.

[0141] If the intra-prediction mode derived from the DIMD chroma mode is the same as the intra-prediction mode derived from the DM mode, the intra-prediction mode with the second largest histogram amplitude value may be used as the DIMD chroma mode. A predetermined CU level flag may be signaled to indicate whether or not the aforementioned DIMD chroma mode is applied.

[0142] TIMD(Fusion for template-based intra mode derivation)

[0143] For each intra-prediction mode within the MPM, the SATD between the template's predicted and recovered samples can be calculated. The first two intra-prediction modes with the smallest SATD can then be selected as the TIMD mode. These two TIMD modes can be fused using weighted values, and such weighted intra-predictions can currently be used to code CUs. The derivation of TIMD modes may include the position-dependent intra-prediction combination (PDPC) described above.

[0144] The costs of the two selected modes are compared with a predetermined critical value, and a cost factor 2 can be applied as shown in mathematical formula 1 below.

[0145] [Mathematical formula 1]

number

[0146] If the condition in mathematical formula 1 is true, the aforementioned fusion may be applied. Conversely, if the condition in mathematical formula 1 is false, only mode 1 may be used.

[0147] On the other hand, the weighted values ​​of the above modes can be calculated from the respective SATD costs as shown in mathematical formula 2 below.

[0148] [Mathematical formula 2]

number

[0149] intraTMP(Intra template matching prediction)

[0150] Intra template matching prediction (intraTMP)

[0151] The L-shaped template is a special prediction mode in which the best prediction block is copied from the reconstructed part of the current frame that matches the current template. The encoder searches for the template most similar to the current template within the reconstructed part of the current frame within a predefined search range and uses the corresponding block as the prediction block. Subsequently, the encoder signals whether or not this mode is being used, and the same prediction operation can be performed on the decoder side.

[0152] Figure 9 is a diagram illustrating the intra-template matching search area used.

[0153] The predicted signal can be generated by matching the current block's L-shaped causal neighbor with other blocks in a predefined search area, as shown in Figure 9. In Figure 9, R1 is the current CTU, R2 is the top-left CTU, R3 is the top-end CTU, and R4 is the left-side CTU. SAD (Sum of absolute differences) is used as the cost function. Within each area, the decoder searches for a template with the minimum SAD for the current block and uses the corresponding block, i.e., the reference block, as the predicted block. The size of all areas (SearchRange) w SearchRange h The number of SAD comparisons per pixel can be performed a fixed number of times, proportional to the block size (BlkW, BlkH). This can be done as follows:

[0154] [Mathematical formula 3]

number

[0155] Here, "a" controls the gain / complexity trade-off.

[0156] It is a constant. "a" can be 5.

[0157] To improve the speed of the template matching process, the search range of all search areas is subsampled through factor 2. This reduces the template matching search to 4, and after finding the best match, an improvement process is performed. This improvement is carried out through a second template matching search centered on the best match using the reduced range. Here, the reduced range is defined as min(BlkW, BlkH) / 2.

[0158] The intra-template matching tool is activated for CUs with a width and height of 64 or less. The maximum CU size for intra-template matching is configurable. The intra-template matching prediction mode is currently signaled at the CU level via a dedicated flag if DIMD is not being used for the CU.

[0159] This disclosure proposes a method for inducing an IntraTMP mode reference block and an Intra mode when encoding still images or videos on a screen. By predicting the final predicted block through various prediction block combination methods, we expect to improve prediction performance and, consequently, improve coding efficiency.

[0160] According to one embodiment of this disclosure, reference blocks of intra-modes and IntraTMP modes induced by TIMD / DIMD can be combined. Also according to one embodiment of this disclosure, multiple intra-modes and prediction blocks of IntraTMP modes induced by TIMD / DIMD can be combined. Furthermore, according to one embodiment of this disclosure, intra-modes can be induced by TIMD / DIMD using the same form as the template shape. Furthermore, according to one embodiment of this disclosure, IntraTMP signaling can be improved, various IntraTMP modes can be managed in list form, and prediction blocks of IntraTMP can be generated in various ways.

[0161] The above embodiment will be described in detail below with reference to the drawings.

[0162] Example 1

[0163] Figure 10 is a diagram illustrating a video coding method based on intraTMP mode according to one embodiment of the present disclosure. In this embodiment, a method is proposed for generating a new prediction block by combining a prediction block generated by IntraTMP (i.e., a corresponding block in intraTMP) with a prediction block generated in intra-mode induced by DIMD or TIMD. Since a prediction block formed by combining prediction blocks can improve intra-prediction performance, one embodiment of the present disclosure describes a technique for generating multiple prediction blocks based on DIMD and TIMD techniques. Similarly, intra-prediction performance can be improved by generating a prediction block using intra-mode induced by DIMD or TIMD and combining it with a prediction block generated by intraTMP technology to generate a final prediction block.

[0164] Referring to the intraTMP mode process in Figure 10, when the intraTMP flag (flag) (S1001) step is performed in the video decoding device, i.e., in the video decoding process, it may correspond to the step of first acquiring the IntraTMP flag or inducing the value of the flag. However, when the intraTMP flag (S1001) step is performed in the video encoding device, i.e., in the video encoding process, it may correspond to the step of determining whether or not to apply intraTMP and generating the intraTMP flag. Although step S1001 determines whether or not to apply IntraTMP mode, if it is decided, i.e., selected, to apply intraTMP, steps S1002 to S1012 may be performed. Although each step is described below, each step below may be omitted in other embodiments of this disclosure, the order of each step may be changed, and some steps may be further added. In addition, the signaling order of each step and information may be changed.

[0165] After it is decided that intraTMP will be applied, the template shape, i.e., the template position, may be guided (Get template shape) (S1002). For example, step S1002 may correspond to the process of guiding one template shape (template position) among multiple template positions, i.e., template shapes. For example, there can be a variety of template shapes, including the examples shown in Figure 9. Figure 11 is a drawing illustrating a template shape applicable to one embodiment of the present disclosure. For example, template shapes applicable to the present disclosure may include L-shape, above shape, left shape, etc., and the template shape may be explicitly signaled through the bitstream (e.g., template shape information, template shape index for a template shape list, etc.), or a predefined shape may be used under specific conditions (e.g., block size, etc.) without signaling by agreement between encoder / decoder. Another example is that the template shape may be guided based on other information.

[0166] Once the template form is determined, the optimal template can be searched for (search best N templates) (S1003). For example, in step S1003, one or more optimal templates may be searched. Alternatively, a template search can be performed to find N positions in ascending order of error (e.g., cost, etc.) and determine the optimal template. Figure 12 is a diagram illustrating a template search method and an error calculation method that may be applied to one embodiment of the present disclosure. For example, as shown in Figure 12, the template search method can calculate the error (e.g., Sum of difference (SAD)) between adjacent template regions of the current block and adjacent template regions of the reference block, and then search for N (where N is an integer) template positions in ascending order of error value (e.g., cost, etc.) to determine the optimal template form. On the other hand, since this is an embodiment of the present disclosure, various error calculation methods, including SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), MR-SATD (Mean-removed sum of transformed difference), etc., can be used to calculate the error between adjacent template regions of a current block and adjacent template regions of a reference block. It is obvious that other error calculation methods used are also included in the present disclosure. In particular, the encoder may determine information regarding the error value and the decoder may explicitly signal it, but the decoder may also be guided in a certain way under specific conditions by a predefined agreement, or the error calculation method may be guided based on other information, and this is also included in the present disclosure.

[0167] Subsequently, it can be determined whether filtering fusion is applied (S1004). For example, if the example in Figure 10 is performed on the encoder side, step S1004 may include a step of determining whether filtering fusion is applied and encoding the presence or absence of filtering fusion application as information (e.g., in flag form). On the other hand, as an example, if the example in Figure 10 is performed on the decoder side, step S1004 includes a step of determining whether filtering fusion is applied, but may also include a step of decoding the presence or absence of filtering fusion application as information (e.g., in flag form) from the bitstream. On the other hand, as another example, the presence or absence of filtering fusion application may not be obtained from the bitstream, but may be derived from other information or derived to a specific value based on conditions predefined on the decoding side.

[0168] Thereafter, if filtering fusion is not applied (for example, if the value of the filtered fusion flag is false (0) on the decoding side), the template position candidates may be reordered (S1005). In step S1005, the N reference block candidates derived from the N optimal templates obtained through template search may be reordered to improve the performance of fusion. The N reference block candidates may be managed in a reference block list, and reordering may be performed on the corresponding list. As an example, when fusing a specific reference block candidate with an intra-prediction block generated in a specific intra-mode, candidate reordering may be performed. For example, each of the reference block candidates may be fused with a specific intra-mode, but each block may be assigned a weight value of w, 1-w. The weight values ​​applied to each block may be the same, or they may be different, and the error value (e.g., Cost) is obtained through fusion based on the weight values. i) can be calculated. Thereafter, for each candidate, rearrangement of the candidates can be performed in ascending order of the error value (e.g., cost). As an example, the fusion of the corresponding block can be performed based on the following mathematical formula 4.

[0169] 〔Mathematical formula 4〕

Number

[0170] As an example, as in the above mathematical formula 4, the error (e.g., cost) calculated by block fusion can be induced based on the restored samples, that is, the restored samples, ReconSamples, in the template area adjacent to the current block. Also, as in the above mathematical formula 4, the error (e.g., cost) calculated by block fusion can be induced based on the restored samples Template of the template area adjacent to any i-th reference block candidate i Thereof. Also, as in the above mathematical formula 4, the error (e.g., cost) calculated by block fusion can be induced based on the predicted samples Template generated in the intra mode in the template adjacent to the current block Intra Thereof. Also, in the above mathematical formula 4, i can be an integer satisfying 0 <= i < N, w can be a real number, SAD(a, b) is a function for the sum of differences, and can indicate the sum of differences between block a and block b. On the other hand, as described above, SAD(a, b) can be replaced with other error calculation functions. As an example, the restored samples Template i And the predicted samples Template generated in the intra mode Intra Can be induced in the same manner as in FIG. 12. On the other hand, as an example, the weighting value w of the above mathematical formula 4 can be the same as the weighting value w for generating the final predicted block in the intraTMP fusion (S1010) stage. Also, as an example, the Template of the above mathematical formula 4 IntraThe intra mode for generating Intra may be the same as the intra mode used to generate the intra prediction block in the intraTMP fusion (S1010) stage. For example, if intra modes p and q are used to generate the intra prediction block in the intraTMP fusion (S1010) stage, then p and q may be used as the intra modes for generating

[0171] [Equation 5]

Number

[0172] As an example, in Equation 5, a1, a2,..., a k can be k reference block candidates that have not undergone fusion. Also, (a1, IntraMode), (a2, IntraMode),...,(a k , IntraMode) can indicate the reference block candidates that are fused with the block by a specific intra mode. IntraMode can mean an intra prediction mode induced by TIMD or DIMD, etc. k is a real number and can be, for example, 8. On the other hand, when there are multiple intra modes induced by TIMD or DIMD, the rearrangement candidate list can be induced as follows.

[0173] [Equation 6]

Number

[0174] According to mathematical formula 6, there may be multiple induced intra-prediction modes that can be included in the re-sorting candidate list, which may be intra-prediction modes induced by TIMD or DIMD, where p is a real number, for example, 2, and the combination between the induced intra-prediction mode and the reference block may be induced based on the product of the number of reference blocks and the number of induced intra-prediction modes. The candidate blocks in the re-sorting candidate list may be sorted in ascending order of error (e.g., cost) as described above. As an example, the error can be calculated as follows.

[0175] (1) For a block candidate that combines a reference block candidate and an intramode, the error price is derived based on mathematical formula 4.

[0176] (2) For reference block candidates that do not combine intra-mode, guidance is provided based on the sum of the errors (e.g., Sum of difference (SAD)) between the adjacent template regions of the current block and the adjacent template regions of the reference block.

[0177] For example, the sample within the template region used in (1) and (2) to induce and compare error values ​​may be the same sample. On the other hand, for example, if the reordered candidate list includes a reference block candidate that combines with an intra-mode, the signaling of the intra-fusion flag may be omitted. If the signaling of the intra-fusion flag is omitted, the block candidate with the smallest error value may be ultimately selected from the reordered list of reference block candidates, which are sorted in ascending order of error value. Alternatively, for another example, the block candidate selected by the candidate index may be selected. Here, "block candidate" can be used to mean all reference block candidates that combine a reference block candidate with an intra-mode or all reference block candidates that do not combine with an intra-mode.

[0178] Subsequently, a reference block candidate may be determined in the reference block list based on the candidate index (S1006). On the other hand, a reference block candidate may be obtained based on the candidate index signaled from the encoder to the decoder. For example, if Figure 10 is performed on the encoder side, step S1006 may include a step to determine the reference block candidate and the reference block candidate index, and the reference block candidate index may be encoded in a bitstream and signaled. For example, if Figure 10 is performed on the decoder side, step S1006 may include a step to obtain the candidate index (Get candidate index). On the decoder side, a reference block candidate may be selected from among N reference block candidates based on the signaled information. As an example, information for selecting a reference block candidate may be signaled to select one of the top K candidates out of N candidates with small error values ​​induced by the SAD substrate. Here, as an example, N may be 8 and K may be 4 (K is an integer, K <= N). On the other hand, as an example, this step may be omitted, in which case the reference block with the smallest SAD may be selected through IntraTMP template search. Alternatively, if the signaling of candidate information is omitted at this stage, the block with the smallest error (e.g., guided by SAD) may be selected by the rearranged order of the rearranged reference block candidate list based on the mathematical formula 4.

[0179] Subsequently, a fractional search may be performed (S1007). The fractional search stage may be performed based on whether or not a sub-pixel search is performed. Whether or not a sub-pixel search is performed may be determined by the encoder and signaled to the decoder, and the decoder can perform the current stage based on the information signaled from the encoder. Alternatively, as another example, the decoder may be guided under specific conditions by predefined rules between the encoder and decoder, which are not explicitly signaled information, or it may be guided based on other information. For example, if a sub-pixel search is performed, based on the reference block selected in the S1006 stage, a half-pixel search may be additionally performed around the reference block as shown in Figure 13 to improve the block vector position to the position with the lowest error (N is an integer). Figure 13 is a diagram illustrating a half-pixel search method that may be applied to one embodiment of the present disclosure. On the other hand, the half-pixel search is one example related to the present disclosure, and the sub-pixel positions may be 1 / 4, 1 / 2, or 3 / 4. Furthermore, information regarding the direction in which to proceed with subpixel search may be explicitly signaled to the bitstream. For example, in the figure below, one of the following pieces of information can be shown relative to the best integer pixel position: left-top, top, top-right, left, right, left-bottom, bottom, bottom-right, etc. On the other hand, as another example, information regarding the direction in which to proceed with subpixel search can be guided without explicit signaling.For example, using the best integer pixel position as a reference in Figure 13, the error between the adjacent template region of the current block and the adjacent template region of the reference block can be calculated for the top-left, top, top-right, left, right, bottom-left, bottom, and bottom-right positions, and the position with the smallest error can be guided without any separately signaled information. On the other hand, when the decoder performs the guidance of the corresponding position without information, there may be predefined rules on the encoder and decoder sides, and these rules may be based on whether or not specific conditions are satisfied, and the guidance of the corresponding position can also be guided based on other information. As an example, possible error calculation methods include SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of transformed difference).

[0180] Thereafter, TMP filtering (Filtered TMP) (S1008) may be performed. TMP filtering may include a process of applying a filter based on filter coefficients trained to minimize the Mean Squared Error (MSE) between the current template area and the reference template area, as shown in Figure 14. Figure 14 is a diagram illustrating a TMP filtering process that may be applied to one embodiment of this disclosure. As an example, the application of the filter may be based on the following mathematical formula 7. Whether or not to apply the filter may be determined by the encoder and signaled to the decoder side via the bitstream, or it may be guided on the decoder side without separate information signaling by a predefined rule between the encoder and decoder, or it may be performed only under specific conditions or guided based on other information.

[0181] [Mathematical formula 7]

number

[0182] In the aforementioned mathematical formula 7, Output i c0 can be the i-th sample of the current block, c0, c1, c2, c3, c4, c5 can be filter coefficients for TMP filtering, and c is the Output within the reference block. i It can be a sample at the same position as C, N can be an upper sample adjacent to C, and S can be a lower sample adjacent to C. Also, W can be a left sample adjacent to C, E can be a right sample adjacent to C, and B can be a bias term, which can have a value of, for example, (1 << (bit depth - 1)).

[0183] Subsequently, the presence or absence of intrafusion can be determined (S1009). As an example, if the example in Figure 10 is performed on the encoder side, it may include a step to determine the presence or absence of intrafusion. The presence or absence of intrafusion may be encoded and signaled in the bitstream, or it may not be signaled and may be induced to a specific value under predefined conditions on the decoder side. As an example, if the example in Figure 10 is performed on the decoder side, it determines the presence or absence of intrafusion, but it may also include a step to determine the presence or absence of intrafusion based on information signaled from the bitstream.

[0184] On the other hand, if the value of the intra fusion flag is a specific value (e.g., 1, true), intraTMP fusion (S1010) may be performed. In this stage, the predicted block predicted in intra mode and the IntraTMP reference block may be combined. For example, the IntraTMP reference block may mean the reference block of the block vector position with the smallest template error (e.g., template cost). Or it may mean the block with the smallest template error (e.g., template cost) reordered in the candidates reordering stage (S1005). Or it may mean the reference block of the block vector position pointed to by the candidate index. As mentioned above, this may be information signaled from the bitstream. Or it may mean a block to which a filter has been applied by the Filtered TMP stage (S1008). Alternatively, it may mean a reference block of a block vector position that has been refined by the Fractional search stage (S1007). Or it may mean a reference block selected by all or some of the above combinations. For example, it may mean a reference block to which both the Filtered TMP (S1008) and Fractional search (S1007) processes have been applied.

[0185] In one embodiment, the final prediction block can be generated as shown in the following mathematical formula.

[0186] [Mathematical formula 8]

number

[0187] For example, Pred final This could be the final prediction block, PredTMP This could be an IntraTMP reference block, and Pred Intra s can be a prediction block generated in the best prediction mode induced by TIMD or DIMD, s can be an integer, w0 and w1 are real numbers, and w0 + w1 = 2 s This is possible. For example, w0 and w1 in the above mathematical formula 8 can be derived based on the template error. For example, the template error between a restored sample in a template area adjacent to an IntraTMP reference block and a restored sample in an adjacent template area of ​​the current block can be defined as TmpCost. Furthermore, a predicted sample can be generated in an adjacent template area of ​​the current block using the optimal mode induced by TIMD or DIMD, and the template error with a restored sample in an adjacent template area of ​​the current block can be determined. This can be called IntraCost. Therefore, the weighted values ​​w0 and w1 can be derived in the following proportions.

[0188] JPEG2026512097000012.jpg16138

[0189] TmpCost and IntraCost can be calculated using SAD, SATD, SSE, MRSAD, MRSSE, MRSATD, etc. On the other hand, as an example, the weighted value derivation can be performed when the following conditions are satisfied.

[0190] [Mathematical formula 9]

number

[0191] On the other hand, as an example, if the above conditions are not met, a fixed weight can be used as shown below.

[0192] [Mathematical formula 10]

number

[0193] For example, in the above mathematical formula 10, k could be, for example, 13. In yet another embodiment, the IntraTMP reference block may be far away from the current block, and a process to compensate for the error may be necessary to take this into consideration. For example, a real number α can be multiplied by TmpCost, and the error-based weighted value can be derived using α × TmpCost and IntraCost. α could be, for example, 0.186. This is just one example, and different α values ​​are possible. For example, the weighted values ​​w0 and w1 can be derived in the following ratio.

[0194] JPEG2026512097000015.jpg16154

[0195] For example, the alpha value applied to TmpCost can currently vary depending on the block size (number of pixels). For instance, the alpha values ​​could be 0.228 (16 pixels), 0.211 (32 pixels), 0.194 (64 pixels), 0.174 (128 pixels), 0.154 (256 pixels), 0.145 (512 pixels), and 0.135 (1024 pixels). This is just one example, and different alpha values ​​are possible. In this case, the weights w0 and w1 can be derived in the following ratios.

[0196] JPEG2026512097000016.jpg16154

[0197] For example, the α value applied to TmpCost can vary depending on qp. For instance, if qp is 22 and 27, α might be 0.196; if qp is 32 and 37, α might be 0.176. This is just one example, and different α values ​​are possible. Here, the weights w0 and w1 can be derived in the following ratios.

[0198] JPEG2026512097000017.jpg16154

[0199] For example, the α value applied to TmpCost can vary depending on the template configuration of IntraTMP (e.g., template position). For instance, if the configuration is L-shaped, α could be 0.186, while if the template is located at the top or left side, α could be 0.13. This is just one example, and different α values ​​are possible. For example, the weights w0 and w1 can be derived in the following ratios.

[0200] JPEG2026512097000018.jpg16154

[0201] In yet another embodiment, the weights (e.g., w0, w1) applied to the mathematical formula may differ depending on the current size of the block. For example, when the width and height of the block are w and h, respectively, the weights can be applied based on the following mathematical formula 11.

[0202] [Mathematical formula 11]

number

[0203] On the other hand, mathematical formula 11 is just one example for determining weight values, and different weight values ​​are possible. Also, while mathematical formula 11 has shown the product of the width and height of the current block as a condition for determining weight values, the condition for determining weight values ​​may also be expressed as a range of the width or height of the current block (for example, the width or height of the current block is less than or greater than a specific value). Or it may be applied only to some of the current blocks. For example, when the width and height of the current block are w and h, respectively, different weight values ​​can be applied if w*h is greater than or equal to 256. For example, weight values ​​may be derived as in mathematical formula 12.

[0204] [Mathematical formula 12]

number

[0205] On the other hand, mathematical formula 12 is just one example for determining weight values, and different weight values ​​are possible. Also, while mathematical formula 12 has shown the product of the current block's width and height as a condition for determining weight values, the condition for determining weight values ​​may also be expressed as a range of the current block's width or height (for example, the current block's width or height being less than / greater than a specific value).

[0206] On the other hand, if it is decided that intrafusion will not be applied (for example, if the value of the intrafusion flag is false on the decoder side), only intraTMP (S1011) may be performed. That is, only the intraTMP-based prediction block may be induced.

[0207] On the other hand, if it is decided in step S1004 that filtered fusion will be applied, then it may be decided (S1012) whether intraTMP-based filtering will be applied (Filter based IntraTMP fusion). Step S1012 may be performed when the filtered fusion flag is a specific value (e.g., true, 1), and may relate to the example in Figure 15. Figure 15 is a diagram illustrating a plurality of reference template areas and a current template area according to one embodiment of the present disclosure. According to the example illustrated in Figure 15, a filter may be applied as shown in mathematical formula 13, based on filter coefficients trained to minimize the Mean Squared Error (MSE) between the current template area and the plurality of reference template areas. The number of reference blocks to be fused may also be explicitly signaled in the bitstream, guided by predefined rules between the encoder and decoder, or guided based on other signaled information.

[0208] [Mathematical formula 13]

number

[0209] For example, in mathematical formula 13, Output i This is the i-th sample of the current block, c0, c1, c2, ..., c n-1 , c n is the filter coefficient, and S1 is the Output in the first reference block. i This is a sample located in the same position (collocated), and S2 is an Output within the second reference block. i This is a sample in the same location, and S3 is Output within the third reference block. i This is a sample at the same location as S n-1 This is the Output in the (n-1)th reference block. i This could mean a sample at the same position. Also, variable B is a bias term and can have values ​​such as (1 << (bit depth - 1)).

[0210] Example 2

[0211] This embodiment proposes a method for generating new prediction blocks by combining prediction blocks generated by IntraTMP with prediction blocks generated by intra-modes induced by DIMD or TIMD. The proposed method can further enhance intra-prediction performance by combining multiple induced intra-modes with IntraTMP reference blocks. More specifically, as can be seen in the DIMD and TIMD techniques, prediction blocks formed by combining prediction blocks can lead to improved intra-prediction performance. Therefore, prediction blocks formed by combining various prediction blocks using DIMD or TIMD can combine with prediction blocks generated by IntraTMP techniques to lead to improved intra-prediction performance. As an example, an IntraTMP reference block may mean the reference block of the block vector position with the smallest template cost. Or it may mean the block with the smallest template error after being reordered in the candidate reordering stage of Example 1. Or it may mean the reference block of the block vector position pointed to by the candidate index of Example 1. The candidate index can be known through signaling. Or it may mean the block to which a filter has been applied by the filtering TMP stage of Example 1. Alternatively, it may mean a reference block of an improved block vector position by the fractional position search step of Example 1. Or it may mean a reference block selected by all or some of the above combinations. For example, it may mean a reference block to which filtering TMP and fractional position search have all been applied. In one embodiment, the final predicted block can be generated as shown in the following mathematical formula 14.

[0212] [Mathematical formula 14]

number

[0213] For example, Pred final This could be the final prediction block, Pred TMP This could be an IntraTMP reference block, and Pred Intras can be a prediction block coupled with multiple intra-modes induced by TIMD or DIMD. Also, s can be an integer, w0 and w1 are real numbers, and w0 + w1 = 2 s It is possible.

[0214] In one embodiment, the Pred of the mathematical formula 14 Intra This could be a prediction block generated by blending multiple intramodes induced by TIMD. For example, when two intramodes with the smallest error values ​​(e.g., SATD) in TIMD modes are selected, and the error values ​​(e.g., SATD) of the two intramodes are denoted as costMode1 and costMode2, if the conditions expressed in the following mathematical formula 15 are satisfied, the prediction blocks for the two intramodes are blended and Pred Intra This can be generated, and the weight values ​​that can be applied to each intra prediction block may be the same as those in mathematical formula 16.

[0215] [Mathematical formula 15]

number

[0216] [Mathematical formula 16]

number

[0217] Furthermore, the prediction block generated by blending multiple intramodes induced by TIMD is the template in mathematical formula 4 of the candidate re-alignment stage in Example 1. Intra This could mean that, in one embodiment, the Pred of mathematical formula 14 IntraThis can be a prediction block generated by blending multiple intra-modes induced by DIMD. For example, K intra-prediction modes with large histogram amplitudes may be selected by DIMD, and the prediction blocks generated based on these modes can be blended to form the final prediction block. As an example, K is an integer greater than or equal to 1, and the value of K may be determined by an agreement between the encoder / decoder. For example, K may be 5. For example, if K is 5, and five intra-modes with large histogram amplitudes are selected, with amplitude values ​​for each mode being costMode1, costMode2, costMode3, costMode4, and costMode5, then six prediction blocks including the planar mode can be blended. As an example, the blending weight ratio may be as follows. Weight 1, Weight 2, Weight 3, Weight 4, and Weight 5 can correspond to the respective weights for the induced 1st intra-prediction mode, 2nd intra-prediction mode, 3rd intra-prediction mode, 4th intra-prediction mode, and 5th intra-prediction mode, i.e., the 1st weight, 2nd weight, 3rd weight, 4th weight, and 5th weight, respectively. On the other hand, the sum of the ratios of each weight is Pred IntraThis results in a 3 / 4 ratio, which could mean that a fixed weight ratio of 1 / 4 is applied to the weight of the planner mode (weightPlanar). Additionally, the first weight value weight1 for the first prediction block based on each first intra-prediction mode can be derived based on the error value (e.g., amplitude value) of the first intra-prediction mode. For example, the first weight value can be derived based on costMode1 / (costMode1+costMode2+costMode3+costMode4+costMode5). Similarly, the second weight value weight2 can be derived based on the error value (e.g., amplitude value) of the second intra-prediction mode. The second weight value weight2 can be derived based on costMode2 / (costMode1+costMode2+costMode3+costMode4+costMode5). The third weight value weight3 can be derived based on the error value (e.g., amplitude value) of the third intra-prediction mode. The third weight value, weight3, can be derived based on costMode3 / (costMode1+costMode2+costMode3+costMode4+costMode5). The fourth weight value, weight4, can be derived based on the error value (e.g., amplitude value) of the fourth intra-prediction mode. The fourth weight value, weight4, can be derived based on costMode4 / (costMode1+costMode2+costMode3+costMode4+costMode5). The fifth weight value, weight5, can be derived based on the error value (e.g., amplitude value) of the fifth intra-prediction mode. The fifth weight value, weight5, can be derived based on costMode5 / (costMode1+costMode2+costMode3+costMode4+costMode5). On the other hand, as another example, if K is 5, five intra-modes with large histogram amplitudes are selected, and the amplitude values ​​for each mode may be costMode1, costMode2, costMode3, costMode4, and costMode5, and the weights (weight1, weight2, weight3, weight4, and weight5) when blending the five prediction blocks excluding the planner mode can be derived in the same way as described above.On the other hand, the prediction block generated by blending multiple intramodes induced by DIMD is the template for mathematical formula 4 in the candidate re-sorting stage of Example 1. Intra This could mean...

[0218] Example 3

[0219] This embodiment proposes a method for generating a new prediction block by combining a prediction block generated by IntraTMP with a prediction block generated in intra-mode induced by DIMD or TIMD. In this embodiment, intra-prediction performance can be improved by inducing intra-mode to be identical to the template shape of IntraTMP. For example, if the template region of IntraTMP is L-shaped, it may be effective to induce intra-mode using all the previously restored samples on the left and upper sides of the current block. Similarly, if the template region of IntraTMP is located at the top or on the left side, it may be effective to induce intra-mode using the previously restored samples on the upper side and the previously restored samples on the left side of the current block, respectively. Accordingly, according to one embodiment of this disclosure, the final prediction block can be generated as shown in the following mathematical formula 17.

[0220] [Mathematical formula 17]

number

[0221] As one example, Pred final This can mean the final prediction block, Pred TMP This can mean an IntraTMP reference block, Pred Intra This could mean the predicted block predicted in intra-mode using the top edge of the current block, or the left side, or both the top and left sides. Also, s can be an integer, w0 and w1 are real numbers, and w0 + w1 = 2 sThis is possible. On the other hand, here, the IntraTMP reference block may mean the reference block of the block vector position having the smallest template error. Or it may mean the reference block of the block vector position pointed to by the candidate index described in Example 1. Here, the candidate index may be explicitly signaled. Or, in another embodiment, the intraTMP reference block may mean the block having the smallest template error after being realigned in the candidate realignment stage of Example 1. Or it may mean the block to which a filter has been applied by the filtering TMP stage of Example 1. Or it may mean the reference block of the block vector position improved by the fractional position search stage of Example 1. Or it may mean the reference block selected by all or some combination of the above. For example, it may mean the reference block to which both filtering TMP and fractional position search have been applied.

[0222] Figure 16 is a diagram illustrating the case where the template area of ​​intraTMP according to one embodiment of the present disclosure is located at the top. As an example, when the template area of ​​IntraTMP is located at the top, as shown in Figure 16, the intra mode is induced using only the already restored sample at the top of the current block, and this is used to Pred Intra This can generate an intra-mode. For example, the intra-mode can be induced based on the DIMD or TIMD mode. The induced intra-mode is the template of mathematical formula 4 in the candidate re-sorting stage in Example 1. Intra This could mean...

[0223] Figure 17 is a diagram illustrating the case where the template area of ​​intraTMP according to one embodiment of the present disclosure is located on the left side. As an example, when the template area of ​​IntraTMP is located on the left side, as shown in Figure 17, intra mode is induced using only the previously restored samples on the left side of the current block, and this is used for Pred IntraThis can generate the intra-mode, which can be derived based on the DIMD or TIMD mode, and the derived intra-mode is the template of mathematical formula 4 in the candidate re-sorting stage in Example 1. Intra This could mean...

[0224] Example 4

[0225] This embodiment proposes a signaling method for information related to IntraTMP. Effective signaling is necessary considering various techniques such as the template form of IntraTMP, the presence or absence of fractional position search, and filtering TMP. This disclosure provides an effective signaling method for intraTMP-related information, thereby improving compression performance.

[0226] Table 3 below illustrates the signaling syntax of intraTMP-related information according to one embodiment of this disclosure.

[0227] [Table 3]

[0228] For example, information regarding the application of intraTMP may be signaled. For example, information regarding the application of intraTMP may be a tmpFlag (e.g., first syntax), where a first value (e.g., 1) indicates that intraTMP is applied, and a second value (e.g., 0) indicates that intraTMP is not applied. For example, information regarding the presence or absence of fusion between intraTMP reference blocks may be signaled. For example, information regarding the presence or absence of fusion between intraTMP reference blocks may be a fusionFlag (e.g., second syntax), where a first value (e.g., 1) indicates that fusion between intraTMP reference blocks is applied, and a second value (e.g., 0) indicates that fusion between intraTMP reference blocks is not applied. For example, the second syntax may be signaled based on the value of the first syntax. For example, the second syntax may be signaled only when the value of the first syntax is a specific value (e.g., 1).

[0229] For example, information regarding whether or not filtering is applied can be signaled. For example, this information may be an ftmpFlag (e.g., a third syntax), which, if it has a first value (e.g., 1), can indicate that filtering is applied, and if it has a second value (e.g., 0), can indicate that filtering is not applied. For example, the third syntax may be signaled based on the value of the second syntax. For example, the third syntax may be signaled only when the value of the second syntax is a specific value (e.g., 0). For example, if the value of the third syntax is a specific value (e.g., true), the filtering TMP stage of Example 1 can be performed.

[0230] For example, information regarding whether or not a subpixel search is performed can be signaled. For example, this information may be tmpIsSubPel (e.g., the fourth syntax), which, if it has a first value (e.g., 0), can indicate that a subpixel search is not performed, and if it has a value other than the first value (e.g., 1, ..., n), it may include information regarding subpixel searches, such as 1 / 4-pixel, 1 / 2-pixel, 3 / 4-pixel, etc. For example, the fourth syntax may be signaled based on the value of the second syntax. For example, the fourth syntax may be signaled only when the value of the second syntax is a specific value (e.g., 0). For example, if the value of the fourth syntax is a specific value (e.g., a value other than 0), the subpixel search step of Example 1 can be performed.

[0231] For example, when a subpixel search is performed, information regarding the direction in which the subpixel search should be performed may be signaled. For example, this information may be tmpSubPelIdx (e.g., the fifth syntax) and may be signaled based on the value of the fourth syntax. For example, signaling may only occur when the value of the fourth syntax is a specific value (e.g., a value other than 0). For example, the fifth syntax can indicate a direction such as the upper left, top, upper right, left, right, lower left, bottom, lower right, etc., and can indicate one of many directions. Based on the fourth syntax and / or the fifth syntax, the fractional position search stage of Example 1 can be performed.

[0232] For example, information regarding the template form may be signaled. For example, index information of the template form (i.e., template position) may be signaled, and this information may be templateIdx (e.g., the sixth syntax), which may be signaled based on the value of the second syntax. For example, signaling may occur only when the value of the second syntax is a specific value (e.g., 0). For example, since the sixth syntax may indicate the template position, i.e., what the template form is, it may be index information for pointing to one of various template forms such as an L-shaped form, an upper end form, or a left-side form. This may represent the template form determination stage in Example 1.

[0233] For example, information for an index pointing to one of the candidate intraTMP reference blocks may be signaled. For example, this information could point to one of the various reference blocks included in the reference block candidate list, and could be a candidateIdx (e.g., syntax 7), which may be signaled based on the value of syntax 2. For example, signaling may only occur when the value of syntax 2 is a specific value (e.g., 0).

[0234] For example, information regarding the merging of an intraTMP reference block and an intra-prediction block in a specific intra-mode may be signaled. For example, this information may be an intrafusionFlag (e.g., the 8th syntax), where if the value of the 8th syntax is a first value (e.g., 1), the intraTMP fusion step of Example 1 may be performed, and if the value of the 8th syntax is a second value (e.g., 0), the intraTMP reference block and the intra-prediction block may not be merged. Furthermore, the 8th syntax may be signaled based on the value of the second syntax. For example, it may be signaled only when the value of the second syntax is a specific value (e.g., 0).

[0235] As an example, when fusing a plurality of intraTMP reference block candidates, information regarding the number of intraTMP reference blocks to be fused may be signaled. As an example, the relevant information may be fusionIdx (e.g., the ninth syntax), and based on the value of the ninth syntax, the number of reference blocks may be determined. Also, the ninth syntax may be signaled based on the value of the second syntax. For example, it may be signaled only when the value of the second syntax is a specific value (e.g., 1). On the other hand, for example, when the value of the ninth syntax is 3, it may mean fusing three candidate intraTMP reference blocks in ascending order of the aforementioned error (e.g., cost, expense, or amplitude value, etc.).

[0236] On the other hand, the signaling order of the first syntax to the ninth syntax may be determined as in Table 3 above. However, since this corresponds to one embodiment of the present disclosure, the signaling order of the syntax may be changed. As an example, the order between the third syntax and the eighth syntax may be changed. For example, it is also possible that the third syntax is signaled first, and after the fourth syntax, the fifth syntax, the sixth syntax, and the seventh syntax are signaled in order, the eighth syntax is signaled last. As yet another example, the sixth syntax may be signaled first, and after the seventh syntax, the eighth syntax, the third syntax, and the fourth syntax are signaled in order, the fifth syntax may be signaled last. Or as yet another example, the sixth syntax may be signaled first, and after the seventh syntax, the fourth syntax, the fifth syntax, and the third syntax are signaled in order, the eighth syntax may be signaled last. However, all of the aforementioned orders are merely examples of the present disclosure and may be changed to other possible combinations, which is self-evidently also included in the present disclosure.

[0237] As an example, the signaling order of intraTMP-related information may be the same as that in Table 4 below. As an example, the signaling order of the 6th syntax and the 3rd syntax may be changed. templateIdx can be signaled after fusionFlag, and the following signaling order is a structure that can perform fusion between intraTMP reference blocks for the upper and left forms as well.

[0238]

Table 4

[0239] In this case, the 6th syntax may be signaled before the 3rd syntax and may be obtained based only on the value of the 1st syntax instead of the 2nd syntax. For example, in Table 4, the 6th syntax can be obtained only when the value of the 1st syntax is a specific value (e.g., 1). On the other hand, in the example of the said Table 4, the signaling order may also be changed. As an example, in the signaling order among the 6th tax, the 3rd syntax, the 4th syntax, the 5th syntax, the 7th syntax, and the 8th syntax, the 5th syntax may be signaled first, followed by the 7th syntax, the 8th syntax, the 6th syntax, the 3rd syntax, and finally the 4th syntax. Or, as another example, the 5th syntax may be signaled first, followed by the 7th syntax, the 3rd syntax, the 4th syntax, the 6th syntax, and finally the 8th syntax. However, this is also an embodiment according to the present disclosure and may be changed to other possible combinations.

[0240] On the other hand, since the descriptions of the 1st syntax to the 9th syntax are the same as those described above, redundant descriptions are omitted.

[0241] As another example, some information may not be signaled. For instance, the signaling of the sixth syntax, templateIdx, may be omitted, as shown below. In this case, information about the template form, such as templateIdx information, may be induced by an agreement between the encoder / decoder or induced to a specific value under certain conditions, without signaling.

[0242] [Table 5]

[0243] On the other hand, even in the example in Table 5, the signaling order can be changed. In the signaling order between the third to fifth syntaxes, the seventh syntax, and the eighth syntax, the seventh syntax may be signaled first, followed by the eighth syntax, the third syntax, and the fourth syntax, and then the fifth syntax may be signaled last. On the other hand, as another example, the seventh syntax may be signaled first, followed by the eighth syntax, the fourth syntax, and the fifth syntax, and then the third syntax may be signaled last. However, this is also just one embodiment of the present disclosure and may be changed to other possible combinations. On the other hand, as in Table 5, the sixth syntax may not be signaled, but it is also possible that one or more syntaxes among the third to eighth syntaxes may not be signaled, and this is also included in the present disclosure.

[0244] Example 5

[0245] IntraTMP can include a variety of techniques to improve coding performance, such as subpixel search, fusion, and filtering. This embodiment defines various IntraTMP techniques and proposes a method for managing them in list form, enabling effective application and signaling of IntraTMP techniques to improve coding performance.

[0246] As an example, the mode for generating prediction blocks in IntraTMP can be defined as follows:

[0247] (1) Default mode:

[0248] - A mode that makes the reference block with the smallest SAD error (cost) after IntraTMP search the prediction block.

[0249] - Alternatively, after IntraTMP search, a mode is used to predict reference blocks selected by index signaling from a list of reference block candidates sorted in descending order of SAD error.

[0250] In addition to SAD, other error calculation methods such as SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of transformed difference) can be used.

[0251] (2) Filtering mode:

[0252] - A mode in which the Filtered TMP (Filtered TMP) stage of Example 1 is applied to the prediction block of (1) to generate a prediction block.

[0253] (3) Sub-pixel search mode:

[0254] - A mode in which the Fractional Search step of Example 1 is applied to the prediction block of (1) to generate a prediction block.

[0255] (4) Intra fusion mode:

[0256] - Apply the IntraTMP fusion stage of Example 1 to the prediction block of (1) to generate a prediction block combined with the intra mode

[0257] The intra mode can be induced based on TIMD or DIMD.

[0258] (5) Cost-based fusion mode:

[0259] - After IntraTMP search, combine the top n reference block candidates with low errors in the reference block candidate list sorted in ascending order of errors (e.g., SAD) with cost-based weighting values to generate a prediction block (n is a real number)

[0260] The weighting values for combination can be induced based on cost, for example, when the error calculation method is SAD, the weighting value ratio can be as follows.

[0261] JPEG2026512097000029.jpg3459

[0262] In addition to SAD, error calculation methods such as SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), MR-SATD (Mean-removed sum of transformed difference) can be used.

[0263] (6) Fixed-weight fusion mode:

[0264] - After the IntraTMP search, a mode is used to generate a predicted block by combining the top n reference blocks with the lowest SAD errors (e.g., SAD) from a list of reference block candidates sorted in descending order of error (e.g., SAD) using a fixed weighting (n is a real number).

[0265] For example, if n=2, then w1=1 / 2 and w2=1 / 2 are possible.

[0266] For example, if n=3, then w1=22 / 64, w2=21 / 64, and w3=21 / 64 are possible.

[0267] In addition to SAD, other error calculation methods such as SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of transformed difference) can be used.

[0268] (7) Filter-based fusion mode:

[0269] - After IntraTMP search, a list of candidate reference blocks sorted in descending order of error (e.g., SAD) is used to generate a combined predicted block by applying the Filter-based IntraTMP fusion method of Example 1 to the top n reference blocks with the lowest errors (n is a real number).

[0270] In addition to SAD, other error calculation methods such as SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of transformed difference) can be used.

[0271] For example, the defined modes may be included in the candidate list, and information (e.g., an index) for the list may be signaled. On the other hand, for example, the method of constructing the candidate list may be defined by an agreement between the encoder and decoder. The size of the candidate list may also be defined by an agreement between the encoder and decoder. Furthermore, one mode may be signaled within the constructed candidate list.

[0272] For example, a candidate list may be constructed for the defined modes (1) to (7) as shown in Table 6 below. In this case, specific information (e.g., an index) may be signaled to select a single candidate, thereby providing information on the mode to be used in IntraTMP. On the other hand, since Table 6 below also represents one embodiment of the present disclosure, the method of constructing the candidate list may be changed. For example, the list may be constructed sequentially from the modes with the highest selectivity.

[0273] [Table 6]

[0274] Table 6 above was created to include modes (1) through (8), but some modes may be omitted, and mode (8) may be omitted. That is, the candidate list may include only modes (1) through (7). For example, mode (8) in Table 6 above may be a new mode induced by a combination of two or more modes from the defined modes (1) through (7). For example, one mode may be induced based on the combination of mode (2) and mode (3). In another example, one mode may be induced based on the combination of mode (3) and mode (4). In yet another example, one mode may be induced based on the combination of mode (3) and mode (5). And in yet another example, one mode may be induced based on the combination of mode (3) + mode (4) + mode (7). However, this is just an example, and in order to constitute mode (8), all possible combinations are possible by utilizing one or more modes from modes (1) through (7), and there is no limit to the number of modes that can be combined.

[0275] As another example, the defined modes (1) to (7) above may be included in the candidate list as single modes, or the candidate list may be composed of new modes combined based on modes (1) to (7). This will be explained with reference to Table 7.

[0276] [Table 7]

[0277] Referring to Table 7, information for the candidate list (e.g., an index for selecting a candidate) may be signaled. In this case, the method of constructing the candidate list may be defined by an agreement between the encoder and decoder. The size of the candidate list may also be defined by an agreement between the encoder and decoder. Furthermore, information for identifying a single mode in the constructed candidate list may be signaled. In addition, the mode used for IntraTMP by the decoder may be determined through the information for the candidate list. On the other hand, as an example, the method of constructing the candidate list may be changed, for example, by constructing it sequentially from modes with high selectivity.

[0278] Example 6

[0279] IntraTMP can include a variety of techniques to improve coding performance, such as subpixel search and fusion filtering. In this example, we define various techniques for IntraTMP and propose combinations for generating prediction blocks. In this example, intra-predictive performance can be improved by effectively generating prediction blocks with IntraTMP.

[0280] The modes for generating prediction blocks in IntraTMP may include modes (1) to (7), as described in Example 5. In one embodiment, each prediction block generated in the defined modes (1) to (7) may become the final prediction block. Mode selection may be performed through signaling or guided by an agreement between encoders and decoders. On the other hand, as an example, a prediction block based on one of the modes (1) to (7) may become the final prediction block of IntraTMP. As another example, a new prediction block can be generated by combining two or more prediction blocks based on the modes defined in Example 5, and the generated prediction block may become the final prediction block. For example, a prediction block based on the combination of modes (2) and (3) may become the final prediction block of IntraTMP. As yet another example, a prediction block based on the combination of modes (2) and (4) may become the final prediction block of IntraTMP. As yet another example, a prediction block based on the combination of modes (3) and (4) may become the final prediction block of IntraTMP. As another example, a prediction block based on a combination of (3) mode + (5) mode can become the final prediction block of IntraTMP. As yet another example, a prediction block based on a combination of (3) mode + (6) mode can become the final prediction block of IntraTMP. As yet another example, a prediction block based on a combination of (3) mode + (7) mode can become the final prediction block of IntraTMP. As yet another example, a prediction block based on a combination of (4) mode + (5) mode can become the final prediction block of IntraTMP. As yet another example, a prediction block based on a combination of (4) mode + (6) mode can become the final prediction block of IntraTMP. As yet another example, a prediction block based on a combination of (4) mode + (7) mode can become the final prediction block of IntraTMP. As yet another example, a prediction block based on a combination of (4) mode + (7) mode can become the final prediction block of IntraTMP.As another example, a prediction block based on a combination of (2) mode + (3) mode + (4) mode can become the final prediction block of IntraTMP. In addition, a final prediction block can be generated based on any combination of possible modes.

[0281] In yet another embodiment, a new prediction block may be generated for defined modes (1) to (7) based on the combination of different template regions, and the generated prediction block may become the final prediction block. Mode information (or combination information) may be signaled, or it may be induced by an agreement between the encoder / decoder.

[0282] For example, prediction blocks selected for L-shape search in mode (1), prediction blocks selected for above-shape search in mode (1), prediction blocks selected for left-shape search in mode (1), prediction blocks selected for L-shape search in mode (2), prediction blocks selected for above-shape search in mode (2), prediction blocks selected for left-shape search in mode (2), prediction blocks selected for L-shape search in mode (3), prediction blocks selected for above-shape search in mode (3), and prediction blocks selected for left-shape search in mode (3) may be induced. In this case, the induced prediction blocks may be combined. For example, the prediction block selected for L-shape search in mode (1) and the prediction block selected for above-shape search in mode (1) may be combined. Also, the prediction block selected for L-shape search in mode (1) and the prediction block selected for above-shape search in mode (3) may be combined. This is just one example, and a final prediction block can be generated for all possible combinations.

[0283] Examples 1 to 6 described above are numbered for clarity of explanation, and each example can be applied individually or in combination. Furthermore, the names of the first to ninth syntaxes are for clarity of explanation and are not intended to limit the use of names in this disclosure.

[0284] Examples of video encoding and decoding methods

[0285] Figure 18 is a diagram illustrating a video decoding method that can be performed by a video decoding device or a video encoding method that can be performed by a video encoding device according to one embodiment of the present disclosure.

[0286] Figure 18 can be based on Examples 1 to 6 of this disclosure.

[0287] First, an example of a video decoding method will be explained with reference to Figure 18.

[0288] As an example, the video decoding method shown in Figure 18 can be performed by the video decoding device described above with reference to other drawings.

[0289] First, in the video decoding method, a first prediction block may be generated based on intra-template matching prediction (S1810). For example, the intra-template matching prediction may be intraTMP, and the first prediction block may be the intraTMP reference block described above, with respect to intraTMP as described above. Thereafter, a second prediction block may be generated based on a specific intra-prediction mode (S1820), and a final prediction block may be generated based on the first prediction block and the second prediction block (S1830). Here, for example, the specific intra-prediction mode may be an intra-prediction mode induced by TIMD or DIMD. For example, the second prediction block may be a plurality of prediction blocks generated based on a plurality of intra-prediction modes induced by TIMD or DIMD. For example, the specific intra-prediction mode may be induced based on the template form of the intra-template matching prediction. For example, the template form is obtained from the bitstream, but may be induced to a specific value under an agreement between encoder / decoder or under specific conditions. For example, the first prediction block may be generated based on a prediction mode selected from the intra-template matching prediction mode list. For example, the selected prediction mode may be chosen based on prediction mode index information. This index information may be signaled in a bitstream. For example, the intra-template matching prediction mode list may include at least one of the following: default mode, filtering mode, subpixel search mode, intra-fusion mode, error-based fusion mode, or filter-based fusion mode. For example, the intra-template matching prediction mode list may include prediction modes generated by at least one combination of the following: default mode, filtering mode, subpixel search mode, intra-fusion mode, error-based fusion mode, or filter-based fusion mode.For example, the error-based fusion mode may be based on errors calculated using SAD (Sum of difference), SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), or MR-SATD (Mean-removed sum of transformed difference). For example, the first prediction block may be a prediction block formed by combining multiple intra-template matching prediction blocks.

[0290] According to this disclosure, when encoding still images or videos on a screen, by inducing reference blocks and intra modes in IntraTMP mode and generating the final predicted block through various predicted block combination methods, it is expected that prediction performance will be improved, and consequently, coding efficiency will be improved.

[0291] First, an example of a video encoding method will be explained with reference to Figure 18.

[0292] As an example, the video encoding method shown in Figure 18 can be performed by a video encoding device described above with reference to other drawings.

[0293] First, in the video decoding method, a first prediction block may be generated (S1810) based on intra-template matching prediction. For example, the intra-template matching prediction may be intraTMP, and the first prediction block may be the intraTMP reference block described above, with respect to intraTMP as described above. Thereafter, a second prediction block may be generated (S1820) based on a specific intra-prediction mode, and a final prediction block may be generated (S1830) based on the first prediction block and the second prediction block. Here, for example, the specific intra-prediction mode may be an intra-prediction mode induced by TIMD or DIMD. For example, the second prediction block may be a plurality of prediction blocks generated based on a plurality of intra-prediction modes induced by TIMD or DIMD. For example, the specific intra-prediction mode may be induced based on the template form of the intra-template matching prediction. For example, the template form may be signaled by a bitstream, but it may be induced to a specific value under an agreement between encoder / decoder or under specific conditions. For example, the first prediction block may be generated based on a prediction mode selected from the intra-template matching prediction mode list. For example, the selected prediction mode may be chosen based on prediction mode index information. This index information may be signaled in a bitstream. For example, the intra-template matching prediction mode list may include at least one of the following: default mode, filtering mode, subpixel search mode, intra-fusion mode, error-based fusion mode, or filter-based fusion mode. For example, the intra-template matching prediction mode list may include prediction modes generated by at least one combination of the following: default mode, filtering mode, subpixel search mode, intra-fusion mode, error-based fusion mode, or filter-based fusion mode.For example, the error-based fusion mode may be based on errors calculated using SAD (Sum of difference), SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), or MR-SATD (Mean-removed sum of transformed difference). For example, the first prediction block may be a prediction block formed by combining multiple intra-template matching prediction blocks.

[0294] On the other hand, although not shown in the diagrams, in the video encoding method performed on the video encoding side, information for intra-template matching prediction is first encoded, and information for prediction block fusion may be encoded. On the encoding side, decisions can be made regarding intra-template matching prediction (e.g., whether or not intra-template matching prediction is performed) and regarding prediction block fusion (e.g., whether or not prediction block fusion is performed) before encoding the information. As an example, the information for intra-template matching prediction and the information for prediction block fusion may include the information described in Examples 1 to 6 above and the information described with reference to Tables 3 to 7.

[0295] According to this disclosure, when encoding still images or videos within a screen, the prediction performance can be improved by inducing reference blocks and intra modes in IntraTMP mode and generating the final prediction block through various prediction block combination methods, thereby improving coding efficiency.

[0296] Furthermore, as an example, a computer-readable medium recording a bitstream generated by a video encoding method may be provided, and a method for transmitting the bitstream generated by the video encoding method may also be provided.

[0297] On the other hand, since the video decoding and encoding method shown in Figure 18 is one embodiment of the present disclosure, it is obvious that certain steps may be changed, the order of the steps may be changed, or some steps may be added or deleted, and such modifications are also included in the present disclosure.

[0298] According to this disclosure, when encoding still images or videos within a screen, by inducing an IntraTMP mode reference block (i.e., a prediction block) and an Intra mode, and generating a final prediction block through various prediction block combination methods, it is possible to improve prediction performance and, consequently, improve coding efficiency.

[0299] The exemplary methods of this disclosure are presented as a series of actions for clarity of explanation, but this is not intended to restrict the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. To embody the methods relating to this disclosure, additional steps may be included in addition to the exemplary steps, or the remaining steps may be included with some exceptions, or additional steps may be included with some exceptions.

[0300] In this disclosure, a video encoding device or video decoding device that performs a predetermined operation (stage) can perform an operation (stage) to confirm the conditions or status for performing the operation (stage). For example, if it is stated that a predetermined operation is performed when a predetermined condition is satisfied, the video encoding device or video decoding device can perform an operation to confirm whether the predetermined condition is satisfied, and then perform the predetermined operation.

[0301] The various embodiments of this disclosure are not intended to list all possible combinations, but rather to illustrate representative aspects of this disclosure, and the matters described in the various embodiments may apply independently or in combination of two or more.

[0302] Furthermore, various embodiments of this disclosure can be embodied in hardware, firmware, software, or a combination thereof. In the case of hardware embodiment, it can be embodied in one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0303] Furthermore, the video decoding and video encoding devices to which the embodiments of this disclosure are applied may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communications, mobile streaming devices, storage media, video cameras, on-demand video (VoD) service providers, OTT video (Over the top video) devices, internet streaming service providers, 3D video devices, image-phone video devices, and medical video devices, and may be used to process video signals or data signals. For example, OTT video (Over the top video) devices may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0304] Figure 19 illustrates a content streaming system to which the embodiments of this disclosure can be applied.

[0305] As shown in Figure 19, a content streaming system to which an embodiment of the present disclosure is applied may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0306] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and transmits this bitstream to the streaming server. In another example, if a multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server can be omitted.

[0307] The bitstream can be generated by an image encoding method and / or image encoding apparatus to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0308] The streaming server transmits multimedia data to the user's device based on the user's request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server can transmit multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server can play a role in controlling the commands and responses between the devices within the content streaming system.

[0309] The streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0310] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices such as smartwatches, smart glasses, HMDs (head-mounted displays), digital TVs, desktop computers, and digital signage.

[0311] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received from each server can be processed in a distributed manner.

[0312] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) that enable the operation of various embodiments to be performed on a device or computer, and non-transitory computer-readable medium on which such software or commands etc. are stored and can be executed on a device or computer. [Industrial applicability]

[0313] The embodiments described herein can be used for encoding / decoding images.

Claims

1. In video decoding methods, Steps to generate a first prediction block based on intra-template matching predictions, The steps include generating a second prediction block based on a specific intra-prediction mode, and A video decoding method comprising the step of generating a final prediction block based on the first prediction block and the second prediction block.

2. The video decoding method according to claim 1, wherein the specific intra prediction mode is an intra prediction mode induced by TIMD or DIMD.

3. The video decoding method according to claim 1, wherein the second prediction block is a plurality of prediction blocks generated based on a plurality of intra-prediction modes induced by TIMD or DIMD.

4. The video decoding method according to claim 1, wherein the specific intra prediction mode is induced based on the template form of the intra template matching prediction.

5. The video decoding method according to claim 4, wherein the template form is obtained from a bitstream.

6. The video decoding method according to claim 1, wherein the first prediction block is generated based on a prediction mode selected from the intra-template matching prediction mode list.

7. The video decoding method according to claim 6, wherein the selected prediction mode is selected based on prediction mode index information.

8. The video decoding method according to claim 6, wherein the intra-template matching prediction mode list includes at least one of the following: default mode, filtering mode, subpixel search mode, intra-fusion mode, error-based fusion mode, or filter-based fusion mode.

9. The video decoding method according to claim 6, wherein the intra-template matching prediction mode list includes prediction modes generated by a combination of at least one of the following: default mode, filtering mode, subpixel search mode, intra-fusion mode, error-based fusion mode, or filter-based fusion mode.

10. The video decoding method according to claim 8, wherein the error-based fusion mode is based on an error calculated using SAD (Sum of difference), SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), or MR-SATD (Mean-removed sum of transformed difference).

11. The video decoding method according to claim 1, wherein the first prediction block is a prediction block formed by combining a plurality of intra-template matching prediction blocks.

12. In video encoding methods, Steps to generate a first prediction block based on intra-template matching predictions, The steps include generating a second prediction block based on a specific intra-prediction mode, and A video encoding method comprising the step of generating a final prediction block based on the first prediction block and the second prediction block.

13. In a recording medium storing a bitstream generated by a video encoding method, the video encoding method is: Steps to generate a first prediction block based on intra-template matching predictions, The steps include generating a second prediction block based on a specific intra-prediction mode, and A medium comprising the step of generating a final prediction block based on the first prediction block and the second prediction block.

14. In a method for transmitting a bitstream generated by a video encoding method, the video encoding method is: Steps to generate a first prediction block based on intra-template matching predictions, The steps include generating a second prediction block based on a specific intra-prediction mode, and A method comprising the step of generating a final prediction block based on the first prediction block and the second prediction block.

Images