Image encoding / decoding method and device, and recording medium for storing bitstream

Abstract

Provided are an image encoding / decoding method and device, a recording medium for storing a bitstream, and a transmission method. The image decoding method may comprise the steps of: obtaining a transform coefficient from a bitstream; deriving a prediction sample of a current block by performing prediction on the current block; obtaining a residual sample of the current block by performing inverse transform on the transform coefficient; deriving a residual prediction sample of the current block; and generating a reconstructed sample of the current block on the basis of the residual sample, the prediction sample, and the residual prediction sample, wherein the residual prediction sample is derived on the basis of a template of the current block.

Description

Video encoding / decoding method, device, and recording medium storing a bitstream

[0001] The present disclosure relates to an image encoding / decoding method, an apparatus, and a recording medium storing a bitstream. Specifically, the present disclosure relates to an image encoding / decoding method, an apparatus, and a recording medium storing a bitstream based on a second prediction method and a method for determining a transformation kernel in the second prediction method.

[0002] Recently, the demand for high-resolution, high-quality video, such as UHD (Ultra High Definition) video, is increasing across various application fields. Furthermore, interest in and demand for immersive media, including VR (Virtual Reality), AR (Artificial Reality), and holograms, are also on the rise. Additionally, the broadcasting of video with characteristics distinct from reality, such as game footage, is also increasing. As video data becomes higher in resolution and quality, the relative volume of data increases compared to conventional video data; consequently, transmission and storage costs rise when video data is transmitted using existing wired or wireless broadband lines or stored using conventional storage media. To address these issues arising from the increase in data resolution and quality, high-efficiency video encoding and decoding technologies for video with higher resolution and quality are required.

[0003] Intra prediction is a technique for eliminating spatial redundancy within a single frame. Recently, in addition to traditional angular or non-angular intra prediction, various prediction techniques are being used, such as Intra Template Matching Prediction (IntraTMP), Intra Block Copy (IBC), Matrix-based Intra Prediction (MIP), and Neural Network-based Intra Prediction (NNIP).

[0004] Although compression performance has been improved compared to previous standards through these various intra-prediction techniques, there is a problem in that spatial redundancy still exists in the residual data after prediction.

[0005] The present disclosure aims to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] In addition, the present disclosure aims to provide a recording medium storing a bitstream generated by an image decoding method or device according to the present disclosure.

[0007] Furthermore, the present disclosure aims to provide a method for applying a secondary prediction to a residual block after intra-prediction in order to solve the above-mentioned problems.

[0008] A video decoding method according to one embodiment of the present disclosure comprises the steps of obtaining a transformation coefficient from a bitstream, performing a prediction on the current block to derive a prediction sample of the current block, performing an inverse transformation on the transformation coefficient to obtain a residual sample of the current block, deriving a residual prediction sample of the current block, and generating a reconstructed sample of the current block based on the residual sample, the prediction sample, and the residual prediction sample, wherein the residual prediction sample may be derived based on a template of the current block.

[0009] In the above image decoding method, the prediction sample of the current block is derived using the matching block of the current block, and the residual prediction sample can be derived based on the template of the current block and the template of the matching block.

[0010] In the above image decoding method, the prediction mode of the current block is determined as either a matrix-based intra prediction mode or a neural network-based intra prediction mode, and the residual prediction sample can be derived by performing a prediction on a sample included in the template of the current block.

[0011] In the above image decoding method, the template of the current block may include at least one of a sample included in the upper region of the current block and a sample included in the left region of the current block.

[0012] In the above image decoding method, the step of acquiring residual samples of the current block includes the step of deriving an intra-prediction mode for determining a transform kernel of the current block and the step of determining a transform kernel based on the intra-prediction mode for determining the transform kernel, and an inverse transform may be performed on the transform coefficients based on the determined transform kernel.

[0013] In the above image decoding method, the intra prediction mode for determining the transformation kernel can be derived based on either the directional information of the residual prediction sample or the directional information of the sample included in the surrounding region for the current block.

[0014] In the above image decoding method, the intra prediction mode for determining the transformation kernel is derived using an occurrence frequency histogram, and the occurrence frequency histogram can be generated by accumulating the intra prediction modes of the surrounding blocks of the current block.

[0015] In the above image decoding method, the intra prediction mode for determining the transformation kernel can be derived into an intra prediction mode having a maximum value in the occurrence frequency histogram.

[0016] In the above image decoding method, the intra prediction mode of the surrounding block can be accumulated in the occurrence frequency histogram based on the size of the surrounding block.

[0017] In the above image decoding method, the surrounding blocks may include blocks spatially adjacent to the current block and blocks not spatially adjacent to the current block.

[0018] In the above image decoding method, the intra prediction mode for determining the conversion kernel can be derived into either a DC mode or a planner mode.

[0019] In the above image decoding method, the step of determining the transformation kernel may include the step of determining a transformation set of the current block based on an intra-prediction mode for determining the transformation kernel, and the step of determining a transformation kernel of the current block among the transformation kernels included in the transformation set.

[0020] The above image decoding method further includes the step of obtaining information regarding whether to perform residual prediction from a bitstream, and the residual prediction sample of the current block can be derived based on the information.

[0021] In the above image decoding method, the step of deriving residual prediction samples for the current block can be performed based on the size of the current block.

[0022] In the above image decoding method, the prediction sample of the current block is derived using a plurality of matching blocks, and the residual prediction sample can be derived based on the template of the plurality of matching blocks.

[0023] A video encoding method according to one embodiment of the present disclosure comprises the steps of: performing a prediction on a current block to derive a prediction sample of the current block; deriving a residual prediction sample of the current block; generating a residual sample of the current block based on the prediction sample and the residual prediction sample; performing a transformation on the residual sample to obtain a transformation coefficient; and encoding the transformation coefficient, wherein the residual prediction sample may be derived based on a template of the current block.

[0024] A non-transient computer-readable recording medium storing a bitstream generated by an image encoding method according to one embodiment of the present disclosure can store the bitstream generated by the image encoding method.

[0025] A bitstream transmission method according to one embodiment of the present disclosure can transmit a bitstream generated by the image encoding method.

[0026] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0027] According to the present disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency may be provided.

[0028] Additionally, according to the present disclosure, a method for applying a second prediction to a residual block after intra prediction may be provided.

[0029] In addition, according to the present disclosure, the amount of residual signal can be saved and coding efficiency can be improved.

[0030] In addition, according to the present disclosure, prediction accuracy can be improved.

[0031] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0032] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present disclosure applies.

[0033] FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present disclosure is applied.

[0034] FIG. 3 is a schematic diagram illustrating a video coding system to which the present disclosure can be applied.

[0035] FIG. 4 is a diagram illustrating a secondary prediction method after intra-template matching prediction according to one embodiment of the present disclosure.

[0036] FIG. 5 is a diagram illustrating an intra-prediction mode applicable in a secondary prediction according to one embodiment of the present disclosure.

[0037] FIG. 6 is a diagram illustrating a secondary prediction method after intra-block copying according to one embodiment of the present disclosure.

[0038] FIG. 7 is a diagram illustrating a secondary prediction method after a primary prediction using a template according to one embodiment of the present disclosure.

[0039] FIG. 8 is a diagram illustrating a secondary prediction method after redundant intra prediction according to one embodiment of the present disclosure.

[0040] FIG. 9 is a flowchart illustrating a decoding method according to one embodiment of the present disclosure.

[0041] FIG. 10 is a flowchart illustrating an encoding method according to one embodiment of the present disclosure.

[0042] FIG. 11 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0043] A video decoding method according to one embodiment of the present disclosure comprises the steps of obtaining a transformation coefficient from a bitstream, performing a prediction on the current block to derive a prediction sample of the current block, performing an inverse transformation on the transformation coefficient to obtain a residual sample of the current block, deriving a residual prediction sample of the current block, and generating a reconstructed sample of the current block based on the residual sample, the prediction sample, and the residual prediction sample, wherein the residual prediction sample may be derived based on a template of the current block.

[0044] The present disclosure is subject to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the present disclosure. Similar reference numerals in the drawings refer to the same or similar functions across various aspects. The shapes and sizes of elements in the drawings may be provided illustratively for clearer explanation. The detailed description of the exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present disclosure in relation to one embodiment. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of the exemplary embodiments is limited only by the appended claims, together with all equivalents to those claimed therein, provided they are properly described.

[0045] In this disclosure, terms such as first, second, etc. may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of this disclosure, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0046] The components shown in the embodiments of the present disclosure are depicted independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for convenience of explanation; however, at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are included within the scope of the rights of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0047] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit this disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. Additionally, some components of this disclosure may not be essential components performing an essential function in this disclosure, but may be optional components merely for enhancing performance. This disclosure may be implemented by including only the components essential to embody the essence of this disclosure, excluding components used merely for performance enhancement, and a structure including only the essential components, excluding optional components used merely for performance enhancement, is also included within the scope of this disclosure.

[0048] In the embodiments, the term "at least one" may mean one of a number of 1 or more, such as 1, 2, 3, and 4. In the embodiments, the term "a plurality of" may mean one of a number of 2 or more, such as 2, 3, and 4.

[0049] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In describing the embodiments of this specification, if it is determined that a detailed description of related known configurations or functions may obscure the gist of this specification, such detailed description is omitted, and the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0050] Glossary of Terms

[0051] In the following, “image” may refer to a single picture constituting a video, or it may refer to the video itself. For example, “encoding and / or decoding of an image” may mean “encoding and / or decoding of an image”, and may also mean “encoding and / or decoding of one of the images constituting the video”.

[0052] In the following, "video" and "video" may be used interchangeably with the same meaning. Additionally, the target image may be an image to be encoded and / or an image to be decoded. Furthermore, the target image may be an input image fed into an encoding device and an input image fed into a decoding device. Here, the target image may have the same meaning as the current image.

[0053] In the following, the terms encoder and image encoding device may be used interchangeably.

[0054] In the following, the decoder and the image decoder may be used interchangeably with each other.

[0055] In the following, "image," "picture," "frame," and "screen" may be used interchangeably with the same meaning.

[0056] In the following, “target block” may be an encoding target block that is the target of encoding and / or a decoding target block that is the target of decoding. Additionally, the target block may be a current block that is the target of current encoding and / or decoding. For example, “target block” and “current block” may be used interchangeably.

[0057] In the following description, "block" and "unit" may be used interchangeably. Additionally, to distinguish it from a block, "unit" may refer to a block containing a luminance (Luma) component block and a corresponding chroma (Chroma) component block. For example, a Coding Tree Unit (CTU) may consist of a single luminance component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated with it.

[0058] In the following, “sample,” “pixel,” and “pixel” may be used interchangeably with the same meaning. Here, a sample may represent a basic unit constituting a block.

[0059] In the following, “inter” and “inter-screen” may be used interchangeably with the same meaning.

[0060] In the following, “intra” and “in-screen” may be used interchangeably with the same meaning.

[0061]

[0062] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present disclosure applies.

[0063] The encoding device (100) may be an encoder, a video encoding device, or an image encoding device. The video may include one or more images. The encoding device (100) may sequentially encode one or more images.

[0064] Referring to FIG. 1, the encoding device (100) may include an image segmentation unit (110), an intra prediction unit (120), a motion prediction unit (121), a motion compensation unit (122), a switch (115), a subtractor (113), a converter (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse converter (170), an adder (117), a filter unit (180), and a reference picture buffer (190).

[0065] Additionally, the encoding device (100) can generate a bitstream containing encoded information through encoding of an input image and can output the generated bitstream. The generated bitstream can be stored on a computer-readable recording medium or streamed via a wired / wireless transmission medium.

[0066] The video segmentation unit (110) can divide the input video into various forms to increase the efficiency of video encoding / decoding. That is, the input video consists of multiple pictures, and a single picture can be processed by hierarchically dividing it for compression efficiency, parallel processing, etc. For example, a single picture can be divided into one or more tiles or slices and then divided again into multiple CTUs (Coding Tree Units). Alternatively, a single picture can first be divided into multiple sub-pictures defined as groups of rectangular slices, and each sub-picture can be divided into the said tiles / slices. Here, the sub-pictures can be utilized to support the function of partially and independently encoding / decoding and transmitting the picture. Since multiple sub-pictures can each be restored individually, they have the advantage of being easy to edit in applications that configure multi-channel inputs into a single picture. In addition, the tiles can be divided horizontally to create bricks. Here, a brick can be utilized as the basic unit of parallel processing within a picture. Additionally, a single CTU can be recursively partitioned into a Quadtree (QT), and the terminal node of the partition can be defined as a Coding Unit (CU). The CU can be divided into a Prediction Unit (PU) and a Transform Unit (TU) to perform prediction and partitioning. Meanwhile, the CU can be utilized as the prediction unit and / or the transformation unit itself. Here, for flexible partitioning, each CTU can be recursively partitioned into a Multi-Type Tree (MTT) as well as a Quadtree (QT). The partitioning of the CTU into a Multi-Type Tree can begin at the terminal node of the QT, and the MTT can be composed of a Binary Tree (BT) and a Triple Tree (TT).For example, the MTT structure can be classified into vertical binary splitting mode (SPLIT_BT_VER), horizontal binary splitting mode (SPLIT_BT_HOR), vertical ternary splitting mode (SPLIT_TT_VER), and horizontal ternary splitting mode (SPLIT_TT_HOR). Additionally, when splitting, the minimum block size (MinQTSize) of the quad tree for the luminance block can be set to 16x16, the maximum block size (MaxBtSize) of the binary tree to 128x128, and the maximum block size (MaxTtSize) of the triple tree to 64x64. Furthermore, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the triple tree can be set to 4x4, and the maximum depth (MaxMttDepth) of the multi-type tree can be set to 4. Additionally, to increase the encoding efficiency of the I slice, a dual tree can be applied that uses different CTU splitting structures for the luminance and chrominance components. On the other hand, in P and B slices, the luminance and color difference CTBs (Coding Tree Blocks) within the CTU can be divided into a single tree that shares a coding tree structure.

[0067] The encoding device (100) may perform encoding on an input image in an intra mode and / or inter mode. Alternatively, the encoding device (100) may perform encoding on an input image in a third mode other than the intra mode and inter mode (e.g., IBC mode, Palette mode, etc.). However, if the third mode has functional characteristics similar to the intra mode or inter mode, it may be classified as an intra mode or inter mode for convenience of explanation. In this disclosure, the third mode will be classified and described separately only when a specific description of the third mode is required.

[0068] When intra mode is used as the prediction mode, the switch (115) can be switched to intra, and when inter mode is used as the prediction mode, the switch (115) can be switched to inter. Here, intra mode may mean an intra-frame prediction mode, and inter mode may mean an inter-frame prediction mode. The encoding device (100) can generate a prediction block for an input block of an input image. Additionally, after the prediction block is generated, the encoding device (100) can encode a residual block using the residual of the input block and the prediction block. The input image may be referred to as the current image that is the subject of current encoding. The input block may be referred to as the current block that is the subject of current encoding or the encoding target block.

[0069] When the prediction mode is an intra mode, the intra prediction unit (120) may use a sample of a block that has already been encoded / decoded around the current block as a reference sample. The intra prediction unit (120) may perform spatial prediction for the current block using the reference sample and generate prediction samples for the input block through spatial prediction. Here, intra prediction may mean intra-frame prediction.

[0070] In the intra prediction method, non-directional prediction modes such as DC mode and Planar mode, and directional prediction modes (e.g., 65 directions) may be applied. Here, the intra prediction method can be expressed as an intra prediction mode or an intra-frame prediction mode.

[0071] When the prediction mode is an inter mode, the motion prediction unit (121) can search for the region that best matches the input block from the reference image during the motion prediction process and derive a motion vector using the searched region. At this time, the search region can be used as the region. The reference image can be stored in the reference picture buffer (190). Here, the reference image can be stored in the reference picture buffer (190) when encoding / decoding of the reference image is processed.

[0072] The motion compensation unit (122) can generate a prediction block for the current block by performing motion compensation using a motion vector. Here, inter-prediction may mean inter-frame prediction or motion compensation.

[0073] The motion prediction unit (121) and motion compensation unit (122) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform inter-frame prediction or motion compensation, based on the encoding unit, it can determine whether the motion prediction and motion compensation method of the prediction unit included in the corresponding encoding unit is a Skip Mode, Merge Mode, Advanced Motion Vector Prediction (AMVP) Mode, or Intra Block Copy (IBC) Mode, and can perform inter-frame prediction or motion compensation according to each mode.

[0074] In addition, based on the above-mentioned inter-frame prediction method, the AFFINE mode of sub-PU-based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and GPM (Geometric Partitioning Mode) mode of PU-based prediction may be applied. Furthermore, to improve the performance of each mode, HMVP (History based MVP), PAMVP (Pairwise Average MVP), CIIP (Combined Intra / Inter Prediction), AMVR (Adaptive Motion Vector Resolution), BDOF (Bi-Directional Optical-Flow), BCW (Bi-predictive with CU Weights), LIC (Local Illumination Compensation), TM (Template Matching), OBMC (Overlapped Block Motion Compensation), etc. may be applied.

[0075] Among these, AFFINE mode is a technology used in both AMVP and MERGE modes and also offers high encoding efficiency. Conventional video coding standards have the disadvantage of failing to properly compensate for real-world movements, such as zoom in / out and rotation, because they perform Motion Compensation (MC) by considering only the translation of blocks. To address this, a 4-parameter affine motion model using two control point motion vectors (CPMV) and a 6-parameter affine motion model using three control point motion vectors can be applied to inter-prediction. Here, CPMV is a vector representing one of the affine motion models of the top-left, top-right, or bottom-left corners of the current block.

[0076] The subtractor (113) can generate a residual block using the difference between the input block and the prediction block. The residual block may also be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or both transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal in block units.

[0077] The transformation unit (130) can generate a transform coefficient by performing a transform on the remaining block and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the remaining block. When a transform skip mode is applied, the transformation unit (130) may skip the transform on the remaining block.

[0078] A quantized level can be generated by applying quantization to a conversion coefficient or a residual signal. In the following embodiments, the quantized level may also be referred to as a conversion coefficient.

[0079] For example, a 4x4 luminance residual block generated through intra prediction can be transformed using a Discrete Sine Transform (DST)-based basis vector, while the remaining residual blocks can be transformed using a Discrete Cosine Transform (DCT)-based basis vector. Additionally, the transformation blocks for a single block can be divided into a quad tree form using Residual Quad Tree (RQT) technology, and after performing transformation and quantization on each transformation block divided by RQT, a coded block flag (cbf) can be transmitted to increase coding efficiency in the case where all coefficients become zero.

[0080] As another alternative, the Multiple Transform Selection (MTS) technique can be applied to perform transformations using multiple transformation bases selectively. In other words, instead of dividing a CU into TUs via RQT, a function similar to TU division can be performed using the Sub-block Transform (SBT) technique. Specifically, SBT is applied only to inter-frame prediction blocks and, unlike RQT, divides the current block into ½ or ¼ sizes in the vertical or horizontal direction, and then performs a transformation on only one of the blocks. For example, if divided vertically, a transformation can be performed on the leftmost or rightmost block, and if divided horizontally, a transformation can be performed on the topmost or bottommost block.

[0081] In addition, Low Frequency Non-Separable Transform (LFNST), a secondary transform technique that further transforms the residual signal converted to the frequency domain through DCT or DST, can also be applied. LFNST performs additional transformation on the 4x4 or 8x8 low-frequency region in the upper left corner, thereby allowing the residual coefficients to be concentrated in the upper left corner.

[0082] The quantization unit (140) can generate a quantized level by quantizing a transformation coefficient or residual signal according to a quantization parameter (QP, Quantization parameter) and can output the generated quantized level. At this time, the quantization unit (140) can quantize the transformation coefficient using a quantization matrix.

[0083] For example, a quantizer using QP values ​​from 0 to 51 can be used. Alternatively, if the image size is larger and higher coding efficiency is required, QP values ​​from 0 to 63 can be used. Additionally, a Dependent Quantization (DQ) method using two quantizers instead of a single one can be applied. DQ performs quantization using two quantizers (e.g., Q0, Q1), but can be applied so that the quantizer to be used for the next transform coefficient is selected based on the current state through a state transition model, even without signaling information regarding the use of a specific quantizer.

[0084] The entropy encoding unit (150) can generate a bitstream and output a bitstream by performing entropy encoding according to a probability distribution on values ​​calculated by the quantization unit (140) or coding parameter values ​​calculated during the encoding process. The entropy encoding unit (150) can perform entropy encoding on information regarding a sample of an image and information for decoding an image. For example, information for decoding an image may include syntax elements, etc.

[0085] When entropy coding is applied, a small number of bits are allocated to symbols with a high probability of occurrence and a large number of bits are allocated to symbols with a low probability of occurrence, thereby representing the symbols and reducing the size of the bit sequence for the symbols to be encoded. The entropy coding unit (150) may use encoding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) for entropy coding. For example, the entropy coding unit (150) may perform entropy coding using a Variable Length Coding (VLC) table. In addition, the entropy encoding unit (150) may perform arithmetic encoding using the derived binarization method, probability model, and context model after deriving a binarization method of the target symbol and a probability model of the target symbol / bin.

[0086] In this regard, when applying CABAC, in order to reduce the size of the probability table stored in the decoder, the table probability update method may be changed to a table update method using a simple formula. In addition, two different probability models may be used to obtain more accurate symbol probability values.

[0087] The entropy encoding unit (150) can convert a 2-dimensional block form coefficient into a 1-dimensional vector form through a transform coefficient scanning method to encode a transform coefficient level (quantized level).

[0088] Coding parameters may include information (flags, indexes, etc.) that is encoded in the encoding device (100) and signaled to the decoding device (200), such as syntax elements, as well as information derived during the encoding process or decoding process, and may refer to information required when encoding or decoding images.

[0089] Here, signaling a flag or index may mean that in an encoder, the corresponding flag or index is entropy encoded and included in a bitstream, and in a decoder, the corresponding flag or index is entropy decoded from the bitstream.

[0090] The encoded current image can be used as a reference image for other images processed later. Accordingly, the encoding device (100) can restore or decode the encoded current image again, and can store the restored or decoded image as a reference image in the reference picture buffer (190).

[0091] The quantized level can be dequantized in the dequantization unit (160) and inverse transformed in the inverse transform unit (170). The dequantized and / or inverse transformed coefficients can be added to the prediction block through the adder (117). A reconstructed block can be generated by adding the dequantized and / or inverse transformed coefficients and the prediction block. Here, the dequantized and / or inverse transformed coefficients refer to coefficients for which at least one of dequantization and inverse transformation has been performed, and may refer to the reconstructed residual block. The dequantization unit (160) and the inverse transform unit (170) can be performed as the reverse process of the quantization unit (140) and the transformation unit (130).

[0092] The restoration block may pass through a filter section (180). The filter section (180) may apply a deblocking filter, Sample Adaptive Offset (SAO), Adaptive Loop Filter (ALF), Bilateral filter (BIF), LMCS (Luma Mapping with Chroma Scaling), etc., to the restoration sample, restoration block, or restoration image as a whole or part of the filtering technique. The filter section (180) may also be referred to as an in-loop filter. In this case, the term in-loop filter is also used as a name that excludes LMCS.

[0093] Deblocking filters can remove block distortion occurring at the boundaries between blocks. To determine whether to perform deblocking, the decision to apply the filter to the current block can be made based on samples contained in a few columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering intensity.

[0094] To compensate for encoding errors using a sample adaptive offset, an appropriate offset value can be added to the sample value. The sample adaptive offset can correct the offset from the original image on a sample-by-sample basis for the deblocked image. One method may be to divide the samples included in the image into a certain number of regions, determine the region to be offset, and apply the offset to that region, or to apply the offset by considering the edge information of each sample.

[0095] A bilateral filter (BIF) can also correct the offset from the original image on a sample-by-sample basis for the deblocked image.

[0096] An adaptive loop filter can perform filtering based on a comparison of the reconstructed image and the original image. After dividing the samples included in the image into predetermined groups, a filter to be applied to each group can be determined, thereby performing filtering differently for each group. Information regarding whether to apply an adaptive loop filter can be signaled per coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may vary depending on each block.

[0097] In LMCS (Luma Mapping with Chroma Scaling), Luma mapping (LM) refers to remapping luminance values ​​through a piece-wise linear model, and Chroma scaling (CS) refers to a technique that scales the residual values ​​of the chrominance component according to the average luminance value of the predicted signal. In particular, LMCS can be utilized as an HDR correction technique that reflects the characteristics of HDR (High Dynamic Range) video.

[0098] The restored block or restored image that has passed through the filter unit (180) can be stored in the reference picture buffer (190). The restored block that has passed through the filter unit (180) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (180). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0099] FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present disclosure is applied.

[0100] The decoding device (200) may be a decoder, a video decoding device, or an image decoding device.

[0101] Referring to FIG. 2, the decoding device (200) may include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), a motion compensation unit (250), an adder (201), a switch (203), a filter unit (260), and a reference picture buffer (270).

[0102] The decoding device (200) can receive a bitstream output from the encoding device (100). The decoding device (200) can receive a bitstream stored in a computer-readable recording medium or a bitstream stream streamed through a wired / wireless transmission medium. The decoding device (200) can perform decoding on the bitstream in intra mode or inter mode. Additionally, the decoding device (200) can generate a restored image or a decoded image through decoding and can output the restored image or the decoded image.

[0103] If the prediction mode used for decoding is intra mode, the switch (203) can be switched to intra. If the prediction mode used for decoding is inter mode, the switch (203) can be switched to inter.

[0104] The decoding device (200) can decode the input bitstream to obtain a reconstructed residual block and generate a prediction block. Once the reconstructed residual block and the prediction block are obtained, the decoding device (200) can generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may be referred to as the current block.

[0105] The entropy decoding unit (210) can generate symbols by performing entropy decoding according to the probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the inverse process of the entropy encoding method described above.

[0106] The entropy decoding unit (210) can convert a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a conversion coefficient scanning method to decode a conversion coefficient level (quantized level).

[0107] The quantized level can be dequantized in the dequantization unit (220) and inversely transformed in the inverse transformation unit (230). The quantized level can be generated as a restored residual block as a result of performing dequantization and / or inverse transformation. At this time, the dequantization unit (220) can apply a quantization matrix to the quantized level. The dequantization unit (220) and the inverse transformation unit (230) applied to the decoding device can apply the same technology as the dequantization unit (160) and the inverse transformation unit (170) applied to the aforementioned encoding device.

[0108] When an intra mode is used, the intra prediction unit (240) can generate a prediction block by performing a spatial prediction on the current block using sample values ​​of already decoded blocks around the block to be decoded. The intra prediction unit (240) applied to the decoding device can apply the same technology as the intra prediction unit (120) applied to the aforementioned encoding device.

[0109] When an inter mode is used, the motion compensation unit (250) can generate a prediction block by performing motion compensation on the current block using a motion vector and a reference image stored in the reference picture buffer (270). The motion compensation unit (250) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform motion compensation, it can be determined whether the motion compensation method of the prediction unit included in the corresponding encoding unit is a skip mode, merge mode, AMVP mode, or current picture reference mode based on the encoding unit, and motion compensation can be performed according to each mode. The motion compensation unit (250) applied to the decoder can apply the same technology as the motion compensation unit (122) applied to the aforementioned encoding unit.

[0110] The adder (201) can generate a restored block by adding the restored residual block and the prediction block. The filter unit (260) can apply at least one of the following to the restored block or the restored image: an inverse-LMCS, a deblocking filter, a sample adaptive offset, and an adaptive loop filter. The filter unit (260) applied to the decoder can apply the same filtering technology as the filter unit (180) applied to the aforementioned encoding device.

[0111] The filter unit (260) can output a restored image. The restored block or the restored image can be stored in a reference picture buffer (270) and used for inter-frame prediction. The restored block that has passed through the filter unit (260) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (260). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0112] FIG. 3 is a schematic diagram illustrating a video coding system to which the present disclosure can be applied.

[0113] A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and / or image information or data to the decoding device (20) via a digital storage medium or network in the form of a file or streaming.

[0114] An encoding device (10) according to one embodiment may include a video source generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to one embodiment may include a receiving unit (21), a decoding unit (22), and a rendering unit (23). The encoding unit (12) may be called a video / image encoding unit, and the decoding unit (22) may be called a video / image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The receiving unit (21) may be included in the decoding unit (22). The rendering unit (23) may include a display unit, and the display unit may be composed of a separate device or an external component.

[0115] The video source generation unit (11) can acquire video / image through a process of capturing, synthesizing, or generating video / image. The video source generation unit (11) may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / image, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and can generate video / image (electronically). For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0116] The encoding unit (12) can encode the input video / image. The encoding unit (12) can perform a series of procedures such as prediction, conversion, and quantization for compression and encoding efficiency. The encoding unit (12) can output the encoded data (encoded video / image information) in the form of a bitstream. The detailed configuration of the encoding unit (12) can also be configured in the same way as the encoding device (100) of FIG. 1 described above.

[0117] The transmission unit (13) can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit (21) of the decoding device (20) via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit (21) can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

[0118] The decoding unit (22) can decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12). The detailed configuration of the decoding unit (22) can also be configured to be identical to the decoding device (200) of FIG. 2 described above.

[0119] The rendering unit (23) can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0120]

[0121] Recently, various intra prediction techniques to eliminate spatial redundancy have been studied. In addition to traditional angular and non-angular intra prediction methods, techniques such as Intra Template Matching Prediction (IntraTMP), Intra Block Copy (IBC), Matrix-based Intra Prediction (MIP), and Neural Network-based Intra Prediction (NNIP) have been introduced, improving prediction accuracy and contributing to improved compression performance.

[0122] However, even when applying these various intra-prediction techniques, there is a problem in that spatial redundancy still exists in the residual data after prediction.

[0123] This specification proposes a quadratic prediction method that performs additional prediction on residual samples after intra-prediction (i.e., primary prediction). Through this, a method is proposed to reduce residual signals and improve compression efficiency.

[0124] Meanwhile, the sample of the prediction block obtained by performing a second-order prediction can be called the residual prediction sample.

[0125] In the secondary prediction method according to the embodiments of the present disclosure, the decoded pixel can be generated as in Equation 1.

[0126]

[0127]

[0128] In mathematical equation 1, P primary represents the prediction value obtained through primary prediction, and P residualrepresents the residual prediction value obtained through secondary prediction. And, R secondary represents the decoded secondary prediction residual value. The value of a decoded pixel is P primary, P residual and R secondary It can be calculated as the sum of.

[0129] Meanwhile, the residual prediction value obtained through second prediction may refer to a residual prediction sample, and the decoded second prediction residual value may refer to a value obtained by inversely transforming the transformation coefficients obtained from the corresponding bitstream in the decoder after the residual value after second prediction is transformed and transmitted as a bitstream in the encoder.

[0130] Meanwhile, in the embodiments described below, primary prediction is mentioned as intra-template matching prediction, intra-block copying, matrix-based intra prediction, and neural network-based intra prediction, but this is merely an example and primary prediction methods may vary and are not limited in this disclosure.

[0131]

[0132] FIG. 4 is a diagram illustrating a secondary prediction method after intra-template matching prediction according to an embodiment of the present disclosure. Intra-template matching prediction means a method of searching for an optimal matching block in a restored area of ​​a current picture using template matching, and copying it to generate a prediction block of the current block.

[0133] Referring to FIG. 4, the neighboring L-shaped regions (i.e., the left, top, and top-left regions) of the current block (400) can be defined as the current template (401). Then, within the predefined search range of the reconstructed area (402) of the current picture, the reference template (411) most similar to the current template (401) can be searched. Then, the prediction block of the current block (400) can be derived based on the corresponding matching block (410) of the determined reference template (411). For example, a prediction sample can be generated by copying the matching block (410) to the current block (400).

[0134] Then, a residual of current block (420) can be obtained based on the difference between the current block (400) and the predicted block. Since spatial redundancy may still exist in the residual (420), to eliminate this, a residual of current template (421) can be additionally obtained based on the difference between the current template (401) and the reference template (411). Subsequently, under the assumption that the residual of current template (421) and the residual of current block (420) are similar, a prediction can be performed based on the obtained residual of current template (421).

[0135] Meanwhile, in FIG. 4, the size of the current template (401) can be determined as (w x L2) + (L1 x h) + (L1 x L2). And, the size of the residual (421) of the current template can be determined as (w x L4) + (L3 x h) + (L3 x L4). Here, w and h represent the width and height of the current block, and the values ​​of L1, L2, L3 and L4 can be determined as any positive integer.

[0136] According to one embodiment of the present disclosure, a secondary prediction may be performed by performing a directional and / or non-directional intra prediction on all or part of the residual samples of the current template.

[0137] In the encoder, prediction can be performed on all or some of the residual samples of the current template based on a predetermined number of directional and / or non-directional intra-prediction modes.

[0138] And, based on a predetermined cost function, an intra prediction mode having the lowest cost value may be selected. Specifically, for each intra prediction mode, a cost such as a rate-distortion cost (RD cost) is calculated, and an intra prediction mode with the smallest distortion may be selected. Here, having the smallest distortion may mean that the distortion with the residual (420) of the current block is the smallest, or that the distortion with the residual (421) of the current template is the smallest.

[0139] And, based on the selected prediction mode, a prediction can be performed to derive residual prediction samples.

[0140] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0141] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0142] Meanwhile, the selected prediction mode information and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0143] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0144] Additionally, in the decoder, a prediction is performed on the residual (421) of the current template according to the selected prediction mode information to derive a residual prediction sample, and a first residual sample can be derived by summing this with a second residual sample.

[0145] In addition, intra-template matching prediction can be performed in the decoder to derive prediction samples based on the matching block. Subsequently, by summing the first residual sample and the prediction sample, a recovery sample of the current block can be derived.

[0146] Meanwhile, information regarding the selected prediction mode may be expressed in the form of a specific flag or index.

[0147] Meanwhile, the aforementioned predetermined number of directional and / or non-directional intra-prediction modes may be defined differently or identically depending on the size of the current block.

[0148] According to one embodiment of the present disclosure, a second prediction may be performed by performing matrix-based intra prediction on all or part of the residual samples of the current template. Here, matrix-based intra prediction may refer to a method of performing prediction using a predefined weight matrix.

[0149] In the encoder, prediction can be performed by applying a predetermined number of weight matrices to all or some of the residual samples of the current template.

[0150] And, a weight matrix having the lowest cost value based on a predetermined cost function may be selected. Specifically, for each weight matrix, a cost such as a bit rate-distortion cost may be calculated. Here, having the smallest distortion may mean that the distortion with respect to the residual (420) of the current block is the smallest, or that the distortion with respect to the residual (421) of the current template is the smallest.

[0151] And, matrix-based intra-prediction is performed based on the selected weight matrix to derive residual prediction samples.

[0152] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0153] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0154] Meanwhile, information regarding the selected weight matrix and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0155] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0156] In addition, the decoder performs matrix-based intra-prediction based on information regarding the selected weight matrix to derive residual prediction samples, and these can be combined with secondary residual samples to derive primary residual samples.

[0157] In addition, intra-template matching prediction can be performed in the decoder to derive prediction samples based on the matching block. Subsequently, by summing the first residual sample and the prediction sample, a recovery sample of the current block can be derived.

[0158] Meanwhile, information regarding the selected weight matrix can be expressed in the form of a specific flag or index.

[0159] Meanwhile, the aforementioned predetermined number of weight matrices may be defined differently or identically depending on the size of the current block, quantization parameters (QP), etc.

[0160] According to one embodiment of the present disclosure, a second prediction may be performed by performing a neural network-based intra prediction on all or part of the residual samples of the current template. Here, a neural network-based intra prediction may refer to a method of performing a prediction using a predefined neural network model.

[0161] In the encoder, neural network-based intra prediction can be performed by applying a predetermined number of neural network models to all or some of the residual samples of the current template.

[0162] And, based on a predetermined cost function, a neural network model having the lowest cost value may be selected. Specifically, for each neural network model, a cost such as a bit rate-distortion cost is calculated, and a neural network model with the smallest distortion may be selected. Here, having the smallest distortion may mean that the distortion with respect to the residual (420) of the current block is the smallest, or that the distortion with respect to the residual (421) of the current template is the smallest.

[0163] And, based on the selected neural network model, neural network-based intra-prediction is performed to derive residual prediction samples.

[0164] And, a residual sample after secondary prediction can be obtained based on the sample included in the residual (420) of the current block and the residual prediction sample. For example, a residual sample after secondary prediction can be calculated based on the difference between the residual block sample of the current block and the residual prediction sample.

[0165] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0166] Meanwhile, information regarding the selected neural network model and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0167] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0168] In addition, in the decoder, neural network-based intra-prediction is performed according to information regarding the selected neural network model to derive residual prediction samples, and these can be combined with secondary residual samples to derive primary residual samples.

[0169] In addition, intra-template matching prediction can be performed in the decoder to derive prediction samples based on the matching block. Subsequently, by summing the first residual sample and the prediction sample, a recovery sample of the current block can be derived.

[0170] Meanwhile, information regarding the selected neural network model may be expressed in the form of a specific flag or index.

[0171] Meanwhile, the aforementioned predetermined number of neural network models may be defined differently or identically depending on the size of the current block, quantization parameters (QP), etc.

[0172] Meanwhile, the shape and size of the template in FIG. 4 are merely examples, and the size and shape of the template can be determined in various ways depending on the protocol between the encoder and decoder, the size of the block, related syntax elements, etc. For example, only one of the top area or the left area of ​​the current block may be used as the current template.

[0173] Meanwhile, in the embodiments regarding the secondary prediction method following the intra-template matching prediction described above, a single method may be performed, or one or more methods may be performed in the encoder. When one or more methods are used, the method having the lowest cost value based on a predetermined cost function in the encoder may be selected, and information regarding the selected method may be signaled to the decoder. Information regarding the selected method may be expressed in the form of a predetermined flag or index. In the decoder, the secondary prediction method may be performed using the selected method based on the information.

[0174]

[0175] FIG. 5 is a diagram illustrating an intra-prediction mode applicable in a second prediction according to one embodiment of the present disclosure. FIG. 5 assumes a case where prediction is performed based on nine intra-prediction modes for all or part of the residual samples of the current template.

[0176] Referring to Fig. 5, prediction can be performed on all or part of the residual samples of the current template based on a total of 9 intra prediction modes, including vertical, horizontal, and DC.

[0177] And, based on a predetermined cost function, an intra prediction mode having the lowest cost value can be selected. Specifically, for each intra prediction mode, a cost such as a bit rate-distortion cost is calculated, and among the nine intra prediction modes, the intra prediction mode with the smallest distortion can be selected.

[0178] And, based on the selected prediction mode, a prediction can be performed to obtain residual prediction samples.

[0179] Meanwhile, the nine intra-prediction modes shown in Fig. 5 are merely examples, and the number or types of applicable intra-prediction modes are not limited thereto.

[0180]

[0181] FIG. 6 is a diagram illustrating a secondary prediction method after intra-block copying according to an embodiment of the present disclosure. Intra-block copying refers to a method of searching for an optimal prediction block in a restored area of ​​a current picture using a block vector, and copying it to generate a prediction block of the current block.

[0182] Referring to FIG. 6, the matching block (610) most similar to the current block (600) can be found in the reconstructed area (602) of the current picture. Here, the vector indicating the matching block (610) can be called the block vector (603).

[0183] And, the prediction block of the current block (600) can be derived based on the matching block (610).

[0184] Then, a residual of current block (620) can be obtained based on the difference between the current block (600) and the matching block (610). Since spatial redundancy may still exist in the residual (620), in order to remove it, a residual of neighboring region of current block (621) can be additionally obtained based on the difference between the neighboring region of current block (601) and the neighboring region of matching block (611). Subsequently, under the assumption that the residual of neighboring region of current block (621) and the residual of current block (620) are similar, a prediction can be performed based on the obtained residual of neighboring region of current block (621).

[0185] Meanwhile, in FIG. 6, the size of the surrounding area (601) of the current block can be determined as (w x L2) + (L1 x h) + (L1 x L2). And, the size of the residual (621) of the surrounding area of ​​the current block can be determined as (w x L4) + (L3 x h) + (L3 x L4). Here, w and h represent the width and height of the current block, and the values ​​of L1, L2, L3, and L4 can be determined as any positive integer.

[0186] According to one embodiment of the present disclosure, a secondary prediction may be performed by performing directional and / or non-directional intra prediction on all or part of the residual samples of the surrounding area of ​​the current block.

[0187] In the encoder, prediction can be performed on all or part of the residual samples of the surrounding region of the current block based on a predetermined number of directional and / or non-directional intra-prediction modes.

[0188] And, based on a predetermined cost function, an intra prediction mode having the lowest cost value may be selected. Specifically, for each intra prediction mode, a cost such as a bit rate-distortion cost is calculated, and the intra prediction mode with the smallest distortion may be selected. Here, having the smallest distortion may mean that the distortion with respect to the residual (620) of the current block is the smallest, or that the distortion with respect to the residual (621) of the surrounding area of ​​the current block is the smallest.

[0189] And, based on the selected prediction mode, a prediction can be performed to obtain residual prediction samples.

[0190] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0191] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0192] Meanwhile, the selected prediction mode information and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0193] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0194] In addition, the decoder performs a prediction based on the selected prediction mode information to derive a residual prediction sample, and can derive a first-order residual sample by summing this with a second-order residual sample.

[0195] In addition, intra-block copying can be performed in the decoder to derive a prediction sample based on the matching block. Subsequently, a recovery sample of the current block can be derived by summing the first residual sample and the prediction sample.

[0196] Meanwhile, information regarding the selected prediction mode may be expressed in the form of a specific flag or index.

[0197] Meanwhile, the intra prediction mode applicable in the secondary prediction according to Fig. 5 can also be applied in this embodiment.

[0198] Meanwhile, the aforementioned predetermined number of directional and / or non-directional intra-prediction modes may be defined differently or identically depending on the size of the current block.

[0199] According to one embodiment of the present disclosure, a second prediction may be performed by performing matrix-based intra prediction on all or part of the residual samples of the surrounding region of the current block. Here, matrix-based intra prediction may refer to a method of performing prediction using a predefined weight matrix.

[0200] In the encoder, prediction can be performed by applying a predetermined number of weight matrices to all or part of the residual samples in the surrounding region of the current block.

[0201] And, a weight matrix having the lowest cost value based on a predetermined cost function may be selected. Specifically, for each weight matrix, a cost such as a bit rate-distortion cost is calculated, and a weight matrix with the smallest distortion may be selected. Here, having the smallest distortion may mean that the distortion with respect to the residual (620) of the current block is the smallest, or that the distortion with respect to the residual (621) of the surrounding area of ​​the current block is the smallest.

[0202] And, based on the selected weight matrix, predictions can be performed to derive residual prediction samples.

[0203] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0204] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0205] Meanwhile, information regarding the selected weight matrix and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0206] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0207] In addition, the decoder performs matrix-based intra-prediction based on information regarding the selected weight matrix to derive residual prediction samples, and these can be combined with secondary residual samples to derive primary residual samples.

[0208] In addition, intra-block copying can be performed in the decoder to derive a prediction sample based on the matching block. Subsequently, a recovery sample of the current block can be derived by summing the first residual sample and the prediction sample.

[0209] Meanwhile, information regarding the selected weight matrix can be expressed in the form of a specific flag or index.

[0210] Meanwhile, the aforementioned predetermined number of weight matrices may be defined differently or identically depending on the size of the current block, quantization parameters (QP), etc.

[0211] According to one embodiment of the present disclosure, a second prediction may be performed by performing a neural network-based intra prediction on all or part of the residual samples of the surrounding region of the current block. Here, the neural network-based intra prediction may refer to a method of performing a prediction using a predefined neural network model.

[0212] In the encoder, neural network-based intra prediction can be performed by applying a predetermined number of neural network models to all or part of the residual samples in the surrounding region of the current block.

[0213] And, based on a predetermined cost function, a neural network model having the lowest cost value may be selected. Specifically, for each neural network model, a cost such as a bit rate-distortion cost is calculated, and a neural network model with the smallest distortion may be selected. Here, having the smallest distortion may mean that the distortion with respect to the residual (620) of the current block is the smallest, or that the distortion with respect to the residual (621) of the surrounding area of ​​the current block is the smallest.

[0214] And, based on the selected neural network model, predictions can be performed to derive residual prediction samples.

[0215] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0216] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0217] Meanwhile, information regarding the selected neural network model and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0218] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0219] In addition, in the decoder, neural network-based intra-prediction is performed according to information regarding the selected neural network model to derive residual prediction samples, and these can be combined with secondary residual samples to derive primary residual samples.

[0220] In addition, intra-block copying can be performed in the decoder to derive a prediction sample based on the matching block. Subsequently, a recovery sample of the current block can be derived by summing the first residual sample and the prediction sample.

[0221] Meanwhile, information regarding the selected neural network model may be expressed in the form of a specific flag or index.

[0222] Meanwhile, the aforementioned predetermined number of neural network models may be defined differently or identically depending on the size of the current block, quantization parameters (QP), etc.

[0223] Meanwhile, the shape and size of the surrounding area in Fig. 6 are just one example, and the size and shape of the surrounding area can be determined in various ways depending on the protocol between the encoder and decoder, the size of the block, related syntax elements, etc. For example, only one of the top area or the left area of ​​the current block may be used for secondary prediction.

[0224] Meanwhile, in the embodiments regarding the secondary prediction method after intra-block copying described above, one method may be performed, or one or more methods may be performed in the encoder. When one or more methods are used, the method having the lowest cost value based on a predetermined cost function in the encoder is selected, and information regarding the selected method may be signaled to the decoder. Information regarding the selected method may be expressed in the form of a predetermined flag or index.

[0225] Meanwhile, in the secondary prediction method after intra-block copying, a predetermined region and a template may have the same meaning.

[0226]

[0227] FIG. 7 is a diagram illustrating a secondary prediction method after a primary prediction using a template according to an embodiment of the present disclosure. Here, the primary prediction using a template may refer to a method of performing a prediction of the current block using a template of the current block, such as matrix-based intra prediction or neural network-based intra prediction.

[0228] In FIG. 7, the top area of ​​the current block (700) can be referred to as the top template (710), and the left area can be referred to as the left template (720). The top template and the left template of the current block can be referred to as the current template.

[0229] Referring to FIG. 7, a prediction block of the current block can be derived using an upper template (710) and / or a left template (720). For example, in the case of matrix-based intra prediction, a predefined weight matrix can be applied to the upper template and / or the left template to generate a prediction block of the current block. For another example, in the case of neural network-based intra prediction, pixels of the upper template and / or pixels of the left template can be used as inputs to a neural network model to generate a prediction block of the current block.

[0230] And, the residual of the current block can be obtained based on the difference between the current block (700) and the prediction block. Since spatial redundancy may still exist in the residual, the residual of the template can be obtained to remove it.

[0231] Specifically, a predicted value of the top template can be generated by performing matrix-based intra prediction or neural network-based intra prediction on the top template (710). That is, a predicted value of the top template (710) can be generated by applying a predefined weight matrix to neighboring pixels of the top template (711), or by using the neighboring pixels (711) of the top template as input to a neural network model. Similarly, a predicted value of the left template can be generated by performing matrix-based intra prediction or neural network-based intra prediction on the left template (720). That is, a predicted value of the left template (720) can be generated by applying a predefined weight matrix to neighboring pixels of the left template (721), or by using the neighboring pixels (721) of the left template as input to a neural network model.

[0232] Then, the residual of the upper template can be obtained based on the difference between the predicted value of the upper template and the upper template. Similarly, the residual of the left template can be obtained based on the difference between the predicted value of the left template and the left template. Subsequently, a prediction can be performed based on the residual of the upper template and / or the residual of the left template.

[0233] According to one embodiment of the present disclosure, a secondary prediction may be performed by performing a directional and / or non-directional intra prediction on all or part of the residual samples of the current template.

[0234] In the encoder, prediction can be performed for all or part of the residual samples of the upper template and / or the residual samples of the left template based on a predetermined number of directional and / or non-directional intra-prediction modes.

[0235] Additionally, an intra prediction mode having the lowest cost value based on a predetermined cost function may be selected. Specifically, for each intra prediction mode, a cost such as a bit rate-distortion cost is calculated, and the intra prediction mode with the smallest distortion may be selected. Here, having the smallest distortion may mean that the distortion with respect to the residual of the current block is the smallest, or that the distortion with respect to the residual of the upper template and / or the residual of the left template of the current block is the smallest.

[0236] And, based on the selected prediction mode, a prediction can be performed to obtain residual prediction samples.

[0237] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0238] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0239] Meanwhile, the selected prediction mode information and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0240] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0241] Additionally, in the decoder, predictions are performed on the residuals of the upper template and / or the residuals of the left template according to the selected prediction mode information to derive residual prediction samples, and these can be combined with the secondary residual samples to derive primary residual samples.

[0242] In addition, a first prediction using a template can be performed in the decoder to derive a prediction sample. Subsequently, a recovery sample of the current block can be derived by summing the first residual sample and the prediction sample.

[0243] Meanwhile, information regarding the selected prediction mode may be expressed in the form of a specific flag or index.

[0244] Meanwhile, the intra prediction mode applicable in the secondary prediction according to Fig. 5 can also be applied in this embodiment.

[0245] Meanwhile, the aforementioned predetermined number of directional and / or non-directional intra-prediction modes may be defined differently or identically depending on the size of the current block.

[0246] According to one embodiment of the present disclosure, a second prediction may be performed by performing matrix-based intra prediction on all or part of the residual samples of the upper template and / or the residual samples of the left template. Here, matrix-based intra prediction may refer to a method of performing prediction using a predefined weight matrix.

[0247] In the encoder, prediction can be performed by applying a predetermined number of weight matrices to all or part of the residual samples of the upper template and / or the residual samples of the left template.

[0248] Additionally, a weight matrix having the lowest cost value can be selected based on a predetermined cost function. Specifically, for each weight matrix, a cost such as a bit rate-distortion cost is calculated, and the weight matrix with the smallest distortion can be selected. Here, having the smallest distortion may mean that the distortion with respect to the residual of the current block is the smallest, or that the distortion with respect to the residual of the upper template and / or the residual of the left template of the current block is the smallest.

[0249] And, matrix-based intra-prediction is performed based on the selected weight matrix, and residual prediction samples can be obtained.

[0250] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0251] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0252] Meanwhile, information regarding the selected weight matrix and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0253] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0254] In addition, the decoder performs matrix-based intra-prediction based on information regarding the selected weight matrix to derive residual prediction samples, and these can be combined with secondary residual samples to derive primary residual samples.

[0255] In addition, a first prediction using a template can be performed in the decoder to derive a prediction sample. Subsequently, a recovery sample of the current block can be derived by summing the first residual sample and the prediction sample.

[0256] Meanwhile, information regarding the selected weight matrix can be expressed in the form of a specific flag or index.

[0257] Meanwhile, the aforementioned predetermined number of weight matrices may be defined differently or identically depending on the size of the current block, quantization parameters (QP), etc.

[0258] According to one embodiment of the present disclosure, a second prediction may be performed by performing a neural network-based intra prediction on all or part of the residual samples of the upper template and / or the residual samples of the left template. Here, the neural network-based intra prediction may refer to a method of performing a prediction using a predefined neural network model.

[0259] In the encoder, neural network-based intra prediction can be performed by applying a predetermined number of neural network models to all or part of the residual samples of the upper template and / or the residual samples of the left template.

[0260] Additionally, a neural network model having the lowest cost value can be selected based on a predetermined cost function. Specifically, for each neural network model, a cost such as a bit rate-distortion cost is calculated, and the neural network model with the smallest distortion can be selected. Here, having the smallest distortion may mean that the distortion with respect to the residual of the current block is the smallest, or that the distortion with respect to the residual of the upper template and / or the residual of the left template of the current block is the smallest.

[0261] And, based on the selected neural network model, predictions can be performed to derive residual prediction samples.

[0262] And, based on the selected neural network model, neural network-based intra-prediction is performed on the residuals of the top template and / or the residuals of the left template to obtain residual prediction samples.

[0263] In addition, residual samples after secondary prediction can be obtained based on the residual samples and residual prediction samples of the current block. For example, residual samples after secondary prediction can be calculated based on the difference between the residual samples and residual prediction samples of the current block.

[0264] Meanwhile, the first residual sample may refer to the residual sample of the current block, and the second residual sample may refer to the residual prediction sample differed from the first residual sample.

[0265] Meanwhile, information regarding the selected neural network model and secondary residual samples can be signaled from the encoder to the decoder. At this time, the secondary residual samples can be transmitted after being transformed, quantized, and entropy encoded.

[0266] In the decoder, entropy decoding and inverse quantization are performed on the bitstream to recover the transform coefficients, and an inverse transform is applied to the transform coefficients to derive secondary residual samples.

[0267] In addition, in the decoder, neural network-based intra-prediction is performed according to information regarding the selected neural network model to derive residual prediction samples, and these can be combined with secondary residual samples to derive primary residual samples.

[0268] In addition, a first prediction using a template can be performed in the decoder to derive a prediction sample. Subsequently, a recovery sample of the current block can be derived by summing the first residual sample and the prediction sample.

[0269] Meanwhile, information regarding the selected neural network model may be expressed in the form of a specific flag or index.

[0270] Meanwhile, the aforementioned predetermined number of neural network models may be defined differently or identically depending on the size of the current block, quantization parameters (QP), etc.

[0271] Meanwhile, the shape and size of the template in Fig. 7 are just one example, and the size and shape of the template, as well as the range and number of surrounding pixels of the template, can be determined in various ways depending on the protocol between the encoder and decoder, the block size, related syntax elements, etc.

[0272] Meanwhile, in the embodiments relating to the secondary prediction method following the primary prediction using the aforementioned template, a single method may be performed, or one or more methods may be performed in the encoder. When one or more methods are used, the method having the lowest cost value based on a predetermined cost function in the encoder is selected, and information regarding the selected method may be signaled to the decoder. Information regarding the selected method may be expressed in the form of a predetermined flag or index.

[0273]

[0274] FIG. 8 is a diagram illustrating a secondary prediction method after multiple intra prediction according to one embodiment of the present disclosure. Here, multiple intra prediction refers to a method of improving prediction accuracy by combining multiple intra prediction results.

[0275] Figure 8 assumes a case where a second prediction is performed after multiple intra template matching. Multiple intra template matching may refer to a method of combining multiple intra template matching results.

[0276] In FIG. 8, the adjacent L-shaped regions of the current block (800) (i.e., the left, top, and top-left regions) can be defined as the current template (801).

[0277] Additionally, within a predefined search range of the restored area of ​​the current picture, a predetermined number of reference templates similar to the current template (801) may be searched. FIG. 8 assumes the case where three reference templates are searched: reference template 1 (811), reference template 2 (821), and reference template 3 (831).

[0278] And, the residual of the current block (840) can be derived based on the matching block corresponding to the current block in each reference template. Referring to FIG. 8, the residual of the current block can be derived based on the matching block 1 (810) corresponding to the current block in reference template 1 (811), the matching block 2 (820) corresponding to the current block in reference template 2 (821), and the matching block 3 (830) corresponding to the current block in reference template 3 (831).

[0279] Meanwhile, the residual of the current block can be derived as shown in Equation 2.

[0280]

[0281]

[0282] R in mathematical equation 2 BC represents the residual of the current block. And, B MK (k=0, 1.., n-1) represents the matching block corresponding to the current block in each reference template. And, B C represents the current block. Therefore, in duplicate intra-template matching, the residual of the current block can be obtained based on the difference between the current block and the average value of multiple matching blocks.

[0283] In second-order prediction, spatial redundancy may still exist in the residuals of the current block, so to eliminate this, the residual of the current template (residue of current template, 841) may be additionally obtained based on the difference between the current template and the reference template.

[0284] According to one embodiment of the present disclosure, in a secondary prediction method after redundant intra prediction, the residual of the current template can be derived using a plurality of reference templates.

[0285] For example, the residual of the current template can be derived using Equation 3.

[0286]

[0287]

[0288] T in mathematical equation 3 BC represents the residual of the current template. And, T RK (k=0, 1.., n-1) represents each reference template. Therefore, in duplicate intra-template matching, the residual of the current template is the average value (T) of the current template (Tc) and the reference templates. R It can be obtained based on the difference between ).

[0289] Meanwhile, Equation 3 derives the residual of the current template based on the average value of all reference templates, but this is just one example, and among multiple reference templates, the template most efficient for quadratic prediction can be selected and used.

[0290] Meanwhile, information regarding the selected reference template can be explicitly transmitted from the encoder to the decoder. Here, the information regarding the reference template can be expressed in the form of a template number, a specific flag, an index, etc.

[0291] Meanwhile, the similarity between each reference template and the current template can be calculated using the Sum of Absolute Differences (SAD) method or the Sum of Absolute Transformed Differences (SATD) method, and a predetermined number of reference templates may be implicitly selected in order of lowest similarity.

[0292] Meanwhile, when residuals of the current template are derived, a second prediction may be performed based on all or part of the residual samples of the current template, and residual prediction samples may be obtained. At this time, the method for performing the second prediction may be performed using a predetermined intra prediction mode, matrix-based intra prediction, neural network-based intra prediction, etc., as in the embodiments described above.

[0293] Meanwhile, the secondary prediction method after duplicate intra prediction in Fig. 8 may be a secondary prediction after duplicate intra template matching or a secondary prediction after multiple intra block copy (multiple IBC). Here, multiple intra block copy may refer to a method of combining multiple intra block copy results.

[0294] Meanwhile, Figure 8 uses three reference templates, but this is just one example, and a predetermined number of intra-prediction results may be used.

[0295]

[0296] In this specification, a transform kernel may refer to a transform core used when applying a transform from a spatial domain to a frequency domain. Additionally, a transform set may refer to a group containing transform kernels. Furthermore, a transform set may refer to a kernel cluster.

[0297] A secondary transform refers to a transformation performed based on the correlation of the coefficients generated by the primary transform after the primary transform from the spatial domain to the frequency domain has been performed. Here, the primary transform and the primary transform may have the same meaning. By performing a secondary transform that converts to a more compressed representation than the primary transform, high-performance compression efficiency can be achieved compared to when only the primary transform is performed.

[0298] The first and second transformations can be performed as separable transforms or non-separable transforms.

[0299] A separate transformation can refer to a transformation in which the vertical transformation and the horizontal transformation are performed independently. Here, the vertical transformation refers to a transformation in the vertical direction to which a vertical transformation kernel is applied. Similarly, the horizontal transformation refers to a transformation in the horizontal direction to which a horizontal transformation kernel is applied. For example, in a separate transformation, the vertical transformation may be performed after the horizontal transformation has been performed. Alternatively, the horizontal transformation may be performed after the vertical transformation has been performed.

[0300] Inseparable transformation can mean that the transformation is performed in a single step, rather than in the horizontal and vertical directions, using an inseparable transformation kernel in the form of a matrix.

[0301] In separate and inseparable transformations, basis vectors or combinations of basis vectors may be used. Basis vectors are vectors of size corresponding to pixels or coefficients in the 2D space where the transformation is performed, and can be designed by considering the overall characteristics and similarities of the pixels and / or coefficients in the 2D space where the transformation is performed.

[0302] For second-order transforms such as LFNST (Low frequency non-separable transform), various sizes of non-separable second-order transform kernels can be used to improve coding efficiency and transform complexity for specific input block sizes.

[0303] In the case of NSPT (Non-separable primary transform), although it is a primary transform, various sizes of non-separable primary transform kernels can be used for specific input block sizes to improve coding efficiency and transform complexity, similar to LFNST.

[0304] MTS (Multiple transform selection) may mean using a selected transform kernel among multiple transform kernels for the transformation of the current block.

[0305] MTSS (Multiple transform set selection) may mean that a transform set is selected from among multiple transform sets, and a selected transform kernel from among multiple transform kernels included in the selected transform set is used for the transformation of the current block. In this case, the transform kernel may be selected after the transform set is selected, or the selection of the transform set and the selection of the transform kernel may be performed simultaneously.

[0306] The characteristics of the residual signal of a block on which a second-order prediction was performed may differ from the characteristics of the residual signal of a block on which only a first-order prediction was performed.

[0307] Hereinafter, the present specification provides a method for efficiently deriving a virtual intra prediction mode (VIM) for determining a transformation set / transform kernel of a residual block after a second prediction. Through this, a transformation set and / or transformation kernel suitable for the characteristics of the pixel and / or coefficient to be coded can be selected, and transformation efficiency can be improved.

[0308] Meanwhile, through a virtual intra-prediction mode derivation method, in the aforementioned embodiments, the secondary residual sample in the encoder is transformed and then transmitted after undergoing quantization and entropy encoding, and in the decoder, entropy decoding and inverse quantization are performed on the bitstream to restore the transformation coefficients, and the secondary residual sample can be derived by applying an inverse transformation to the transformation coefficients.

[0309] Meanwhile, a method for deriving a virtual intra-predicted mode may refer to a method for deriving an intra-predicted mode used when determining the set of transformations for the transformation / inverse transformation of the current block among multiple sets of transformations. Additionally, it may refer to a method for deriving an intra-predicted mode used when determining the transformation kernel for the transformation / inverse transformation of the current block among multiple transformation kernels. Meanwhile, the transformation kernel may refer to a transformation kernel in a separable transformation and / or a non-separable transformation.

[0310] Meanwhile, the virtual intra-prediction mode induction method and the virtual intra-prediction mode designation method may have the same meaning.

[0311] Meanwhile, in the method for deriving a virtual intra-prediction mode according to an embodiment of the present disclosure, the current block may refer to a residual block after a secondary prediction.

[0312]

[0313] According to one embodiment of the present disclosure, an intra prediction mode for determining a transformation kernel can be derived using the gradient of a residual prediction sample or the gradient of a surrounding sample of the current block. Specifically, an intra prediction mode for determining a transformation kernel is derived based on directional information of all or part of the residual prediction samples or all or part of the surrounding samples of the current block, which are prediction samples obtained by performing a second prediction, and a transformation kernel of the current block can be determined based on the derived intra prediction mode.

[0314] Meanwhile, the surrounding samples of the current block may include adjacent samples and non-adjacent samples of the current block. Here, non-adjacent samples may refer to samples included in a defined area in the encoder and decoder, or samples in an area indicated by the additional information of the current block.

[0315] Meanwhile, directional information may refer to information obtained by calculating the gradients of pixels. Specifically, directional information may refer to information obtained by accumulating or processing gradient information calculated by applying filters to the corresponding pixels.

[0316] First, the gradients of the residual prediction samples or the surrounding samples of the current block can be calculated. At this time, the applied filter may be a boundary detection filter such as a Sobel filter, a Roberts cross filter, a Prewitt filter, a Scharr filter, and a Laplacian filter.

[0317] And, a histogram of gradient can be generated based on the calculated gradient.

[0318] Meanwhile, the slopes of only some of the residual prediction samples or surrounding samples of the current block can be calculated, as well as all of them. In this case, a slope histogram can be generated based on the slopes of only some of them.

[0319] In addition, an intra-prediction mode for determining the transformation kernel can be derived using the gradient histogram.

[0320] For example, an intra-prediction mode for determining a transformation kernel can be derived from an intra-prediction mode corresponding to the most accumulated directionality in the gradient histogram. Here, most accumulated may mean that the amplitude is the largest in the histogram, and the largest amplitude may mean that it has the largest value in the histogram.

[0321] As another example, the intra prediction mode for determining the transformation kernel can be derived from any one of N prediction modes corresponding to the most accumulated directionality in the gradient histogram. In this case, the N prediction modes can be referred to as candidate modes. Here, N is an arbitrary positive integer.

[0322] First, the bit rate-distortion cost can be calculated for each candidate mode in the encoder.

[0323] And, the intra prediction mode for determining the transformation kernel can be driven to the intra prediction mode with the smallest cost value among the candidate modes.

[0324] In addition, information regarding the intra-prediction mode for determining the transformation kernel may be signaled to the decoder. Here, the information regarding the intra-prediction mode for determining the transformation kernel may include at least one of a flag regarding whether the intra-prediction mode for determining the transformation kernel is used, a transformation set index, a transformation kernel index, and an index regarding the intra-prediction mode for determining the transformation kernel.

[0325] Furthermore, in the decoder, the transformation set and / or transformation kernel of the current block can be determined based on information regarding the intra-prediction mode for determining the transformation kernel. Then, based on the determined transformation kernel, an inverse transformation can be performed on the transformation coefficients obtained from the bitstream. Through this, residual samples can be obtained.

[0326] Meanwhile, in the example described above, the cost value is calculated as the bit rate-distortion cost, but this is just one example, and the cost value can be calculated by a predetermined cost function.

[0327] As another example, the intra prediction mode for determining the transformation kernel can be derived from the intra prediction mode with the smallest assigned prediction mode number among the N prediction modes corresponding to the most accumulated directionality in the gradient histogram. Here, N is an arbitrary positive integer.

[0328] As another example, the intra prediction mode for determining the transformation kernel can be derived from the intra prediction mode with the largest assigned prediction mode number among the N prediction modes corresponding to the most accumulated directionality in the gradient histogram. Here, N is an arbitrary positive integer.

[0329] As another example, if the amplitudes of the slope histograms are all below a specific value, the intra prediction mode for determining the transform kernel can be determined as a predetermined intra prediction mode. Here, the specific value and the predetermined intra prediction mode may be determined by an agreement between the encoder and the decoder, or may be determined by the encoder and transmitted to the decoder. For example, the predetermined intra prediction mode may be either Planar mode or DC mode.

[0330]

[0331] According to one embodiment of the present disclosure, an intra prediction mode for determining a transformation kernel can be derived using a histogram of occurrence (HoC). Accordingly, when an intra prediction mode for determining a transformation kernel is derived by the histogram of occurrence, a transformation kernel is determined based on the intra prediction mode, and a transformation / inverse transformation can be performed based on the transformation kernel.

[0332] Meanwhile, in this embodiment, the frequency histogram may have the same meaning as the incidence rate histogram.

[0333] The occurrence frequency histogram can be generated based on the intra-prediction mode of the neighboring block of the current block where the transformation / inverse transformation is performed. Specifically, the occurrence frequency histogram can be generated by accumulating the intra-prediction modes of the neighboring blocks.

[0334] At this time, the intra prediction modes of surrounding blocks can be accumulated as is to generate an occurrence frequency histogram.

[0335] Alternatively, to generate an occurrence frequency histogram, intra-prediction modes of surrounding blocks can be accumulated based on the number of pixels of the current block.

[0336] Alternatively, to generate an occurrence frequency histogram, intra prediction modes of surrounding blocks can be accumulated based on the size of surrounding blocks.

[0337] For example, the intra-prediction mode of a neighboring block can be accumulated in proportion to the size of that neighboring block. That is, a value calculated by multiplying the size of that neighboring block by an arbitrary real number can be accumulated in the occurrence frequency histogram. Here, the arbitrary real number can be determined by an agreement between the encoder and the decoder.

[0338] Meanwhile, the size of the surrounding block may be determined by at least one of the width and height of the surrounding block. For example, the size of the surrounding block may refer to the value obtained by multiplying the width and height of the surrounding block. As another example, the intra-prediction mode of the surrounding block may be accumulated in the occurrence frequency histogram based on at least one of the width and height of the surrounding block.

[0339] Meanwhile, the size of the corresponding surrounding block can be determined by the number of pixels included in the corresponding surrounding block. For example, the intra-prediction mode of the surrounding block can be accumulated in proportion to the number of pixels included in the corresponding surrounding block. Specifically, a value calculated by multiplying the number of pixels included in the corresponding surrounding block by a predetermined real number can be accumulated in the occurrence frequency histogram. Here, the predetermined real number can be defined by the agreement between the encoder and the decoder.

[0340] Meanwhile, the number of pixels included in the corresponding surrounding block and the number of samples in the corresponding surrounding block may have the same meaning. For example, the intra prediction mode of the surrounding block can be accumulated in proportion to the number of samples in the corresponding surrounding block.

[0341] Meanwhile, if a neighboring block has two or more intra-prediction modes, such as SGPM mode, TIMD mode, or DIMD mode, the amplitude value of each of the said intra-prediction modes may be accumulated in the occurrence frequency histogram. In this case, it may be accumulated as is in the occurrence frequency histogram, or it may be accumulated by a value calculated by multiplying the number of pixels included in the neighboring block by an arbitrary real number. Alternatively, it may be accumulated by a value calculated by multiplying the number of pixels included in the current block by an arbitrary real number.

[0342] Meanwhile, any real number can be determined by the agreement between the encoder and decoder, or determined by the encoder and signaled to the decoder.

[0343] Meanwhile, in this embodiment, the neighboring blocks may include adjacent neighboring blocks to the current block and non-adjacent neighboring blocks to the current block. Here, being adjacent to the current block may mean being spatially adjacent to the current block.

[0344] Here, neighboring blocks adjacent to the current block may be referred to as adjacent blocks, and neighboring blocks not adjacent to the current block may be referred to as non-adjacent blocks. Non-adjacent blocks may be included in a predetermined area set by the encoder and decoder, or in any area indicated by the additional information of the current block.

[0345] Therefore, the occurrence frequency histogram can be generated using the intra-prediction mode of adjacent and non-adjacent blocks.

[0346] And, when an occurrence frequency histogram is generated, an intra-prediction mode for determining a transformation kernel can be derived based on the generated occurrence frequency histogram.

[0347] For example, an intra-prediction mode for determining a transformation kernel can be derived from the intra-prediction mode having the largest value in the frequency histogram. Here, having the largest value means having the largest amplitude in the histogram, which can mean the most accumulated.

[0348] As another example, an intra prediction mode for determining the transformation kernel can be derived using the M intra prediction modes with the highest accumulation among the intra prediction modes of the occurrence frequency histogram. Specifically, the intra prediction mode for determining the transformation kernel can be derived from either the current block's intra prediction mode or the aforementioned M intra prediction modes. In this case, the aforementioned current block's intra prediction mode and the M intra prediction modes can be referred to as candidate modes. Here, M is an arbitrary positive integer.

[0349] First, the rate-distortion cost (RD cost) can be calculated for each candidate mode in the encoder.

[0350] And, the intra prediction mode for determining the transformation kernel can be derived from the intra prediction mode with the smallest distortion among the candidate modes.

[0351] And, information regarding the intra prediction mode can be signaled to a decoder. Here, the information regarding the intra prediction mode for determining the transformation kernel may include at least one of a flag regarding whether the intra prediction mode for determining the transformation kernel is used, a transformation set index, a transformation kernel index, and an index regarding the intra prediction mode for determining the transformation kernel.

[0352] And, in the decoder, the conversion kernel of the current block can be determined based on information about the intra-prediction mode for determining the conversion kernel.

[0353] Meanwhile, in the example described above, the cost value is calculated as the bit rate-distortion cost, but this is just one example, and the cost value can be calculated by a predetermined cost function.

[0354] As another example, the intra prediction mode for determining the transformation kernel can be derived from the intra prediction mode with the smallest assigned prediction mode number among the M intra prediction modes with the highest accumulation in the frequency histogram. Here, M is an arbitrary positive integer.

[0355] As another example, the intra prediction mode for determining the transformation kernel can be derived from the intra prediction mode with the largest assigned prediction mode number among the M prediction modes most accumulated in the frequency histogram intra prediction modes. Here, M is an arbitrary positive integer.

[0356] Meanwhile, in the example described above, the M intra prediction modes are selected in order of highest accumulation in the frequency histogram, but the intra prediction mode for determining the transformation kernel can be derived based on the M intra prediction modes randomly determined in the frequency histogram.

[0357] As another example, if the amplitudes of the occurrence frequency histograms are all below a specific value, the intra prediction mode for determining the transform kernel can be determined as a predetermined intra prediction mode. Here, the specific value and the predetermined intra prediction mode may be determined by an agreement between the encoder and the decoder, or may be determined by the encoder and transmitted to the decoder. As an example, the intra prediction mode for determining the transform kernel can be determined as either Planar mode or DC mode.

[0358] Meanwhile, the method for deriving an intra prediction mode for determining a conversion kernel according to the present embodiment may be referred to as an Occurrence-Based Intra Coding (OBIC) method. Here, the OBIC method may refer to a method for deriving an intra prediction mode of the current block based on the frequency of occurrence of the intra prediction mode of surrounding blocks.

[0359]

[0360] According to one embodiment of the present disclosure, an intra prediction mode for determining a conversion kernel can be determined as a predetermined intra prediction mode.

[0361] The higher the accuracy of the second prediction, the more random the residual samples after the second prediction may be. Here, having randomness means that directionality is largely eliminated and the samples do not possess a specific direction. Therefore, the intra prediction mode for determining the transform kernel can be determined as a specific prediction mode, such as Planar mode or DC mode. The specific prediction mode can be determined by an agreement between the encoder and the decoder.

[0362]

[0363] Meanwhile, if an intra-prediction mode for determining a transformation kernel is derived, the transformation kernel of the current block can be determined based on the derived intra-prediction mode for determining the transformation kernel. Then, the transformation / inverse transformation of the current block can be performed based on the determined transformation kernel.

[0364] According to one embodiment, first, a plurality of intra-prediction modes may be grouped and mapped to a transformation set for each group. At this time, the method by which the intra-prediction modes are mapped to the transformation set may be determined by an agreement between the encoder and decoder. For example, this may be performed in a manner such as LFNST and NSPT.

[0365] And, the transformation set of the current block can be determined as a transformation set mapped to an intra-prediction mode for determining the transformation kernel. And, the transformation kernel of the current block can be determined as any one of the transformation kernels included in the determined transformation set.

[0366] For example, the transform kernel of the current block can be determined as the transform kernel with the smallest cost value among the transform kernels included in the determined transform set. Specifically, the bit rate-distortion cost (RD cost) is calculated for each transform kernel in the encoder, and the kernel with the smallest distortion can be selected as the optimal transform kernel. Then, information regarding the optimal transform kernel can be signaled to the decoder. Here, the information regarding the optimal transform kernel may be a flag if the number of transform kernels included in the transform set is two, and a kernel index if there are three or more. In the decoder, the transform kernel is determined based on this information, and the inverse transform can be performed based on the determined transform kernel.

[0367] Meanwhile, according to the above-described embodiment, the transformation kernel with the smallest cost value among the transformation kernels included in the transformation set is determined as the transformation kernel of the current block; however, this is merely an example, and the transformation kernel of the current block may be determined as any one of the transformation kernels included in the transformation set determined by any method. In this case, the arbitrary method may be determined by an agreement between the encoder and the decoder, or information regarding the determined transformation kernel may be signaled.

[0368] According to one embodiment, multiple transformation kernels may exist without a transformation set. In this case, intra-prediction modes may be grouped and mapped to multiple transformation kernels for each group. Furthermore, the method by which intra-prediction modes are mapped to transformation kernels may be determined by an agreement between the encoder and decoder.

[0369] And, when an intra-prediction mode for determining the transformation kernel is derived, the transformation kernel of the current block can be determined based on the derived prediction mode. Specifically, the transformation kernel of the current block can be determined as a transformation kernel mapped to the intra-prediction mode for determining the transformation kernel.

[0370] And, based on the determined conversion kernel, the conversion / inverse conversion of the current block can be performed.

[0371]

[0372] According to one embodiment of the present disclosure, whether the aforementioned secondary prediction method is activated can be determined by secondary prediction activation information. Here, the secondary prediction activation information is information indicating whether the secondary prediction is activated, and may be expressed as an activation flag, an inactivation flag, a predetermined syntax element, etc.

[0373] Specifically, when secondary prediction activation information is explicitly signaled and the decoder indicates that the information is activated, the secondary prediction according to the aforementioned embodiments can be activated.

[0374] Meanwhile, secondary prediction activation information may be signaled at a high level such as a video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), picture header (PH), and slice header (SH), or at a lower level such as a coding tree unit (CTU), coding unit (CU), prediction unit (PU), and transform unit (TU).

[0375] Meanwhile, secondary predictive activation information can be signaled independently of other syntactic elements. Alternatively, secondary predictive activation information can be signaled dependently on other syntactic elements.

[0376] According to one embodiment of the present disclosure, the aforementioned secondary prediction method can be activated without separate signaling when a predetermined condition is satisfied.

[0377] Meanwhile, a predetermined condition may mean that the prediction mode of the current block is a predetermined intra prediction mode. Specifically, if the prediction mode of the current block is a predetermined intra prediction mode, a secondary prediction method may be activated without separate signaling.

[0378] For example, if the prediction mode of the current block is either the intra-template matching mode or the intra-block copying mode, a secondary prediction method can be activated in the decoder without separate signaling.

[0379] Meanwhile, a predetermined condition may refer to the case where the current block size is a predetermined size. Specifically, when the current block size is a predetermined size, a secondary prediction method may be activated without separate signaling.

[0380] For example, for small blocks that are difficult to predict compared to large blocks, a secondary prediction method can be activated without separate signaling. The size of the block for which the secondary prediction method is activated can be determined by an agreement between the encoder and decoder.

[0381] Meanwhile, a predetermined condition may be that the value of the quantization parameter (QP) corresponding to the current block is a predetermined value. Specifically, when the value of the QP corresponding to the current block is a predetermined value, a secondary prediction method may be activated without separate signaling.

[0382] For example, for blocks with low QPs that are difficult to predict compared to blocks with high QPs, a secondary prediction method can be activated without separate signaling. The QP value at which the secondary prediction method is activated can be determined by an agreement between the encoder and decoder.

[0383] Meanwhile, in this specification, information regarding secondary prediction activation and information regarding whether residual prediction is performed may have the same meaning.

[0384]

[0385] FIG. 9 is a flowchart illustrating a decoding method according to one embodiment of the present disclosure. The decoding method of FIG. 9 can be performed by an image decoding device.

[0386] The video decoder can obtain conversion coefficients from the bitstream (S900).

[0387] And, the video decoder can perform a prediction on the current block to derive a prediction sample of the current block (S910).

[0388] And, the image decoder can obtain a residual sample of the current block by performing an inverse transform on the transform coefficients (S920).

[0389] Meanwhile, the step of obtaining residual samples of the current block includes the step of deriving an intra-prediction mode for determining a transformation kernel of the current block and the step of determining a transformation kernel based on the intra-prediction mode for determining the transformation kernel, and an inverse transformation may be performed on the transformation coefficients based on the determined transformation kernel.

[0390] Meanwhile, the intra prediction mode for determining the transformation kernel above can be derived based on either the directional information of the residual prediction sample or the directional information of the sample included in the surrounding region for the current block.

[0391] Meanwhile, the intra prediction mode for determining the transformation kernel is derived using an occurrence frequency histogram, and the occurrence frequency histogram can be generated by accumulating the intra prediction modes of the surrounding blocks of the current block.

[0392] Meanwhile, the intra prediction mode for determining the transformation kernel above can be derived into an intra prediction mode having a maximum value in the occurrence frequency histogram.

[0393] Meanwhile, the intra prediction mode of the surrounding block can be accumulated in the occurrence frequency histogram based on the size of the surrounding block.

[0394] Meanwhile, the surrounding blocks may include blocks spatially adjacent to the current block and blocks not spatially adjacent to the current block.

[0395] Meanwhile, the intra prediction mode for determining the above-mentioned conversion kernel can be derived into either DC mode or planner mode.

[0396] Meanwhile, the step of determining the transformation kernel may include the step of determining a transformation set of the current block based on an intra-prediction mode for determining the transformation kernel, and the step of determining a transformation kernel of the current block among the transformation kernels included in the transformation set.

[0397] And, the image decoder can derive a residual prediction sample of the current block (S930).

[0398] Meanwhile, the above residual prediction sample can be derived based on the template of the current block.

[0399] Meanwhile, the prediction sample of the current block is derived using the matching block of the current block, and the residual prediction sample can be derived based on the template of the current block and the template of the matching block.

[0400] Meanwhile, the prediction mode of the current block is determined as either a matrix-based intra prediction mode or a neural network-based intra prediction mode, and the residual prediction sample can be derived by performing a prediction on the sample included in the template of the current block.

[0401] Meanwhile, the template of the current block may include at least one of a sample included in the upper area of ​​the current block and a sample included in the left area of ​​the current block.

[0402] Meanwhile, the method further includes a step of obtaining information regarding whether to perform residual prediction from a bitstream, and the residual prediction sample of the current block can be derived based on the information.

[0403] Meanwhile, the step of deriving residual prediction samples for the current block can be performed based on the size of the current block.

[0404] Meanwhile, the prediction sample of the current block is derived using a plurality of matching blocks, and the residual prediction sample can be derived based on the template of the plurality of matching blocks.

[0405] And, the image decoding device can generate a restoration sample of the current block based on the residual sample, the prediction sample and the residual prediction sample (S940).

[0406] Meanwhile, the steps described in FIG. 9 can be performed in the same way in a video encoding method. Additionally, a bitstream can be generated by a video encoding method including the steps described in FIG. 9. The bitstream can be stored on a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0407]

[0408] FIG. 10 is a flowchart illustrating an encoding method according to one embodiment of the present disclosure. The encoding method of FIG. 10 can be performed by an image encoding device.

[0409] First, the encoding device can perform a prediction on the current block to derive a prediction sample of the current block (S1000).

[0410] And, the encoding device can derive a residual prediction sample of the current block (S1010).

[0411] Meanwhile, the prediction sample of the current block is derived using the matching block of the current block, and the residual prediction sample can be derived based on the template of the current block and the template of the matching block.

[0412] Meanwhile, the prediction mode of the current block is determined as either a matrix-based intra prediction mode or a neural network-based intra prediction mode, and the residual prediction sample can be derived by performing a prediction on the sample included in the template of the current block.

[0413] Meanwhile, the template of the current block may include at least one of a sample included in the upper area of ​​the current block and a sample included in the left area of ​​the current block.

[0414] Meanwhile, it may further include a step for determining information regarding whether to perform residual prediction.

[0415] Meanwhile, the step of deriving residual prediction samples for the current block can be performed based on the size of the current block.

[0416] Meanwhile, the above residual prediction sample can be derived based on the template of the current block.

[0417] Meanwhile, the prediction sample of the current block is derived using a plurality of matching blocks, and the residual prediction sample can be derived based on the template of the plurality of matching blocks.

[0418] And, the encoding device can generate a residual sample of the current block based on the prediction sample and the residual prediction sample (S1020).

[0419] And, the encoding device can obtain a transformation coefficient by performing a transformation on the residual sample (S1030).

[0420] Meanwhile, the step of performing a transformation on the residual sample includes the step of deriving an intra-prediction mode for determining the transformation kernel of the current block and the step of determining the transformation kernel based on the intra-prediction mode for determining the transformation kernel, and the transformation can be performed on the residual sample based on the determined transformation kernel.

[0421] Meanwhile, the intra prediction mode for determining the transformation kernel above can be derived based on either the directional information of the residual prediction sample or the directional information of the sample included in the surrounding region for the current block.

[0422] Meanwhile, the intra prediction mode for determining the transformation kernel is derived using an occurrence frequency histogram, and the occurrence frequency histogram can be generated by accumulating the intra prediction modes of the surrounding blocks of the current block.

[0423] Meanwhile, the intra prediction mode for determining the transformation kernel above can be derived into an intra prediction mode having a maximum value in the occurrence frequency histogram.

[0424] Meanwhile, the intra prediction mode of the surrounding block can be accumulated in the occurrence frequency histogram based on the size of the surrounding block.

[0425] Meanwhile, the surrounding blocks may include blocks spatially adjacent to the current block and blocks not spatially adjacent to the current block.

[0426] Meanwhile, the intra prediction mode for determining the above-mentioned conversion kernel can be derived into either DC mode or planner mode.

[0427] Meanwhile, the step of determining the transformation kernel may include the step of determining a transformation set of the current block based on an intra-prediction mode for determining the transformation kernel, and the step of determining a transformation kernel of the current block among the transformation kernels included in the transformation set.

[0428] And, the encoding device can encode the conversion coefficients (S1040).

[0429] Meanwhile, a bitstream can be generated by a video encoding method including the steps described in FIG. 10. The bitstream can be stored on a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0430]

[0431] FIG. 11 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0432] As illustrated in FIG. 11, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0433] The encoding server described above compresses content input from multimedia input devices, such as smartphones, cameras, and CCTVs, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and CCTVs, generate the bitstream directly, the encoding server may be omitted.

[0434] The bitstream may be generated by a video encoding method and / or video encoding device to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0435] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0436] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0437] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0438] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0439]

[0440] The above embodiments may be performed in the same or a corresponding way in the encoding device and the decoding device. Additionally, an image may be encoded / decoded using at least one of the above embodiments or a combination of at least one.

[0441] The order in which the above embodiments are applied may differ between the encoding device and the decoder. Alternatively, the order in which the above embodiments are applied may be the same between the encoding device and the decoder.

[0442] The above embodiments may be performed for each of the luminance and chrominance signals. Alternatively, the above embodiments for the luminance and chrominance signals may be performed in the same way.

[0443] In the above embodiments, methods are described based on flowcharts as a series of steps or units; however, the present disclosure is not limited to the order of steps, and some steps may occur in a different order or simultaneously with other steps as described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, other steps may be included, or one or more steps of the flowcharts may be omitted without affecting the scope of the present disclosure.

[0444] The above embodiments may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present disclosure, or they may be those known and available to those skilled in the art of computer software.

[0445] The bitstream generated by the encoding method according to the above embodiment may be stored in a non-transient computer-readable recording medium. Additionally, the bitstream stored in the non-transient computer-readable recording medium may be decoded by the decoding method according to the above embodiment.

[0446] Herein, examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

[0447] Although the present disclosure has been described above with specific details such as specific components, limited embodiments, and drawings, this is provided only to aid in a more comprehensive understanding of the present disclosure and is not limited to the above embodiments, and a person skilled in the art to which the present disclosure belongs can make various modifications and variations from this description.

[0448] Accordingly, the scope of the present disclosure is not limited to the embodiments described above, and all things equivalent or equivalently modified to the claims set forth below, as well as the claims set forth below, shall be considered to be within the scope of the scope of the present disclosure.

[0449] The present disclosure may be used in an apparatus for encoding / decoding images and a recording medium storing a bitstream.

Claims

1. A step of obtaining conversion coefficients from a bitstream; A step of performing a prediction on the current block to derive a prediction sample of the current block; A step of obtaining residual samples of the current block by performing an inverse transformation on the above transformation coefficients; Step of deriving a residual prediction sample of the current block above; and The method includes the step of generating a restoration sample of the current block based on the above residual sample, the above prediction sample, and the above residual prediction sample, and An image decoding method characterized in that the above residual prediction sample is derived based on the template of the above current block.

2. In Paragraph 1, The prediction sample of the current block is derived using the matching block of the current block, and An image decoding method characterized in that the above residual prediction sample is derived based on the template of the current block and the template of the matching block.

3. In Paragraph 1, The prediction mode of the current block above is determined as either a matrix-based intra prediction mode or a neural network-based intra prediction mode, and An image decoding method characterized in that the above residual prediction sample is derived by performing a prediction on a sample included in the template of the current block.

4. In Paragraph 1, An image decoding method characterized in that the template of the current block includes at least one of a sample included in the upper region of the current block and a sample included in the left region of the current block.

5. In Paragraph 1, The step of obtaining a residual sample of the current block above is, Step of inducing an intra-prediction mode for determining the conversion kernel of the current block above; and The method includes the step of determining a transformation kernel based on an intra-prediction mode for determining the transformation kernel above, and An image decoding method characterized by performing an inverse transformation on the transformation coefficients based on the transformation kernel determined above.

6. In Paragraph 5, An image decoding method characterized in that the intra prediction mode for determining the transformation kernel is derived based on either the directional information of the residual prediction sample or the directional information of the sample included in the surrounding region for the current block.

7. In Paragraph 5, The intra-prediction mode for determining the above transformation kernel is derived using an occurrence frequency histogram, and An image decoding method characterized in that the above-mentioned frequency histogram is generated by accumulating the intra-prediction modes of the surrounding blocks of the current block.

8. In Paragraph 7, An image decoding method characterized in that the intra prediction mode for determining the above-mentioned transformation kernel is derived into an intra prediction mode having a maximum value in the above-mentioned frequency histogram.

9. In Paragraph 7, An image decoding method characterized by the intra-prediction mode of the surrounding block being accumulated in the occurrence frequency histogram based on the size of the surrounding block.

10. In Paragraph 7, An image decoding method characterized in that the surrounding blocks include blocks spatially adjacent to the current block and blocks not spatially adjacent to the current block.

11. In Paragraph 5, An image decoding method characterized in that the intra prediction mode for determining the above-mentioned conversion kernel is derived into either a DC mode or a planner mode.

12. In Paragraph 5, The step of determining the above conversion kernel is, A step of determining a transformation set of the current block based on an intra-prediction mode for determining the transformation kernel; and A video decoding method characterized by including the step of determining the transformation kernel of the current block among the transformation kernels included in the transformation set.

13. In Paragraph 1, It further includes a step of obtaining information regarding whether to perform residual prediction from the bitstream, and An image decoding method characterized in that the residual prediction sample of the current block is derived based on the information.

14. In Paragraph 1, An image decoding method characterized in that the step of deriving residual prediction samples for the current block is performed based on the size of the current block.

15. In Paragraph 1, The prediction sample of the current block above is derived using a plurality of matching blocks, and An image decoding method characterized in that the above residual prediction sample is derived based on a template of the above plurality of matching blocks.

16. A step of performing a prediction on the current block to derive a prediction sample of the current block; A step of deriving a residual prediction sample of the current block above; A step of generating a residual sample of the current block based on the above prediction sample and the above residual prediction sample; A step of obtaining transformation coefficients by performing a transformation on the above residual sample; and It includes the step of encoding the above conversion coefficients, An image encoding method characterized in that the above residual prediction sample is derived based on the template of the current block.

17. A bitstream transmission method generated by a video encoding method, The above transmission method includes the step of transmitting the bitstream, and The above image encoding method is, A step of performing a prediction on the current block to derive a prediction sample of the current block; A step of deriving a residual prediction sample of the current block above; A step of generating a residual sample of the current block based on the above prediction sample and the above residual prediction sample; A step of obtaining transformation coefficients by performing a transformation on the above residual sample; and It includes the step of encoding the above conversion coefficients, A transmission method characterized in that the above residual prediction sample is derived based on the template of the above current block.

Images