Method and apparatus for video encoding / decoding, and method for transmitting bitstream

The sub-partition-based prediction mode induction method addresses the inefficiencies in encoding and decoding high-resolution video by dividing blocks into sub-partitions and applying advanced prediction techniques, thereby reducing bandwidth and storage needs.

WO2026235291A1PCT designated stage Publication Date: 2026-11-12HYUNDAI MOTOR CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2026/005147
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-09
Filing Date
2026-04-02
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing technologies fail to efficiently address the challenges of high-resolution video encoding and compression, particularly in the context of wireless communication networks, and the need for improved methods to handle the increasing demand for high-efficiency video encoding and decoding, especially for high-resolution and high-quality video formats such as Ultra High Definition (UHD) video, which results in increased data volume and transmission/storage costs.

Method used

A sub-partition-based prediction mode induction method is employed, utilizing techniques like Matrix-based Intra Prediction (MIP), Neural-Network based Intra Prediction (NNIP), Extrapolation based Intra Prediction (EIP), Intra Template Matching Prediction (IntraTMP), Intra Block Copy (IBC), Decoder-side Intra Mode Derivation (DIMD), Template-based Intra Mode Derivation (TIMD), Overlapped Block Motion Compensation (OBMC), Spatial Geometric Partitioning Mode (SGPM), and Combined Inter-Intra Prediction (CIIP) to enhance encoding and decoding efficiency by subdividing prediction modes in high-resolution video processing.

Benefits of technology

This approach significantly reduces transmission bandwidth and storage space requirements by improving encoding and decoding efficiency, effectively managing the increased data volume associated with high-resolution video formats.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2026005147_12112026_PF_FP_ABST
    Figure KR2026005147_12112026_PF_FP_ABST
Patent Text Reader

Abstract

A method and an apparatus for video encoding / decoding, and a method for transmitting a bitstream are provided. The method for video decoding comprises the steps of: dividing a current block into a plurality of sub-partitions; deriving prediction modes for the plurality of sub-partitions, respectively; and performing prediction on the plurality of sub-partitions on the basis of the derived prediction modes, wherein the prediction modes may include at least one of a directional prediction mode, Matrix-based intra prediction (MIP), neural-network based intra prediction (NNIP), extrapolation based intra prediction (EIP), intra template matching prediction (IntraTMP), intra block copy (IBC), decoder-side intra mode derivation (DIMD), template-based intra mode derivation (TIMD), occurrence-based intra coding (OBIC), overlapped block motion compensation (OBMC), spatial geometric partitioning mode (SGPM), combined inter-intra prediction (CIIP), an intra merge mode, multiple reference line (MRL), and template-based MRL (TMRL).
Need to check novelty before this filing date? Find Prior Art

Description

Video encoding / decoding method, device, and method for transmitting a bitstream

[0001] The present invention relates to an image encoding / decoding method, an apparatus, and a method for transmitting a bitstream. Specifically, the present invention relates to an improved sub-partition-based prediction mode induction method.

[0002] Recently, the demand for high-resolution, high-quality video, such as Ultra High Definition (UHD) video, has been increasing across various application fields. As video data becomes higher in resolution and quality, the relative volume of data increases compared to conventional video data; consequently, transmission and storage costs increase when video data is transmitted using existing wired or wireless broadband lines or stored using existing storage media. To address these issues arising from the increase in video data resolution and quality, high-efficiency video encoding and decoding technologies for videos with higher resolution and quality are required. Accordingly, a sub-partition-based prediction mode induction method is required to improve encoding and decoding efficiency and further reduce transmission bandwidth and storage space by subdividing the prediction modes of the blocks constituting the picture during the encoding and decoding process of high-resolution video.

[0003] The present invention aims to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0004] In addition, the present invention aims to provide a method for transmitting a bitstream generated by an image encoding / decoding method or device according to the present invention.

[0005] A video decoding method according to an embodiment of the present invention comprises the steps of dividing a current block into a plurality of sub-partitions, deriving a prediction mode for each of the plurality of sub-partitions, and performing a prediction for the plurality of sub-partitions based on the derivation prediction mode, wherein the prediction mode is a directional prediction mode, MIP (Matrix-based Intra Prediction), NNIP (Neural-Network based Intra Prediction), EIP (Extrapolation based Intra Prediction), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), DIMD (Decoder-side Intra Mode Derivation), TIMD (Template-based Intra Mode Derivation), OBIC (Occurrence-Based Intra Coding), OBIC (Occurrence-Based Intra Coding), OBMC (Overlapped Block Motion Compensation), SGPM (Spatial Geometric Partitioning Mode), CIIP (Combined Inter-Intra Prediction), Intra Merge Mode, MRL (Multiple It may include at least one of a Reference Line and a Template-based MRL.

[0006] In a video decoding method, the current block is divided into a first sub-partition and a second sub-partition, a first prediction mode for the first sub-partition is induced, a prediction for the first sub-partition is performed based on the first prediction mode, a restoration for the first sub-partition is performed based on the predicted first sub-partition, a second prediction mode for the second sub-partition is induced based on the restored first sub-partition, a prediction for the second sub-partition is performed based on the second prediction mode, and a restoration for the second sub-partition is performed based on the predicted second sub-partition.

[0007] In a video decoding method, in response to two or more prediction modes being derived for a sub-partition, the prediction for the sub-partition can be performed based on prediction samples derived through two or more prediction modes.

[0008] In an image decoding method, a prediction mode for a subpartition can be derived based on reference samples adjacent to the subpartition.

[0009] In an image decoding method, a prediction mode for a subpartition can be derived based on the Histogram of Gradient (HoG) for the pixel values ​​of reference samples adjacent to the subpartition.

[0010] In the image decoding method, the prediction mode for the sub-partition may be at least one prediction mode determined from among the candidates for a predetermined prediction mode.

[0011] In the image decoding method, the predicted partition structure, the prediction mode for the sub-partition, can be derived using the Template-based Intra Mode Derivation (TIMD) method based on the template region adjacent to the sub-partition.

[0012] In an image decoding method, the prediction mode for a sub-partition can be derived based on the prediction mode of a reference block adjacent to the sub-partition.

[0013] In an image decoding method, the prediction mode for a sub-partition can be derived based on the Histogram of Occurrence (HoC) for the prediction mode of a reference block.

[0014] In a video decoding method, the current block is divided into a first sub-partition and a second sub-partition, and a prediction mode for the second sub-partition adjacent to the first sub-partition can be derived based on the prediction mode of the first sub-partition and the prediction mode of the reference block adjacent to the second sub-partition.

[0015] In an image decoding method, a prediction mode for a sub-partition is derived based on a prediction mode of a second reference block indicated by a block vector of a reference block adjacent to the sub-partition, and the prediction mode of the second reference block can be derived based on a Histogram of Gradient (HoG) for the pixel values ​​of reference samples adjacent to the second reference block.

[0016] In an image decoding method, a prediction mode for a sub-partition can be derived based on reference samples adjacent to the current block.

[0017] In a video decoding method, the current block may include at least one sub-partition predicted based on information indicating a signaled prediction mode.

[0018] In a video decoding method, the current block is divided into a first sub-partition and a second sub-partition, and further includes the step of performing blending for the first sub-partition and the second sub-partition, wherein the step of performing blending for the sub-partitions may perform blending in an area where the first extended sub-partition, which is an extension of the first sub-partition, and the second extended sub-partition, which is an extension of the second sub-partition, overlap.

[0019] In the image decoding method, blending is performed in response to the fulfillment of a blending execution condition based on the coding information of a first sub-partition and a second sub-partition, and the blending execution condition may be different between the prediction mode applied to the first sub-partition and the prediction mode applied to the second sub-partition.

[0020] A video decoding method further comprises the step of performing filtering on sub-partitions, wherein the sub-partitions include a first sub-partition and a second sub-partition, and the step of performing filtering on the sub-partitions comprises performing filtering on samples adjacent to the boundary between the first sub-partition and the second sub-partition, wherein the filtering area is n times the height of the first sub-partition and m times the width of the first sub-partition, and n and m may be real numbers greater than or equal to 0.

[0021] In the image decoding method, filtering can be performed in response to the prediction mode applied to the first sub-partition and the prediction mode applied to the second sub-partition being different.

[0022] In the image decoding method, filtering can be performed based on at least one of PDPC (Position Dependent Prediction Combination), ALF (Adaptive Loop Filter), SAO (Sample Adaptive Offset), and deblocking filter.

[0023] A video encoding method according to an embodiment of the present invention comprises the steps of dividing a current block into a plurality of sub-partitions, deriving a prediction mode for each of the plurality of sub-partitions, and performing a prediction for each of the plurality of sub-partitions based on the derivation prediction mode, wherein the prediction mode includes a directional prediction mode, MIP (Matrix-based Intra Prediction), NNIP (Neural-Network based Intra Prediction), EIP (Extrapolation based Intra Prediction), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), DIMD (Decoder-side Intra Mode Derivation), TIMD (Template-based Intra Mode Derivation), OBIC (Occurrence-Based Intra Coding), OBMC (Overlapped Block Motion Compensation), SGPM (Spatial Geometric Partitioning Mode), CIIP (Combined Inter-Intra Prediction), Intra Merge Mode, MRL (Multiple Reference Line), and TMRL (Template-based It may include at least one of MRL.

[0024] A method for transmitting a bitstream according to an embodiment of the present invention, wherein the transmission method comprises the step of transmitting a bitstream generated by an image encoding method, and the image encoding method comprises the steps of dividing a current block into a plurality of sub-partitions, deriving a prediction mode for each of the plurality of sub-partitions, and performing a prediction for each of the sub-partitions based on the derivation prediction mode, wherein the prediction mode is a directional prediction mode, MIP (Matrix-based Intra Prediction), NNIP (Neural-Network based Intra Prediction), EIP (Extrapolation based Intra Prediction), IntraTMP (Intra Template Matching Prediction), and IBC (Intra Block Copy), DIMD (Decoder-side Intra Mode Derivation), TIMD (Template-based Intra Mode Derivation), OBIC (Occurrence-Based Intra Coding), OBMC (Overlapped Block Motion Compensation), SGPM (Spatial Geometric Partitioning Mode), CIIP (Combined Inter-Intra Prediction), and an intra-frame merge mode. It may include at least one of MRL (Multiple Reference Line) and TMRL (Template-based MRL).

[0025] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0026] According to the present invention, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency may be provided.

[0027] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0028] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied.

[0029] FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present invention is applied.

[0030] FIG. 3 is a schematic diagram showing a video coding system to which the present invention can be applied.

[0031] FIG. 4 is a diagram illustrating a sub-partition structure within a block to which the present invention can be applied.

[0032] FIG. 5 is a diagram illustrating a sub-partition-based prediction mode induction method according to an embodiment of the present invention.

[0033] FIG. 6 is a diagram illustrating a sub-partition-based prediction mode induction method according to an embodiment of the present invention.

[0034] FIG. 7 is a diagram illustrating a sub-partition-based prediction mode induction method according to an embodiment of the present invention.

[0035] FIG. 8 is a diagram illustrating a sub-partition-based blending method according to an embodiment of the present invention.

[0036] FIG. 9 is a diagram illustrating a sub-partition-based blending method according to an embodiment of the present invention.

[0037] FIG. 10 is a diagram illustrating a sub-partition-based blending method according to an embodiment of the present invention.

[0038] FIG. 11 is a diagram illustrating a sub-partition-based blending method according to an embodiment of the present invention.

[0039] FIG. 12 is a diagram illustrating a sub-partition-based filtering method according to an embodiment of the present invention.

[0040] FIG. 13 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0041]

[0042] FIG. 14 is a diagram illustrating an exemplary content streaming system to which an embodiment according to the present invention can be applied.

[0043] The present invention is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals in the drawings refer to the same or similar functions across various aspects. The shapes and sizes of elements in the drawings may be provided illustratively for clearer explanation. The detailed description of the exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of exemplary embodiments is limited only by the appended claims, together with all equivalents to those claimed therein, provided they are properly described.

[0044] In the present invention, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0045] The components shown in the embodiments of the present invention are illustrated independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for the convenience of explanation; however, at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are included within the scope of the present invention as long as they do not deviate from the essence of the invention.

[0046] The terms used in this invention are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, some components of this invention may not be essential components performing an essential function in this invention, but may be optional components merely for enhancing performance. This invention may be implemented by including only the components essential to embody the essence of this invention, excluding components used solely for performance enhancement, and a structure including only the essential components, excluding optional components used solely for performance enhancement, is also included within the scope of this invention.

[0047] In the embodiments, the term "at least one" may mean one of a number of 1 or more, such as 1, 2, 3, and 4. In the embodiments, the term "a plurality of" may mean one of a number of 2 or more, such as 2, 3, and 4.

[0048] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of this specification, if it is determined that a detailed description of related known configurations or functions may obscure the gist of this specification, such detailed description is omitted; similar reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0049] Glossary of Terms

[0050] In the following, “image” may refer to a single picture constituting a video, or it may refer to the video itself. For example, “encoding and / or decoding of an image” may mean “encoding and / or decoding of an image”, and may also mean “encoding and / or decoding of one of the images constituting the video”.

[0051] In the following, "video" and "video" may be used interchangeably with the same meaning. Additionally, the target image may be an image to be encoded and / or an image to be decoded. Furthermore, the target image may be an input image fed into an encoding device and an input image fed into a decoding device. Here, the target image may have the same meaning as the current image.

[0052] In the following, the encoder and the image encoding device may be used interchangeably.

[0053] In the following, the decoder and the image decoder may be used interchangeably with each other.

[0054] In the following, "image," "picture," "frame," and "screen" may be used interchangeably with the same meaning.

[0055] In the following, “target block” may be an encoding target block that is the target of encoding and / or a decoding target block that is the target of decoding. Additionally, the target block may be a current block that is the target of current encoding and / or decoding. For example, “target block” and “current block” may be used interchangeably.

[0056] In the following description, "block" and "unit" may be used interchangeably. Additionally, to distinguish it from a block, "unit" may refer to a block containing a luminance (Luma) component block and a corresponding chroma (Chroma) component block. For example, a Coding Tree Unit (CTU) may consist of a single luminance component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated with it.

[0057] In the following, “sample,” “pixel,” and “pixel” may be used interchangeably with the same meaning. Here, a sample may represent a basic unit constituting a block.

[0058] In the following, “inter” and “inter-screen” may be used interchangeably with the same meaning.

[0059] In the following, “intra” and “in-screen” may be used interchangeably with the same meaning.

[0060]

[0061] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied.

[0062] The encoding device (100) may be an encoder, a video encoding device, or an image encoding device. The video may include one or more images. The encoding device (100) may sequentially encode one or more images.

[0063] Referring to FIG. 1, the encoding device (100) may include an image segmentation unit (110), an intra prediction unit (120), a motion prediction unit (121), a motion compensation unit (122), a switch (115), a subtractor (113), a converter (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse converter (170), an adder (117), a filter unit (180), and a reference picture buffer (190).

[0064] Additionally, the encoding device (100) can generate a bitstream containing encoded information through encoding of an input image and can output the generated bitstream. The generated bitstream can be stored on a computer-readable recording medium or streamed via a wired / wireless transmission medium.

[0065] The video segmentation unit (110) can divide the input video into various forms to increase the efficiency of video encoding / decoding. That is, the input video consists of multiple pictures, and a single picture can be processed by hierarchically dividing it for compression efficiency, parallel processing, etc. For example, a single picture can be divided into one or more tiles or slices and then divided again into multiple CTUs (Coding Tree Units). Alternatively, a single picture can first be divided into multiple sub-pictures defined as groups of rectangular slices, and each sub-picture can be divided into the said tiles / slices. Here, the sub-pictures can be utilized to support the function of partially and independently encoding / decoding and transmitting the picture. Since multiple sub-pictures can each be restored individually, they have the advantage of being easy to edit in applications that configure multi-channel inputs into a single picture. In addition, the tiles can be divided horizontally to create bricks. Here, a brick can be utilized as the basic unit of parallel processing within a picture. Additionally, a single CTU can be recursively partitioned into a Quadtree (QT), and the terminal node of the partition can be defined as a Coding Unit (CU). The CU can be divided into a Prediction Unit (PU) and a Transform Unit (TU) to perform prediction and transformation. Meanwhile, the CU can be utilized as the prediction unit and / or the transformation unit itself. Here, for flexible partitioning, each CTU can be recursively partitioned into a Multi-Type Tree (MTT) as well as a Quadtree (QT). The partitioning of the CTU into a Multi-Type Tree can begin at the terminal node of the QT, and the MTT can be composed of a Binary Tree (BT) and a Triple Tree (TT).For example, the MTT structure can be classified into vertical binary splitting mode (SPLIT_BT_VER), horizontal binary splitting mode (SPLIT_BT_HOR), vertical ternary splitting mode (SPLIT_TT_VER), and horizontal ternary splitting mode (SPLIT_TT_HOR). Additionally, when splitting, the minimum block size (MinQTSize) of the quad tree for the luminance block can be set to 16x16, the maximum block size (MaxBtSize) of the binary tree to 128x128, and the maximum block size (MaxTtSize) of the triple tree to 64x64. Furthermore, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the triple tree can be set to 4x4, and the maximum depth (MaxMttDepth) of the multi-type tree can be set to 4. Additionally, to increase the encoding efficiency of the I slice, a dual tree can be applied that uses different CTU splitting structures for the luminance and chrominance components. On the other hand, in P and B slices, the luminance and color difference CTBs (Coding Tree Blocks) within the CTU can be divided into a single tree that shares a coding tree structure.

[0066] The encoding device (100) may perform encoding on an input image in an intra mode and / or inter mode. Alternatively, the encoding device (100) may perform encoding on an input image in a third mode other than the intra mode and inter mode (e.g., IBC mode, Palette mode, etc.). However, if the third mode has functional characteristics similar to the intra mode or inter mode, it may be classified as an intra mode or inter mode for convenience of explanation. In the present invention, the third mode will be classified and described separately only when a specific explanation of the third mode is required.

[0067] When intra mode is used as the prediction mode, the switch (115) can be switched to intra, and when inter mode is used as the prediction mode, the switch (115) can be switched to inter. Here, intra mode may mean an intra-frame prediction mode, and inter mode may mean an inter-frame prediction mode. The encoding device (100) can generate a prediction block for an input block of an input image. Additionally, after the prediction block is generated, the encoding device (100) can encode a residual block using the residual of the input block and the prediction block. The input image may be referred to as the current image that is the subject of current encoding. The input block may be referred to as the current block that is the subject of current encoding or the encoding target block.

[0068] When the prediction mode is an intra mode, the intra prediction unit (120) may use a sample of a block that has already been encoded / decoded around the current block as a reference sample. The intra prediction unit (120) may perform spatial prediction for the current block using the reference sample and generate prediction samples for the input block through spatial prediction. Here, intra prediction may mean intra-frame prediction.

[0069] In the intra prediction method, non-directional prediction modes such as DC mode and Planar mode, and directional prediction modes (e.g., 65 directions) may be applied. Here, the intra prediction method can be expressed as an intra prediction mode or an intra-frame prediction mode.

[0070] When the prediction mode is an inter mode, the motion prediction unit (121) can search for the region that best matches the input block from the reference image during the motion prediction process and derive a motion vector using the searched region. At this time, the search region can be used as the region. The reference image can be stored in the reference picture buffer (190). Here, the reference image can be stored in the reference picture buffer (190) when encoding / decoding of the reference image is processed.

[0071] The motion compensation unit (122) can generate a prediction block for the current block by performing motion compensation using a motion vector. Here, inter-prediction may mean inter-frame prediction or motion compensation.

[0072] The motion prediction unit (121) and motion compensation unit (122) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform inter-frame prediction or motion compensation, based on the encoding unit, it can determine whether the motion prediction and motion compensation method of the prediction unit included in the corresponding encoding unit is a Skip Mode, Merge Mode, Advanced Motion Vector Prediction (AMVP) Mode, or Intra Block Copy (IBC) Mode, and can perform inter-frame prediction or motion compensation according to each mode.

[0073] In addition, based on the above-mentioned inter-frame prediction method, the AFFINE mode of sub-PU-based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and GPM (Geometric Partitioning Mode) mode of PU-based prediction may be applied. Furthermore, to improve the performance of each mode, HMVP (History based MVP), PAMVP (Pairwise Average MVP), CIIP (Combined Intra / Inter Prediction), AMVR (Adaptive Motion Vector Resolution), BDOF (Bi-Directional Optical-Flow), BCW (Bi-predictive with CU Weights), LIC (Local Illumination Compensation), TM (Template Matching), OBIC (Occurrence-Based Intra Coding), OBMC (Overlapped Block Motion Compensation), MRL (Multiple Reference Line), and TMRL (Template-based MRL) may be applied.

[0074] Among these, AFFINE mode is a technology used in both AMVP and MERGE modes and also offers high encoding efficiency. Conventional video coding standards have the disadvantage of failing to properly compensate for real-world movements, such as zoom in / out and rotation, because they perform Motion Compensation (MC) by considering only the translation of blocks. To address this, a 4-parameter affine motion model using two control point motion vectors (CPMV) and a 6-parameter affine motion model using three control point motion vectors can be applied to inter-prediction. Here, CPMV is a vector representing one of the affine motion models of the top-left, top-right, or bottom-left corners of the current block.

[0075] The subtractor (113) can generate a residual block using the difference between the input block and the prediction block. The residual block may also be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or both transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal in block units.

[0076] The transformation unit (130) can generate a transform coefficient by performing a transform on the remaining block and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the remaining block. When a transform skip mode is applied, the transformation unit (130) may skip the transform on the remaining block.

[0077] A quantized level can be generated by applying quantization to a conversion coefficient or a residual signal. In the following embodiments, the quantized level may also be referred to as a conversion coefficient.

[0078] For example, a 4x4 luminance residual block generated through intra-frame prediction can be transformed using a Discrete Sine Transform (DST)-based basis vector, while the remaining residual blocks can be transformed using a Discrete Cosine Transform (DCT)-based basis vector. Additionally, the transformation blocks for a single block can be divided into a quad tree form using Residual Quad Tree (RQT) technology, and after performing transformation and quantization on each transformation block divided by RQT, a coded block flag (cbf) can be transmitted to increase coding efficiency in the case where all coefficients become zero.

[0079] As another alternative, the Multiple Transform Selection (MTS) technique can be applied to perform transformations using multiple transformation bases selectively. In other words, instead of dividing a CU into TUs via RQT, a function similar to TU division can be performed using the Sub-block Transform (SBT) technique. Specifically, SBT is applied only to inter-frame prediction blocks and, unlike RQT, divides the current block into ½ or ¼ sizes in the vertical or horizontal direction, and then performs a transformation on only one of the blocks. For example, if divided vertically, a transformation can be performed on the leftmost or rightmost block, and if divided horizontally, a transformation can be performed on the topmost or bottommost block.

[0080] In addition, Low Frequency Non-Separable Transform (LFNST), a secondary transform technique that further transforms the residual signal converted to the frequency domain through DCT or DST, can also be applied. LFNST performs additional transformation on the 4x4 or 8x8 low-frequency region in the upper left corner, thereby allowing the residual coefficients to be concentrated in the upper left corner.

[0081] The quantization unit (140) can generate a quantized level by quantizing a transformation coefficient or residual signal according to a quantization parameter (QP, Quantization parameter) and can output the generated quantized level. At this time, the quantization unit (140) can quantize the transformation coefficient using a quantization matrix.

[0082] For example, a quantizer using QP values ​​from 0 to 51 can be used. Alternatively, if the image size is larger and higher coding efficiency is required, QP values ​​from 0 to 63 can be used. Additionally, a Dependent Quantization (DQ) method using two quantizers instead of a single one can be applied. DQ performs quantization using two quantizers (e.g., Q0, Q1), but can be applied so that the quantizer to be used for the next transform coefficient is selected based on the current state through a state transition model, even without signaling information regarding the use of a specific quantizer.

[0083] The entropy encoding unit (150) can generate a bitstream and output a bitstream by performing entropy encoding according to a probability distribution on values ​​calculated by the quantization unit (140) or coding parameter values ​​calculated during the encoding process. The entropy encoding unit (150) can perform entropy encoding on information regarding a sample of an image and information for decoding an image. For example, information for decoding an image may include syntax elements, etc.

[0084] When entropy coding is applied, a small number of bits are allocated to symbols with a high probability of occurrence and a large number of bits are allocated to symbols with a low probability of occurrence, thereby representing the symbols and reducing the size of the bit sequence for the symbols to be encoded. The entropy coding unit (150) may use encoding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) for entropy coding. For example, the entropy coding unit (150) may perform entropy coding using a Variable Length Coding (VLC) table. Additionally, the entropy encoding unit (150) may perform arithmetic encoding using the derived binarization method, probability model, and context model after deriving a binarization method of the target symbol and a probability model of the target symbol / bin.

[0085] In this regard, when applying CABAC, in order to reduce the size of the probability table stored in the decoder, the table probability update method may be changed to a table update method using a simple formula. In addition, two different probability models may be used to obtain more accurate symbol probability values.

[0086] The entropy encoding unit (150) can convert a 2-dimensional block form coefficient into a 1-dimensional vector form through a transform coefficient scanning method to encode a transform coefficient level (quantized level).

[0087] Coding parameters may include information (flags, indexes, etc.) that is encoded in the encoding device (100) and signaled to the decoding device (200), such as syntax elements, as well as information derived during the encoding process or decoding process, and may refer to information required when encoding or decoding images.

[0088] Here, signaling a flag or index may mean that in an encoder, the corresponding flag or index is entropy encoded and included in a bitstream, and in a decoder, the corresponding flag or index is entropy decoded from the bitstream.

[0089] The encoded current image can be used as a reference image for other images processed later. Accordingly, the encoding device (100) can restore or decode the encoded current image again, and can store the restored or decoded image as a reference image in the reference picture buffer (190).

[0090] The quantized level can be dequantized in the dequantization unit (160) and inverse transformed in the inverse transform unit (170). The dequantized and / or inverse transformed coefficients can be added to the prediction block through the adder (117). A reconstructed block can be generated by adding the dequantized and / or inverse transformed coefficients and the prediction block. Here, the dequantized and / or inverse transformed coefficients refer to coefficients for which at least one of dequantization and inverse transformation has been performed, and may refer to the reconstructed residual block. The dequantization unit (160) and the inverse transform unit (170) can be performed as the reverse process of the quantization unit (140) and the transformation unit (130).

[0091] The restoration block may pass through a filter section (180). The filter section (180) may apply a deblocking filter, Sample Adaptive Offset (SAO), Adaptive Loop Filter (ALF), Bilateral filter (BIF), LMCS (Luma Mapping with Chroma Scaling), etc., to the restoration sample, restoration block, or restoration image as a whole or part of the filtering technique. The filter section (180) may also be referred to as an in-loop filter. In this case, the term in-loop filter is also used as a name that excludes LMCS.

[0092] Deblocking filters can remove block distortion occurring at the boundaries between blocks. To determine whether to perform deblocking, the decision to apply the filter to the current block can be made based on samples contained in a few columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering intensity.

[0093] To compensate for encoding errors using a sample adaptive offset, an appropriate offset value can be added to the sample value. The sample adaptive offset can correct the offset from the original image on a sample-by-sample basis for the deblocked image. One method may be to divide the samples included in the image into a certain number of regions, determine the region to be offset, and apply the offset to that region, or to apply the offset by considering the edge information of each sample.

[0094] A bilateral filter (BIF) can also correct the offset from the original image on a sample-by-sample basis for the deblocked image.

[0095] An adaptive loop filter can perform filtering based on a comparison of the reconstructed image and the original image. After dividing the samples included in the image into predetermined groups, a filter to be applied to each group can be determined, thereby performing filtering differently for each group. Information regarding whether to apply an adaptive loop filter can be signaled per coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may vary depending on each block.

[0096] In LMCS (Luma Mapping with Chroma Scaling), Luma mapping (LM) refers to remapping luminance values ​​through a piece-wise linear model, and Chroma scaling (CS) refers to a technique that scales the residual values ​​of the chrominance component according to the average luminance value of the predicted signal. In particular, LMCS can be utilized as an HDR correction technique that reflects the characteristics of HDR (High Dynamic Range) video.

[0097] The restored block or restored image that has passed through the filter unit (180) can be stored in the reference picture buffer (190). The restored block that has passed through the filter unit (180) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (180). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0098] FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present invention is applied.

[0099] The decoding device (200) may be a decoder, a video decoding device, or an image decoding device.

[0100] Referring to FIG. 2, the decoding device (200) may include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), a motion compensation unit (250), an adder (201), a switch (203), a filter unit (260), and a reference picture buffer (270).

[0101] The decoding device (200) can receive a bitstream output from the encoding device (100). The decoding device (200) can receive a bitstream stored in a computer-readable recording medium or a bitstream stream streamed through a wired / wireless transmission medium. The decoding device (200) can perform decoding on the bitstream in intra mode or inter mode. Additionally, the decoding device (200) can generate a restored image or a decoded image through decoding and can output the restored image or the decoded image.

[0102] If the prediction mode used for decoding is intra mode, the switch (203) can be switched to intra. If the prediction mode used for decoding is inter mode, the switch (203) can be switched to inter.

[0103] The decoding device (200) can decode the input bitstream to obtain a reconstructed residual block and generate a prediction block. Once the reconstructed residual block and the prediction block are obtained, the decoding device (200) can generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may be referred to as the current block.

[0104] The entropy decoding unit (210) can generate symbols by performing entropy decoding according to the probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the inverse process of the entropy encoding method described above.

[0105] The entropy decoding unit (210) can convert a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a conversion coefficient scanning method to decode a conversion coefficient level (quantized level).

[0106] The quantized level can be dequantized in the dequantization unit (220) and inversely transformed in the inverse transformation unit (230). The quantized level can be generated as a restored residual block as a result of performing dequantization and / or inverse transformation. At this time, the dequantization unit (220) can apply a quantization matrix to the quantized level. The dequantization unit (220) and the inverse transformation unit (230) applied to the decoding device can apply the same technology as the dequantization unit (160) and the inverse transformation unit (170) applied to the aforementioned encoding device.

[0107] When an intra mode is used, the intra prediction unit (240) can generate a prediction block by performing a spatial prediction on the current block using sample values ​​of already decoded blocks around the block to be decoded. The intra prediction unit (240) applied to the decoding device can apply the same technology as the intra prediction unit (120) applied to the aforementioned encoding device.

[0108] When an inter mode is used, the motion compensation unit (250) can generate a prediction block by performing motion compensation on the current block using a motion vector and a reference image stored in the reference picture buffer (270). The motion compensation unit (250) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform motion compensation, it can be determined whether the motion compensation method of the prediction unit included in the corresponding encoding unit is a skip mode, merge mode, AMVP mode, or current picture reference mode based on the encoding unit, and motion compensation can be performed according to each mode. The motion compensation unit (250) applied to the decoder can apply the same technology as the motion compensation unit (122) applied to the aforementioned encoding unit.

[0109] The adder (201) can generate a restored block by adding the restored residual block and the prediction block. The filter unit (260) can apply at least one of the following to the restored block or the restored image: an inverse-LMCS, a deblocking filter, a sample adaptive offset, and an adaptive loop filter. The filter unit (260) applied to the decoder can apply the same filtering technology as the filter unit (180) applied to the aforementioned encoding device.

[0110] The filter unit (260) can output a restored image. The restored block or the restored image can be stored in a reference picture buffer (270) and used for inter-frame prediction. The restored block that has passed through the filter unit (260) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (260). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0111] FIG. 3 is a schematic diagram showing a video coding system to which the present invention can be applied.

[0112] A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and / or image information or data to the decoding device (20) via a digital storage medium or network in the form of a file or streaming.

[0113] An encoding device (10) according to one embodiment may include a video source generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to one embodiment may include a receiving unit (21), a decoding unit (22), and a rendering unit (23). The encoding unit (12) may be called a video / image encoding unit, and the decoding unit (22) may be called a video / image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The receiving unit (21) may be included in the decoding unit (22). The rendering unit (23) may include a display unit, and the display unit may be composed of a separate device or an external component.

[0114] The video source generation unit (11) can acquire video / image through a process of capturing, synthesizing, or generating video / image. The video source generation unit (11) may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / image, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and can generate video / image (electronically). For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0115] The encoding unit (12) can encode the input video / image. The encoding unit (12) can perform a series of procedures such as prediction, conversion, and quantization for compression and encoding efficiency. The encoding unit (12) can output the encoded data (encoded video / image information) in the form of a bitstream. The detailed configuration of the encoding unit (12) can also be configured in the same way as the encoding device (100) of FIG. 1 described above.

[0116] The transmission unit (13) can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit (21) of the decoding device (20) via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit (21) can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

[0117] The decoding unit (22) can decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12). The detailed configuration of the decoding unit (22) can also be configured to be identical to the decoding device (200) of FIG. 2 described above.

[0118] The rendering unit (23) can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0119]

[0120] FIG. 4 is a diagram illustrating a sub-partition structure within a block to which the present invention can be applied.

[0121] Referring to FIG. 4, the current picture may include a plurality of coding units (CUs). The current block may be a coding unit (CU).

[0122] The current block may be divided into multiple sub-partitions. For example, the current block may be divided into four parts in the vertical direction, such as the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4). However, embodiments of the present invention are not limited thereto, and the division structure of the current block may be composed of two or more sub-partitions divided in the horizontal or vertical direction.

[0123] The prediction modes for multiple sub-partitions included within the current block may differ from one another. For example, the prediction mode derived for the first sub-partition (sub-partition1) and the prediction mode derived for the second sub-partition (sub-partition2) may differ from one another. That is, the current block may include multiple sub-partitions to which different prediction modes are applied.

[0124] In one embodiment, the prediction mode may be an in-frame prediction mode. For example, the prediction mode may include a non-directional prediction mode (planar mode, dc mode) and a directional prediction mode (angular mode), etc.

[0125] In another embodiment, the prediction mode may be an inter-frame prediction mode. For example, the prediction mode may include a Merge mode, a Skip mode, an AMVP (Advanced Motion Vector Prediction) mode, and an Affine mode.

[0126]

[0127] Hereinafter, with reference to FIGS. 5 to 7, a method for inducing a prediction mode based on multiple sub-partitions within the current block will be described.

[0128]

[0129] FIG. 5 is a diagram illustrating a sub-partition-based prediction mode induction method according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a sub-partition-based prediction mode induction method according to an embodiment of the present invention.

[0130] Referring to FIGS. 5 and 6, in step S510, the current block may be divided into n sub-partitions in order to induce a prediction mode for each of the plurality of sub-partitions within the current block. n may be an integer greater than or equal to 1. For example, n may be 4. That is, the current block may be divided into a first sub-partition (sub-partition1) to a fourth sub-partition (sub-partition4).

[0131] Next, in step S520, k can be set to 1.

[0132] Next, in step S530, a prediction mode for the k-th sub-partition is derived, and a prediction for the k-th sub-partition can be performed. For example, a prediction mode for the first sub-partition (sub-partition1) is derived, and a prediction for the first sub-partition (sub-partition1) can be performed.

[0133] Next, in step S540, the values ​​of k and n can be compared.

[0134] If k is less than n, in step S540, the value of k can be increased by 1. For example, the value of k can be 2.

[0135] Next, in step S530, a prediction mode for the k-th sub-partition is derived, and a prediction for the k-th sub-partition can be performed. For example, a prediction mode for the second sub-partition (sub-partition2) is derived, and a prediction for the second sub-partition (sub-partition2) can be performed.

[0136] In one embodiment, a prediction for the second sub-partition (sub-partition2) can be performed based on a recovery sample of the first sub-partition (sub-partition1).

[0137] In another embodiment, a prediction for the second sub-partition (sub-partition2) can be performed based on a reference block according to motion information.

[0138] If k is equal to n, the sub-partition-based prediction process is terminated at step S550. For example, the sub-partition-based prediction process may be terminated after predictions for the third sub-partition (sub-partition3) and the fourth sub-partition (sub-partition4) are performed sequentially.

[0139] Referring to FIG. 6, the prediction modes for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) can each be adaptively induced.

[0140] The prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) within the current block can be derived based on reference samples (Reference sample1 to Reference sample4). The reference samples may be restored signals.

[0141] For example, a first reference sample (Reference sample1) may be a plurality of samples adjacent to a first sub-partition and extending in the horizontal and vertical directions. A second reference sample (Reference sample2) may be a plurality of samples adjacent to a second sub-partition and extending in the horizontal and vertical directions. The second reference sample may include some of the samples within the first sub-partition. A third reference sample (Reference sample3) may be a plurality of samples adjacent to a third sub-partition and extending in the horizontal and vertical directions. The third reference sample may include some of the samples within the second sub-partition. A fourth reference sample (Reference sample4) may be a plurality of samples adjacent to a fourth sub-partition and extending in the horizontal and vertical directions. The fourth reference sample may include some of the samples within the third sub-partition.

[0142] In one embodiment, the prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) can be adaptively determined through DIMD (Decoder-side Intra Mode Derivation) information.

[0143] For example, the prediction mode for the first sub-partition (sub-partition1) can be adaptively determined based on the Histogram of Gradient (HoG) calculated based on the gradient values ​​of the first reference sample (Reference sample1). The prediction mode for the second sub-partition (sub-partition2) can be adaptively determined based on the Histogram of Gradient (HoG) calculated based on the gradient values ​​of the second reference sample (Reference sample2). The prediction mode for the third sub-partition (sub-partition3) can be adaptively determined based on the Histogram of Gradient (HoG) calculated based on the gradient values ​​of the third reference sample (Reference sample3). The prediction mode for the fourth sub-partition (sub-partition4) can be adaptively determined based on the Histogram of Gradient (HoG) calculated based on the gradient values ​​of the fourth reference sample (Reference sample4).

[0144] In another embodiment, the prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) may be adaptively determined through a DIMD (Decoder-side Intra Mode Derivation) mode among one or more predetermined prediction mode candidates.

[0145] For example, the prediction mode candidates set for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) may include a horizontal mode and a vertical mode. The prediction mode for the first sub-partition (sub-partition1) may be determined as at least one of a horizontal mode and / or a vertical mode based on a histogram of gradient (HoG) calculated based on the gradient values ​​of the first reference sample (Reference sample1). Similarly, the prediction mode for the second sub-partition (sub-partition2) to the fourth sub-partition (sub-partition4) may be determined as at least one of a horizontal mode and / or a vertical mode based on a histogram of gradient (HoG) calculated based on the gradient values ​​of the second reference sample (Reference sample2) to the fourth reference sample (Reference sample4), respectively.

[0146] In another embodiment, the prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) can be derived through TIMD (Template matching based Intra Mode Derivation) information.

[0147] In another embodiment, the prediction modes induced in each of the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) may be two or more. In this case, the prediction samples for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) may be generated by a weighted combination of prediction samples generated by a plurality of prediction modes.

[0148] The prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) within the current block can be derived based on a portion of the reference samples (Reference sample1 to Reference sample4).

[0149] For example, the prediction mode for the first sub-partition (sub-partition1) may be derived based only on the upper or left reference sample of the first reference sample (Reference sample1) adjacent to the first sub-partition (sub-partition1). The prediction mode for the second sub-partition (sub-partition2) to the fourth sub-partition (sub-partition4) may also be derived based only on the upper or left reference sample of the second reference sample (Reference sample2) to the fourth reference sample (Reference sample4).

[0150] The prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) within the current block can be derived based on adjacent blocks. Adjacent blocks may be blocks adjacent to the current block. However, embodiments of the present invention are not limited thereto, and adjacent blocks may include blocks that are not adjacent to the current block.

[0151] In one embodiment, the prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) can be adaptively determined through Occurrence-Based Intra Coding (OBIC) information.

[0152] For example, the prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) can be adaptively determined based on the Histogram of Occurrence (HoC), which is the frequency of occurrence of the prediction mode used in the surrounding blocks of the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4), respectively.

[0153] The prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) within the current block can be derived based on coding information of adjacent blocks or adjacent sub-partitions.

[0154] For example, the prediction mode of the first sub-partition (sub-partition1) is determined based on the prediction mode for the surrounding blocks of the first sub-partition (sub-partition1), and the prediction mode of the second sub-partition (sub-partition2) can be determined based on the prediction mode for the surrounding blocks of the second sub-partition (sub-partition2) and the prediction mode for the first sub-partition (sub-partition1).

[0155] The prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) within the current block can be derived based on the coding information of the adjacent reference block.

[0156] For example, if a neighboring reference block of a first sub-partition (sub-partition1) has block vector (BV) information, a prediction mode for a corresponding region can be derived based on the DIMD (Decoder-side Intra Mode Derivation) information of the corresponding region indicated by the block vector. The prediction mode for the first sub-partition (sub-partition1) can be determined based on the prediction mode for the corresponding region.

[0157] Meanwhile, prediction mode candidates applicable to the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) within the current block can be adaptively set based on coding information of adjacent reference samples, adjacent blocks, and adjacent sub-partitions. Prediction modes applicable to the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) may include Matrix-based Intra Prediction (MIP), Neural-Network based Intra Prediction (NNIP), DIMD, TIMD, Extrapolation based Intra Prediction (EIP), Intra Template Matching Prediction (IntraTMP), Intra Block Copy (IBC), CIIP, etc.

[0158] And, the prediction mode for the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) within the current block can be induced either explicitly or implicitly.

[0159] For example, a prediction mode for a first sub-partition (sub-partition1) may be transmitted from an encoding device to a decoder. The decoder may apply the signaled prediction mode to the first sub-partition (sub-partition1). A prediction mode for a second sub-partition (sub-partition2) may be derived based on the adjacent reference sample, adjacent blocks, and coding information of adjacent sub-partitions of the second sub-partition (sub-partition2). Prediction modes for a third sub-partition (sub-partition3) and a fourth sub-partition (sub-partition4) may also be derived by either an explicit or implicit method. Coding information refers collectively to a series of information generated, transmitted, or derived during the encoding / decoding process of the current block or surrounding blocks, and may include prediction mode index, segmentation information, and motion information, as well as statistical feature values ​​extracted from reference samples (e.g., HoG, HoC), a list of candidates for available prediction modes, and availability information of reference pixels according to a specific encoding environment (e.g., parallel processing mode).

[0160] In this way, the prediction accuracy of the current block can be improved by individually deriving and applying a prediction mode to each sub-partition within the current block. Through this, the residual signal between the predicted signal and the original signal can be minimized, and consequently, the compression efficiency of the image data can be improved.

[0161] FIG. 7 is a diagram illustrating a sub-partition-based prediction mode induction method according to an embodiment of the present invention.

[0162] Referring to FIG. 7, the current block can be divided into a first sub-partition (sub-partition1) to a fourth sub-partition (sub-partition4). The prediction mode for each of the first sub-partition (sub-partition1) to the fourth sub-partition (sub-partition4) can be adaptively induced.

[0163] The prediction mode for the first sub-partition (sub-partition1) through the fourth sub-partition (sub-partition4) within the current block can be derived based on the reference sample of the current block. For example, the prediction mode for the first sub-partition (sub-partition1) can be derived based on the reference sample of the current block. The reference sample may be a sample adjacent to the upper and left sides of the first sub-partition (sub-partition1). The prediction mode for the second sub-partition (sub-partition2) through the fourth sub-partition (sub-partition4) can be derived based on the reference sample of the current block. The reference sample may be a sample adjacent to the upper side and not adjacent to the left side of the second sub-partition (sub-partition2) through the fourth sub-partition (sub-partition4).

[0164] The sub-partition-based prediction method described in FIGS. 6 and 7 can be adaptively determined based on the encoding / decoding environment or hardware conditions of the image. Specifically, the sub-partition-based prediction method can be adaptively determined based on the shape and size of the current block, the size of the Coding Tree Unit (CTU) and Virtual Pipeline Data Unit (VPDU), whether a separate block structure (Separate Tree) is applied, the quantization parameter (QP), and the encoding information of the luminance block at the same location.

[0165] Whether a sub-partition-based prediction method is applied can be explicitly signaled at the sequence / picture / slice / tile level through upper-level syntax elements within the bitstream. For example, the application of a sub-partition-based prediction method can be determined based on at least one of a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and a slice header (SH).

[0166] Alternatively, whether to apply a sub-partition-based prediction method can be determined based on a specific profile.

[0167] Alternatively, whether a sub-partition-based prediction method is applied can be explicitly signaled at the CTU level through block-level syntax elements within the bitstream.

[0168] In one embodiment, when it is determined that a sub-partition-based prediction method is applied, any one of the aforementioned sub-partition-based prediction methods may be applied in a fixed manner.

[0169] In another embodiment, when it is determined that a sub-partition-based prediction method is applied, any one of the aforementioned sub-partition-based prediction methods may be adaptively determined and applied based on additional information transmitted through a bitstream. For example, the additional information may be information signaled at a higher level (e.g., at the sequence / picture / slice / tile level), CTU, VPDU, or CU level.

[0170] Furthermore, the aforementioned sub-partition-based prediction method can be adaptively applied to various data units, such as image types or encoding structures. Specifically, the sub-partition-based prediction method can be applied not only to I-slices (or pictures, frames) but also to blocks included within P / B slices (or pictures, frames).

[0171] In this way, the prediction accuracy of the current block can be improved by individually deriving and applying a prediction mode to each sub-partition within the current block. Through this, the residual signal between the predicted signal and the original signal can be minimized, and consequently, the compression efficiency of the image data can be improved.

[0172]

[0173] FIG. 8 is a diagram illustrating a sub-partition-based blending method according to an embodiment of the present invention. FIG. 9 is a diagram illustrating a sub-partition-based blending method according to an embodiment of the present invention. FIG. 10 is a diagram illustrating a sub-partition-based blending method according to an embodiment of the present invention.

[0174] Referring to FIG. 8, the current block can be divided into a first sub-partition (sub-partition1) and a second sub-partition (sub-partition2). Blending can be performed on the overlapping region between the first extended region (Extended Region1) in which the first sub-partition (sub-partition1) is extended and the second extended region (Extended Region1) in which the second sub-partition (sub-partition2) is extended.

[0175] Specifically, the current block may be divided into a first sub-partition (sub-partition1) and a second sub-partition (sub-partition2) based on a sub-partition boundary. The first extended region (Extended Region1) may be an area extended in the width or height direction of the first sub-partition (sub-partition1). Similarly, the second extended region (Extended Region2) may be an area extended in the width or height direction of the second sub-partition (sub-partition2). Combination or blending may be performed on the overlapping region that overlaps between the first extended region (Extended Region1) and the second extended region (Extended Region2). The width or height of the overlapping region may be X times the width or height of the sub-partition. X may be a real number greater than or equal to 0. In one embodiment, X may be a pre-set value. In another embodiment, X may be a value that is adaptively set based on the width or height of the sub-partition.

[0176] Blending between the first sub-partition (sub-partition1) and the second sub-partition (sub-partition2) can be performed by applying fixed or adaptive weights based on the sub-partition boundaries to the overlapped region where the first extension region and the second extension region overlap. Specific embodiments regarding the blending between the first sub-partition (sub-partition1) and the second sub-partition (sub-partition2) are described below.

[0177] The blending between the first sub-partition (sub-partition1) and the second sub-partition (sub-partition2) may be the same or similar to a blending method applied to any one of, for example, OBIC (Occurrence-Based Intra Coding), OBMC (overlapped block motion compensation), GPM (Geometric Partitioning Mode), SGPM (Spatial Geometric Partitioning Mode), TIMD (Template-based Intra Mode Derivation), DIMD (Decoder-side Intra Mode Derivation), CIIP (Combined Inter-Intra Prediction), MRL (Multiple Reference Line), and TMRL (Template-based MRL).

[0178] For example, OBIC (Occurrence-Based Intra Coding) can improve mode coding efficiency by statistically managing the occurrence of intra-frame prediction modes used in surrounding blocks and reconstructing the list of intra-frame prediction mode candidates for the current block based on this.

[0179] For example, the sub-block boundary OBMC can apply blending to the top, left, bottom, and right boundary pixels using motion information of surrounding sub-blocks (or block vector information in the case of IBC / IntraTMP).

[0180] For example, GPM (Geometric Partitioning Mode) can generate a final prediction sample by weighting and averaging two prediction signals.

[0181] For example, SGPM (Spatial Geometric Partitioning Mode) can construct candidates with 26 partitioning angles and 3 combinations of in-frame prediction modes for each partitioned block, and weighted blending can be performed on the boundary regions of the partitioned partitions.

[0182] For example, TIMD (Template-based Intra Mode Derivation) can calculate weights based on the SATD cost of each mode and multiply them by the prediction blocks generated for each mode to derive the final prediction blocks.

[0183] For example, DIMD (Decoder-side Intra Mode Derivation) can be applied by modifying weights based on the sample position when the difference in size between the upper or left histogram is more than 2 times.

[0184] For example, CIIP (Combined Inter-Intra Prediction) can perform a weighted sum of inter-prediction signals and intra-prediction signals based on weights.

[0185] In one embodiment, blending between the first sub-partition (sub-partition1) and the second sub-partition (sub-partition2) may be performed on the prediction samples of the first extended region and the prediction samples of the second extended region.

[0186] Specifically, referring to FIGS. 8 and FIGS. 9, in step S910, a first prediction sample for the first extended region (Extended Region 1) can be generated.

[0187] In one embodiment, a first prediction sample may be generated based on the prediction mode induction method described through FIGS. 4 to 7.

[0188] Next, in step S920, a first restoration sample for the first extended region (Extended Region1) can be generated based on the first prediction sample.

[0189] Next, in step S930, a second prediction sample for the second extended region (Extended Region 2) may be generated. The second prediction sample may be generated based on the first restoration sample.

[0190] In one embodiment, a second prediction sample may be generated based on the prediction mode induction method described through FIGS. 4 to 7.

[0191] Next, in step S940, a second restoration sample for the second extended region (Extended Region2) can be generated based on the second prediction sample.

[0192] Next, in step S950, blending can be performed on the overlapped region between the first prediction sample and the second prediction sample.

[0193] Specifically, blending can be performed on the overlapped region between the first prediction sample and the second prediction sample. By performing blending between the first prediction sample and the second prediction sample, the prediction efficiency of the prediction block for the current block can be improved.

[0194] In another embodiment, blending between the first sub-partition (sub-partition1) and the second sub-partition (sub-partition2) may be performed on the prediction samples of the first extended region and the prediction samples of the second extended region.

[0195] Specifically, referring to FIGS. 8 and FIGS. 10, in step S1010, a first prediction sample for the first extended region (Extended Region 1) may be generated. The first prediction sample may be generated based on motion information for the first extended region (Extended Region 1).

[0196] Next, in step S1020, a second prediction sample for the second extended region (Extended Region 2) may be generated. The second prediction sample may be generated based on motion information for the second extended region (Extended Region 2).

[0197] Next, in step S1030, blending can be performed between the first prediction sample and the second prediction sample.

[0198] Specifically, blending can be performed on the overlapped region between the first prediction sample and the second prediction sample. By performing blending between the first prediction sample and the second prediction sample, the prediction efficiency of the prediction block for the current block can be improved.

[0199] In another embodiment, blending between the first sub-partition (sub-partition1) and the second sub-partition (sub-partition2) may be performed on the recovery sample of the first extended area and the recovery sample of the second extended area.

[0200] Specifically, referring to FIGS. 8 and FIGS. 11, in step S1110, a first prediction sample for the first extended region (Extended Region 1) may be generated. The first prediction sample may be generated based on the prediction mode induction method described through FIGS. 4 to 7.

[0201] Next, in step S1120, a first restoration sample for the first extended region (Extended Region1) can be generated based on the first prediction sample.

[0202] Next, in step S1130, a second prediction sample for the second extended region (Extended Region 2) may be generated. The second prediction sample may be generated based on the first restoration sample. The second prediction sample may be generated based on the prediction mode induction method described through FIGS. 4 to 7.

[0203] Next, in step S1140, a second restoration sample for the second extended region (Extended Region2) can be generated based on the second prediction sample.

[0204] Next, in step S1150, blending can be performed between the first restoration sample and the second restoration sample.

[0205] Specifically, blending can be performed on the overlapped region between the first reconstructed sample and the second reconstructed sample. By performing blending between the first reconstructed sample and the second reconstructed sample, blocking artifacts between the boundaries of the sub-partitions can be suppressed, and a high-quality reference sample can be provided.

[0206] Whether to apply the sub-partition-based blending method described in FIGS. 8 to 11 can be determined based on the coding information of adjacent sub-partitions. For example, regarding whether to apply the blending method between sub-partitions, if the prediction modes of adjacent sub-partitions are the same, blending may not be applied between adjacent sub-partitions.

[0207]

[0208] FIG. 12 is a diagram illustrating a sub-partition-based filtering method according to an embodiment of the present invention.

[0209] Referring to FIG. 12, the Current Block may be divided into a first sub-partition (sub-partition 1) to a fourth sub-partition (sub-partition 4). Filtering may be performed on a first filtering region (Filtering Region 1) to a third filtering region (Filtering Region 3), which is a boundary region between the first sub-partition (sub-partition 1) to the fourth sub-partition (sub-partition 4). Each of the first filtering region (Filtering Region 1) to the third filtering region (Filtering Region 3) may include samples around the boundary between adjacent sub-partitions among the first sub-partition (sub-partition 1) to the fourth sub-partition (sub-partition 4). The width or height of the first filtering region (Filtering Region 1) to the third filtering region (Filtering Region 3) may be Y times the width or height of the first sub-partition (sub-partition 1) to the fourth sub-partition (sub-partition 4). Y may be a real number greater than or equal to 0. In one embodiment, Y may be a preset value. In another embodiment, Y may be a value that is adaptively set based on the width or height of the sub-partition.

[0210] Filtering for the first filtering region (Filtering Region 1) to the third filtering region (Filtering Region 3) can be performed by applying fixed or adaptive weights based on the boundaries between adjacent sub-partitions. For example, filtering for the first filtering region (Filtering Region 1) can be performed by applying fixed or adaptive weights to the first sub-partition (sub-partition 1) and the second sub-partition (sub-partition 2). Specific embodiments regarding filtering for the first filtering region (Filtering Region 1) to the third filtering region (Filtering Region 3) are described below.

[0211] Filtering for the first filtering region (Filtering Region 1) to the third filtering region (Filtering Region 3) may be identical or similar to any one of the following filtering methods: PDPC (Position Dependent Prediction Combination), ALF (Adaptive Loop Filter), SAO (Sample Adaptive Offset), and deblocking filtering methods.

[0212] For example, PDPC (Position Dependent Prediction Combination) can minimize discontinuities in the prediction boundary by generating a final prediction sample through a weighted sum of the prediction sample for the current block and surrounding reference samples.

[0213] For example, the Adaptive Loop Filter (ALF) can minimize the mean squared error (MSE) between the original image and the reconstructed image by applying pre-set filter coefficients to the reconstructed samples according to the local characteristics of the image and performing a weighted sum.

[0214] For example, a deblocking filter can suppress blocking artifacts by determining discontinuities between adjacent pixels at the boundary of a restored block and performing adaptive filtering based on boundary strength.

[0215] For example, SAO (Sample Adaptive Offset) can correct local intensity distortion between the original image and the reconstructed image by categorizing the reconstructed samples, which have undergone deblocking filtering, according to edge direction or brightness area and adding an offset value corresponding to each category.

[0216] In one embodiment, filtering for a first filtering region (Filtering Region 1) between a first sub-partition (sub-partition 1) and a second sub-partition (sub-partition 2) may be performed based on predicted samples of the first filtering region (Filtering Region 1). The first filtering region (Filtering Region 1) may include a plurality of samples around the boundary of the first sub-partition (sub-partition 1) and the second sub-partition (sub-partition 2). For reference, predicted samples for the first sub-partition (sub-partition 1) and the second sub-partition (sub-partition 2) may be generated based on the prediction mode derivation method described through FIGS. 4 to 7. Filtering for the second filtering region (Filtering Region 2) and the third filtering region (Filtering Region 3) may also be performed in a similar manner.

[0217] In another embodiment, filtering for a first filtering region (Filtering Region 1) between a first sub-partition (sub-partition 1) and a second sub-partition (sub-partition 2) may be performed based on reconstructed samples of the first filtering region (Filtering Region 1). The first filtering region (Filtering Region 1) may include a plurality of samples around the boundary of the first sub-partition (sub-partition 1) and the second sub-partition (sub-partition 2). For reference, reconstructed samples for the first sub-partition (sub-partition 1) and the second sub-partition (sub-partition 2) may be generated based on the prediction mode derivation method described through FIGS. 4 to 7. Filtering for the second filtering region (Filtering Region 2) and the third filtering region (Filtering Region 3) may also be performed in a similar manner.

[0218] The sub-partition-based filtering method may be selectively applied to each of the first sub-partition (sub-partition1) through the fourth sub-partition (sub-partition4). For example, sub-partition-based filtering may be performed in the first filtering region (Filtering Region1) between the first sub-partition (sub-partition1) and the second sub-partition (sub-partition2), but may not be performed in the second filtering region (Filtering Region2) and the third filtering region (Filtering Region3).

[0219] In one embodiment, whether the sub-partition-based filtering method is applied can be explicitly signaled.

[0220] In one embodiment, whether to apply a sub-partition-based filtering method may be determined based on coding information of adjacent sub-partitions. The coding information may include differences in prediction directionality and average luminance sample values ​​between adjacent sub-partitions. For example, regarding the application of the sub-partition-based filtering method, if the prediction modes of adjacent sub-partitions are the same, filtering may not be applied between adjacent sub-partitions.

[0221] The sub-partition-based filtering method described in Fig. 12 can be adaptively determined based on the encoding / decoding environment or hardware conditions of the image. Specifically, the sub-partition-based filtering method can be adaptively determined based on the shape and size of the current block, the size of the Coding Tree Unit (CTU) and Virtual Pipeline Data Unit (VPDU), whether a separate block structure (Separate Tree) is applied, the quantization parameter (QP), and the encoding information of the luminance block at the same location.

[0222] Whether a sub-partition-based filtering method is applied can be explicitly signaled at the sequence / picture / slice / tile level through upper-level syntax elements within the bitstream. For example, the application of a sub-partition-based prediction method can be determined based on at least one of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Picture Header (PH), and a Slice Header (SH).

[0223] Alternatively, whether to apply a sub-partition-based filtering method can be determined based on a specific profile.

[0224] Alternatively, whether a sub-partition-based filtering method is applied can be explicitly signaled at the CTU level through block-level syntax elements within the bitstream.

[0225] In one embodiment, when it is determined that a sub-partition-based filtering method is applied, any one of the aforementioned sub-partition-based filtering methods may be applied in a fixed manner.

[0226] In another embodiment, when it is determined that a sub-partition-based filtering method is applied, any one of the aforementioned sub-partition-based filtering methods may be adaptively determined and applied based on additional information transmitted through a bitstream. For example, the additional information may be information signaled at a higher level (e.g., at the sequence / picture / slice / tile level), CTU, VPDU, or CU level.

[0227] By performing filtering on the boundaries of sub-partitions, continuity between sub-partitions is improved, and the prediction efficiency of the prediction block for the current block can be improved. In addition, blocking artifacts between the boundaries of sub-partitions can be suppressed, and high-quality reference samples can be provided.

[0228]

[0229] FIG. 13 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0230] Referring to FIG. 13, in step S1310, the current block can be divided into multiple sub-partitions to induce a prediction mode for each of the multiple sub-partitions within the current block.

[0231] Next, in step S1320, a prediction mode for the sub-partition can be induced.

[0232] In one embodiment, the current block may include two or more sub-partitions to which different prediction modes are applied.

[0233] In another embodiment, the prediction mode may include at least one of a directional prediction mode, MIP (Matrix-based Intra Prediction), NNIP (Neural-Network based Intra Prediction), EIP (Extrapolation based Intra Prediction), IntraTMP (Intra Template Matching Prediction), and IBC (Intra Block Copy).

[0234] In another embodiment, the prediction mode for a sub-partition can be derived based on reference samples adjacent to the sub-partition.

[0235] In another embodiment, the prediction mode for a sub-partition can be derived based on the Histogram of Gradient (HoG) for the pixel values ​​of reference samples adjacent to the sub-partition.

[0236] In another embodiment, the prediction mode for the sub-partition may be at least one prediction mode determined from among candidates for a predetermined prediction mode.

[0237] In another embodiment, the prediction mode for a sub-partition can be derived using the Template-based Intra Mode Derivation (TIMD) method based on a template region adjacent to the sub-partition.

[0238] In another embodiment, the prediction mode for the sub-partition can be derived based on the prediction mode of the reference block adjacent to the sub-partition.

[0239] In another embodiment, the prediction mode for the sub-partition can be derived based on the Histogram of Occurrence (HoC) for the prediction mode of the reference block.

[0240] In another embodiment, the current block is divided into a first sub-partition and a second sub-partition, and the prediction mode for the second sub-partition adjacent to the first sub-partition can be derived based on the prediction mode of the first sub-partition and the prediction mode of the reference block adjacent to the second sub-partition.

[0241] In another embodiment, the prediction mode for a sub-partition is derived based on the prediction mode of a second reference block indicated by the block vector of a reference block adjacent to the sub-partition, and the prediction mode of the second reference block may be derived based on the Histogram of Gradient (HoG) for the pixel values ​​of reference samples adjacent to the second reference block.

[0242] In another embodiment, the prediction mode for the sub-partition can be derived based on reference samples adjacent to the current block.

[0243] Next, in step S1330, a prediction for the sub-partition can be performed based on the derived prediction mode.

[0244] In one embodiment, in response to two or more prediction modes being derived for one sub-partition, the prediction for the sub-partition may be performed based on prediction samples derived through two or more prediction modes.

[0245] In another embodiment, the current block may include at least one sub-partition predicted based on information indicating a signaled prediction mode.

[0246] In another embodiment, the method further includes the step of performing blending for sub-partitions, wherein the sub-partitions include a first sub-partition and a second sub-partition, and the step of performing blending for sub-partitions may determine whether a blending execution condition is satisfied based on the coding information of the first sub-partition and the second sub-partition, and in response to the blending execution condition being satisfied, perform blending in the overlapping area of ​​the first extended sub-partition extending the first sub-partition and the second extended sub-partition extending the second sub-partition, and in response to the blending execution condition not being satisfied, do not perform blending in the overlapping area of ​​the first extended sub-partition and the second extended sub-partition.

[0247] In another embodiment, the blending execution conditions may differ between the prediction mode applied to the first sub-partition and the prediction mode applied to the second sub-partition.

[0248] In another embodiment, the step of performing filtering on sub-partitions is further included, wherein the sub-partitions include a first sub-partition and a second sub-partition, and the step of performing filtering on the sub-partitions is to perform filtering on samples adjacent to the boundary between the first sub-partition and the second sub-partition, and the filtering area is n times the height of the first sub-partition and m times the width of the first sub-partition, and n and m may be real numbers greater than or equal to 0.

[0249] In another embodiment, filtering may be performed in response to the prediction mode applied to the first sub-partition and the prediction mode applied to the second sub-partition being different.

[0250] In another embodiment, filtering may be performed based on at least one method among PDPC (Position Dependent Prediction Combination), ALF (Adaptive Loop Filter), SAO (Sample Adaptive Offset), and deblocking filter.

[0251]

[0252] FIG. 14 is a diagram illustrating an exemplary content streaming system to which an embodiment according to the present invention can be applied.

[0253] Referring to FIG. 14, a content streaming system to which an embodiment of the present invention is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0254] The encoding server described above compresses content input from multimedia input devices, such as smartphones, cameras, and CCTVs, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and CCTVs, generate the bitstream directly, the encoding server may be omitted.

[0255] The bitstream above may be generated by a video encoding method and / or video encoding device to which an embodiment of the present invention is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0256] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0257] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0258] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0259] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0260]

[0261] The above embodiments may be performed in the same or a corresponding way in the encoding device and the decoding device. Additionally, an image may be encoded / decoded using at least one of the above embodiments or a combination of at least one.

[0262] The order in which the above embodiments are applied may differ between the encoding device and the decoder. Alternatively, the order in which the above embodiments are applied may be the same between the encoding device and the decoder.

[0263] The above embodiments may be performed for each of the luminance and chrominance signals. Alternatively, the above embodiments for the luminance and chrominance signals may be performed in the same way.

[0264] In the above embodiments, methods are described based on flowcharts as a series of steps or units; however, the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps as described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, that other steps may be included, or that one or more steps of the flowcharts may be omitted without affecting the scope of the present invention.

[0265] The above embodiments may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software.

[0266] The bitstream generated by the encoding method according to the above embodiment may be stored in a non-transient computer-readable recording medium. Additionally, the bitstream stored in the non-transient computer-readable recording medium may be decoded by the decoding method according to the above embodiment.

[0267] Herein, examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0268] Although the present invention has been described above with specific details such as specific components, limited embodiments, and drawings, this is provided only to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments, and a person skilled in the art to which the invention belongs can make various modifications and variations from this description.

[0269] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all modifications equivalent to or equivalent to the claims set forth below, as well as the claims described below, shall be considered to fall within the scope of the concept of the present invention.

[0270]

[0271] The present invention can be used in a device for encoding / decoding images and a recording medium storing a bitstream.

Claims

1. A step of dividing the current block into multiple sub-partitions; A step of inducing a prediction mode for each of the plurality of sub-partitions; and The method includes the step of performing a prediction for the plurality of sub-partitions based on the above-derived prediction mode, wherein An image decoding method characterized in that the prediction mode includes at least one of a directional prediction mode, MIP (Matrix-based Intra Prediction), NNIP (Neural-Network based Intra Prediction), EIP (Extrapolation based Intra Prediction), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), DIMD (Decoder-side Intra Mode Derivation), TIMD (Template-based Intra Mode Derivation), OBIC (Occurrence-Based Intra Coding), OBMC (Overlapped Block Motion Compensation), SGPM (Spatial Geometric Partitioning Mode), CIIP (Combined Inter-Intra Prediction), Intra Merge Mode, MRL (Multiple Reference Line), and TMRL (Template-based MRL).

2. In Paragraph 1, The above current block is divided into a first sub-partition and a second sub-partition, and A first prediction mode for the above first sub-partition is induced, and A prediction for the first sub-partition is performed based on the first prediction mode, and Restoration of the first sub-partition is performed based on the predicted first sub-partition, and Based on the restored first sub-partition, a second prediction mode for the second sub-partition is derived, and A prediction for the second sub-partition is performed based on the second prediction mode, and An image decoding method characterized by performing restoration on the second sub-partition based on the predicted second sub-partition.

3. In Paragraph 1, In response to two or more prediction modes being induced for the above sub-partition, An image decoding method characterized by performing a prediction for the above sub-partition based on prediction samples derived through the above two or more prediction modes.

4. In Paragraph 1, An image decoding method characterized in that the prediction mode for the above sub-partition is derived based on reference samples adjacent to the above sub-partition.

5. In Paragraph 4, An image decoding method characterized in that the prediction mode for the above sub-partition is derived based on the Histogram of Gradient (HoG) for the pixel values ​​of reference samples adjacent to the above sub-partition.

6. In Paragraph 5, An image decoding method characterized in that the prediction mode for the above sub-partition is at least one prediction mode determined from among candidates for a predetermined prediction mode.

7. In Paragraph 4, An image decoding method characterized in that the prediction mode for the above sub-partition is derived using the Template-based Intra Mode Derivation (TIMD) method based on a template region adjacent to the above sub-partition.

8. In Paragraph 1, An image decoding method characterized in that the prediction mode for the above sub-partition is derived based on the prediction mode of a reference block adjacent to the above sub-partition.

9. In Paragraph 8, An image decoding method characterized in that the prediction mode for the above sub-partition is derived based on the Histogram of Occurrence (HoC) for the prediction mode of the above reference block.

10. In Paragraph 8, The above current block is divided into a first sub-partition and a second sub-partition, and An image decoding method characterized in that the prediction mode for a second sub-partition adjacent to the first sub-partition is derived based on the prediction mode of the first sub-partition and the prediction mode of a reference block adjacent to the second sub-partition.

11. In Paragraph 1, The prediction mode for the above sub-partition is derived based on the prediction mode of a second reference block indicated by the block vector of a reference block adjacent to the above sub-partition, and An image decoding method characterized in that the prediction mode of the second reference block is derived based on the Histogram of Gradient (HoG) for the pixel values ​​of reference samples adjacent to the second reference block.

12. In Paragraph 1, An image decoding method characterized in that the prediction mode for the above sub-partition is derived based on reference samples adjacent to the current block.

13. In Paragraph 1, A video decoding method characterized in that the above-mentioned current block includes at least one sub-partition predicted based on information indicating a signaled prediction mode.

14. In Paragraph 1, The above current block is divided into a first sub-partition and a second sub-partition, and The method further includes the step of performing blending for the first sub-partition and the second sub-partition, and The step of performing blending for the above sub-partitions is, A video decoding method characterized by performing blending in an overlapping area between a first extended sub-partition expanded from the first sub-partition and a second extended sub-partition expanded from the second sub-partition.

15. In Paragraph 14, The above blending is performed in response to the fulfillment of blending execution conditions based on the coding information of the first sub-partition and the second sub-partition, and An image decoding method characterized by the fact that the blending execution conditions above are different between the prediction mode applied to the first sub-partition and the prediction mode applied to the second sub-partition.

16. In Paragraph 1, It further includes the step of performing filtering on the above sub-partitions, and The above sub-partition includes a first sub-partition and a second sub-partition, and The step of performing filtering on the above sub-partitions is, Filtering is performed on samples adjacent to the boundary between the first subpartition and the second subpartition, and An image decoding method characterized in that the area where the filtering is performed is n times the height of the first sub-partition and m times the width of the first sub-partition, and n and m are real numbers greater than or equal to 0.

17. In Paragraph 16, An image decoding method characterized in that the filtering is performed in response to the prediction mode applied to the first sub-partition and the prediction mode applied to the second sub-partition being different.

18. In Paragraph 17, The above filtering is, An image decoding method characterized by being performed based on at least one of PDPC (Position Dependent Prediction Combination), ALF (Adaptive Loop Filter), SAO (Sample Adaptive Offset), and a deblocking filter.

19. A step of dividing the current block into multiple sub-partitions; A step of inducing a prediction mode for each of the plurality of sub-partitions; The method includes the step of performing a prediction for each of the plurality of sub-partitions based on the above-described induced prediction mode, An image encoding method characterized in that the prediction mode includes at least one of a directional prediction mode, MIP (Matrix-based Intra Prediction), NNIP (Neural-Network based Intra Prediction), EIP (Extrapolation based Intra Prediction), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), DIMD (Decoder-side Intra Mode Derivation), TIMD (Template-based Intra Mode Derivation), OBIC (Occurrence-Based Intra Coding), OBMC (Overlapped Block Motion Compensation), SGPM (Spatial Geometric Partitioning Mode), CIIP (Combined Inter-Intra Prediction), Intra Merge Mode, MRL (Multiple Reference Line), and TMRL (Template-based MRL).

20. A method for transmitting a bitstream generated by a video encoding method, The above transmission method includes the step of transmitting the bitstream, and The above image encoding method is, Step of dividing the current block into multiple sub-partitions; A step of inducing a prediction mode for each of the above plurality of sub-partitions; The method includes the step of performing a prediction for each of the sub-partitions based on the above-described induced prediction mode, wherein An image encoding method characterized in that the prediction mode includes at least one of a directional prediction mode, MIP (Matrix-based Intra Prediction), NNIP (Neural-Network based Intra Prediction), EIP (Extrapolation based Intra Prediction), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), DIMD (Decoder-side Intra Mode Derivation), TIMD (Template-based Intra Mode Derivation), OBIC (Occurrence-Based Intra Coding), OBMC (Overlapped Block Motion Compensation), SGPM (Spatial Geometric Partitioning Mode), CIIP (Combined Inter-Intra Prediction), Intra Merge Mode, MRL (Multiple Reference Line), and TMRL (Template-based MRL).