Method and Apparatus for Inter Prediction Information Encoding and Decoding Thereof

Abstract

A video encoding / decoding method and apparatus are provided. The video decoding method of the present disclosure may include the steps of: decoding a sub-block merge mode indicator of a current block; decoding a regular merge mode indicator of the current block when the sub-block merge mode indicator indicates that it is not a sub-block merge mode; and decoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates that it is a sub-block merge mode.

Description

Technology Field

[0001] The present invention relates to an image encoding / decoding method, an apparatus, and a recording medium storing a bitstream. Specifically, the present invention relates to an image encoding / decoding method and an apparatus for encoding / decoding various inter-frame prediction modes. Background Technology

[0002] Recently, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition) video, has been increasing across various application fields. As video data becomes higher in resolution and quality, the relative volume of data increases compared to conventional video data; consequently, transmission and storage costs increase when video data is transmitted using existing wired or wireless broadband lines or stored using existing storage media. To address these issues arising from the increase in video data resolution and quality, high-efficiency video encoding and decoding technologies for video with higher resolution and quality are required.

[0003] Various video compression technologies exist, such as inter-frame prediction technology that predicts pixel values ​​in the current picture from previous or subsequent pictures, intra-frame prediction technology that predicts pixel values ​​in the current picture using pixel information within the current picture, transformation and quantization technology for compressing the energy of residual signals, and entropy coding technology that assigns short codes to values ​​with high frequency and long codes to values ​​with low frequency; by utilizing these video compression technologies, video data can be effectively compressed for transmission or storage.

[0004] As the inter-frame prediction modes become more diverse to improve the prediction performance of the video encoding device, the data required to represent the inter-frame prediction modes increases, which may result in compression bit overhead. The problem to be solved

[0005] The present invention aims to provide a method for efficiently encoding / decoding various inter-frame prediction modes.

[0006] In addition, the present invention aims to provide a recording medium storing a bitstream generated by the image encoding / decoding method or device of the present invention. means of solving the problem

[0007] The image decoding method according to the present invention may include the step of decoding a sub-block merge mode indicator of a current block; the step of decoding a regular merge mode indicator of the current block when the sub-block merge mode indicator indicates that it is not a sub-block merge mode; and the step of decoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates a sub-block merge mode.

[0008] In the above image decoding method, the regular merge mode indicator may indicate that, in the case of a first value, a regular merge mode or a merge mode having motion vector difference is used to derive inter-frame prediction information of the current block, and in the case of a second value, an intra-frame combined merge mode or a geometric division merge mode is used to derive inter-frame prediction information of the current block.

[0009] In the above image decoding method, the regular merge mode indicator may determine whether to decode based on the indicator of whether to use a combined prediction within the screen of the current block.

[0010] In the above video decoding method, the regular merge mode indicator may determine whether to decode based on the skip indicator of the current block.

[0011] In the above image decoding method, the regular merge mode indicator may determine whether to decode based on the slice type of the current block.

[0012] In the above image decoding method, the regular merge mode indicator may not be decoded if at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value.

[0013] In the above image decoding method, when the regular merge mode indicator is the first value, the method may further include the step of decoding a merge mode indicator having the motion vector difference of the current block, and when the regular merge mode indicator is the second value, the method may further include the step of decoding a merge mode indicator combined within the screen of the current block.

[0014] In the above image decoding method, the merge mode indicator having the motion vector difference may be considered as a predefined value representing a regular merge mode that is not a merge mode having the motion vector difference when it is not decoded in the step of decoding the merge mode indicator having the motion vector difference.

[0015] In the above image decoding method, the merge mode indicator combined within the screen is,

[0016] In the step of decoding the merge mode indicator combined within the screen, if it is not decoded, it may be set based on at least one of the regular merge mode indicator, the indicator indicating whether the merge mode combined within the screen is used, the skip indicator, and the size of the current block.

[0017] The image encoding method according to the present invention may include the step of encoding a sub-block merge mode indicator of a current block; the step of encoding a regular merge mode indicator of the current block when the sub-block merge mode indicator indicates that it is not a sub-block merge mode; and the step of encoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates that it is a sub-block merge mode.

[0018] In the above video encoding method, the regular merge mode indicator may indicate that, in the case of a first value, a regular merge mode or a merge mode having motion vector difference is used to derive inter-frame prediction information of the current block, and in the case of a second value, an intra-frame combined merge mode or a geometric division merge mode is used to derive inter-frame prediction information of the current block.

[0019] In the above video encoding method, the regular merge mode indicator may determine whether to encode based on the indicator of whether to use a combined prediction within the screen of the current block.

[0020] In the above video encoding method, the regular merge mode indicator may determine whether to encode based on the skip indicator of the current block.

[0021] In the above video encoding method, the regular merge mode indicator may determine whether to encode based on the slice type of the current block.

[0022] In the above video encoding method, the regular merge mode indicator may not be encoded if at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value.

[0023] In the above video encoding method, when the regular merge mode indicator is the first value, the method may further include the step of encoding a merge mode indicator having the motion vector difference of the current block, and when the regular merge mode indicator is the second value, the method may further include the step of encoding a merge mode indicator combined within the frame of the current block.

[0024] In addition, the recording medium according to the present invention can store a bitstream generated by the image encoding method according to the present invention. Effects of the invention

[0025] According to the present invention, encoding efficiency can be improved by defining the encoding / decoding order of various inter-frame prediction mode information.

[0026] In addition, according to the present invention, encoding efficiency can be improved by encoding / decoding inter-frame prediction mode information based on inter-frame prediction mode capability information encoded / decoded at a higher level.

[0027] In addition, according to the present invention, a recording medium storing a bitstream generated by the image encoding / decoding method or device of the present invention may be provided.

[0028] In addition, according to the present invention, the encoding and decoding efficiency of images can be improved. Brief explanation of the drawing

[0029] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied. FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present invention is applied. Figure 3 is a diagram schematically showing the segmentation structure of an image when encoding and decoding an image. Figure 4 is a diagram illustrating an example of an in-screen prediction process. Figure 5 is a diagram illustrating an example of an inter-frame prediction process. Figure 6 is a diagram illustrating the process of transformation and quantization. Figure 7 is a diagram illustrating reference samples available for in-screen prediction. FIG. 8 is a flowchart illustrating an inter-frame prediction information encoding / decoding method according to an embodiment of the present invention. FIGS. 9 to 12 are drawings for explaining the syntax of inter-frame prediction information in merge mode according to various embodiments of the present invention. FIG. 13 is a diagram illustrating an indicator of whether to use a non-regular merge mode that is entropy decoded for a sequence parameter set. FIG. 14 is a diagram illustrating the syntax of inter-screen prediction information in merge mode according to one embodiment of the present invention. Figure 15 is a diagram illustrating whether to use an affine adaptive motion vector difference of a sequence parameter set. FIG. 16 is a diagram illustrating the syntax of inter-screen prediction information in merge mode according to one embodiment of the present invention. FIG. 17 is a diagram illustrating syntax elements representing various inter-screen prediction modes according to one embodiment of the present invention. FIG. 18 is a flowchart illustrating an image decoding method according to one embodiment of the present invention. FIG. 19 is a flowchart illustrating an image encoding method according to one embodiment of the present invention. Specific details for implementing the invention

[0030] The present invention is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals in the drawings refer to the same or similar functions across various aspects. The shapes and sizes of elements in the drawings may be exaggerated for clearer explanation. The detailed description of exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments as examples. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of exemplary embodiments is limited only by the appended claims, together with all equivalents to those claimed therein, provided they are properly described.

[0031] In the present invention, terms such as "first," "second," etc. may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0032] When it is stated that a component of the present invention is “connected” or “connected” to another component, it should be understood that it may be directly connected to or connected to the other component, or that other components may exist in between. On the other hand, when it is stated that a component is “directly connected” or “directly connected” to another component, it should be understood that no other components exist in between.

[0033] The components shown in the embodiments of the present invention are illustrated independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for the convenience of explanation; however, at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are included within the scope of the present invention as long as they do not deviate from the essence of the invention.

[0034] The terms used in this invention are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this invention, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. That is, the description in this invention that a specific configuration "comprising" does not exclude configurations other than that configuration, but means that additional configurations may be included within the scope of the practice or technical concept of this invention.

[0035] Some components of the present invention may not be essential components performing an essential function in the present invention, but may be optional components merely for enhancing performance. The present invention may be implemented by including only the components essential for realizing the essence of the present invention, excluding components used merely for performance enhancement, and a structure including only the essential components, excluding optional components used merely for performance enhancement, is also included within the scope of the rights of the present invention.

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of this specification, if it is determined that a detailed description of related known configurations or functions may obscure the gist of this specification, such detailed description is omitted; similar reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0037] In the following, "image" may refer to a single picture constituting a video, or it may refer to the video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of an image," and may also mean "encoding and / or decoding of one of the images constituting the video."

[0038] In the following, the terms "video" and "video" may be used interchangeably with the same meaning.

[0039] In the following, the target image may be an image to be encoded and / or an image to be decoded. Additionally, the target image may be an input image input to an encoding device and an input image input to a decoding device. Here, the target image may have the same meaning as the current image.

[0040] In the following, the terms "image," "picture," "frame," and "screen" may be used interchangeably with the same meaning.

[0041] In the following, the target block may be an encoding target block that is the target of encoding and / or a decoding target block that is the target of decoding. Additionally, the target block may be a current block that is the target of current encoding and / or decoding. For example, the terms "target block" and "current block" may be used interchangeably with the same meaning.

[0042] In the following, the terms "block" and "unit" may be used interchangeably with the same meaning. Alternatively, "block" may refer to a specific unit.

[0043] In the following, the terms "region" and "segment" may be used interchangeably.

[0044] In the following, a specific signal may be a signal representing a specific block. For example, the original signal may be a signal representing the target block. The prediction signal may be a signal representing the prediction block. The residual signal may be a signal representing the residual block.

[0045] In the embodiments, each of the specified information, data, flag, index and element, attribute, etc., may have a value. A value "0" of the information, data, flag, index and element, attribute, etc., may represent logical false or a first predefined value. That is to say, the value "0", false, logical false, and the first predefined value may be used interchangeably. A value "1" of the information, data, flag, index and element, attribute, etc., may represent logical true or a second predefined value. That is to say, the value "1", true, logical true, and the second predefined value may be used interchangeably.

[0046] When a variable such as i or j is used to represent a row, column, or index, the value of i may be an integer greater than or equal to 0, or an integer greater than or equal to 1. That is to say, in the embodiments, the row, column, and index, etc. may be counted from 0, or from 1.

[0048] Glossary

[0049] Encoder: Refers to a device that performs encoding. In other words, it can mean an encoding device.

[0050] Decoder: Refers to a device that performs decoding. In other words, it can mean a decoding device.

[0051] Block: An MxN array of samples. Here, M and N may represent positive integer values, and a block may commonly represent a two-dimensional array of samples. A block may represent a unit. The current block may represent a block to be encoded during encoding, or a block to be decoded during decoding. Additionally, the current block may be at least one of an encoding block, a prediction block, a residual block, or a transformation block.

[0052] Sample: The basic unit that makes up a block. Bit depth (B d From 0 to 2 depending on ) Bd - It can be expressed as a value up to 1. In the present invention, the term "sample" can be used interchangeably with "pixel" or "pixel." That is, "sample," "pixel," and "pixel" can have the same meaning.

[0053] Unit: This may refer to a unit of image encoding and decoding. In image encoding and decoding, a unit may be a region into which a single image is divided. Additionally, when an image is divided into subdivided units for encoding or decoding, a unit may refer to the divided unit. In other words, a single image can be divided into multiple units. In image encoding and decoding, predefined processing may be performed for each unit. A single unit may be further subdivided into sub-units that have a smaller size than the unit. Depending on the function, a unit may refer to a Block, Macroblock, Coding Tree Unit, Coding Tree Block, Coding Unit, Coding Block, Prediction Unit, Prediction Block, Residual Unit, Residual Block, Transform Unit, Transform Block, etc. Additionally, to distinguish it from a block, a unit may refer to a block of luminance (Luma) components, a corresponding block of chroma (Chroma) components, and syntactic elements for each block. A unit may have various sizes and shapes, and in particular, the shape of a unit may include not only squares but also geometric shapes that can be represented in two dimensions, such as rectangles, trapezoids, triangles, and pentagons. Additionally, unit information may include at least one of the following: the type of unit indicating an encoding unit, a prediction unit, a residual unit, a transformation unit, etc., the size of the unit, the depth of the unit, and the encoding and decoding order of the unit.

[0054] Coding Tree Unit: Consists of a single luminance component (Y) coding tree block and two chrominance component (Cb, Cr) coding tree blocks associated with it. It may also refer to the blocks and the syntactic elements for each block. Each coding tree unit may be partitioned using one or more partitioning methods, such as a quad tree, binary tree, or ternary tree, to form sub-units such as a coding unit, a prediction unit, and a transform unit. It may be used as a term to refer to a sample block that serves as a processing unit in the image decoding process, such as the partitioning of an input image. Here, a quad tree may refer to a quaternary tree.

[0055] If the size of the encoding block falls within a predetermined range, it may be possible to split it into quadtrees only. Here, the predetermined range may be defined as at least one of the maximum and minimum sizes of the encoding block that can be split into quadtrees only. Information indicating the maximum / minimum size of the encoding block for which quadtree-type splitting is allowed may be signaled via a bitstream, and such information may be signaled in at least one unit among a sequence, picture parameter, tile group, or slice (segment). Alternatively, the maximum / minimum size of the encoding block may be a fixed size pre-set in the encoder / decoder. For example, if the size of the encoding block corresponds to 256x256 to 64x64, it may be possible to split it into quadtrees only. Or, if the size of the encoding block is larger than the maximum size of the conversion block, it may be possible to split it into quadtrees only. In this case, the block being split may be at least one of the encoding block or the conversion block. In such cases, information indicating the splitting of the encoding block (e.g., split_flag) may be a flag indicating whether to split into quadtrees. If the size of the encoding block falls within a predetermined range, it may be possible to divide it into a binary tree or a triad tree. In this case, the above description regarding the quad tree may be applied equally to the binary tree or the triad tree.

[0056] Coding Tree Block: This term may be used to refer to any one of the Y coding tree block, Cb coding tree block, or Cr coding tree block.

[0057] Neighbor block: This may refer to a block adjacent to the current block. A block adjacent to the current block may refer to a block whose boundary meets the current block or a block located within a certain distance from the current block. A neighbor block may refer to a block adjacent to a vertex of the current block. Here, a block adjacent to a vertex of the current block may be a block vertically adjacent to a neighbor block horizontally adjacent to the current block, or a block horizontally adjacent to a neighbor block vertically adjacent to the current block. A neighbor block may also refer to a restored neighbor block.

[0058] Reconstructed Neighbor Block: This may refer to a neighbor block that has already been encoded or decoded spatially or temporally around the current block. In this case, a reconstructed neighbor block may refer to a reconstructed neighbor unit. A reconstructed spatial neighbor block may be a block within the current picture that has already been reconstructed through encoding and / or decoding. A reconstructed temporal neighbor block may be a reconstructed block or its neighbor block located at a position corresponding to the current block of the current picture within the reference image.

[0059] Unit Depth: This refers to the degree to which a unit is divided. In a tree structure, the topmost node (Root Node) corresponds to the initial, undivided unit. This topmost node can be referred to as the root node. Additionally, the topmost node can have a minimum depth value. In this case, the topmost node can have a depth of Level 0. A node with a depth of Level 1 can represent a unit created as the initial unit is divided once. A node with a depth of Level 2 can represent a unit created as the initial unit is divided twice. A node with a depth of Level n can represent a unit created as the initial unit is divided n times. A Leaf Node can be the lowest node and can be a node that cannot be further divided. The depth of a Leaf Node can be the maximum level. For example, the predefined value for the maximum level can be 3. It can be said that the Root Node has the shallowest depth, and the Leaf Node has the deepest depth. Additionally, when units are represented as a tree structure, the level at which a unit exists can represent the unit depth.

[0060] Bitstream: Can refer to a sequence of bits containing encoded image information.

[0061] Parameter Set: This corresponds to header information within the structure of the bitstream. At least one of the video parameter set, sequence parameter set, picture parameter set, and adaptation parameter set may be included in the parameter set. Additionally, the parameter set may include tile group, slice header, and tile header information. Furthermore, the tile group may refer to a group containing multiple tiles and may have the same meaning as a slice.

[0062] An adaptive parameter set may refer to a set of parameters that can be referenced and shared across different pictures, subpictures, slices, tile groups, tiles, or bricks. Additionally, subpictures, slices, tile groups, tiles, or bricks within a picture may reference different adaptive parameter sets to utilize information within those sets.

[0063] Additionally, within a picture, different adaptation parameter sets can be referenced using identifiers of different adaptation parameter sets in subpictures, slices, tile groups, tiles, or bricks.

[0064] Additionally, within a slice, tile group, tile, or brick in a subpicture, different adaptation parameter sets can be referenced using the identifiers of different adaptation parameter sets.

[0065] Additionally, an adaptive parameter set can refer to different adaptive parameter sets within a tile or brick using the identifier of a different adaptive parameter set.

[0066] Additionally, within a brick in a tile, different adaptation parameter sets can be referenced using the identifiers of different adaptation parameter sets.

[0067] Information regarding an adaptive parameter set identifier is included in the parameter set or header of the above subpicture, so that an adaptive parameter set corresponding to the adaptive parameter set identifier can be used in the subpicture.

[0068] Information regarding an adaptive parameter set identifier is included in the parameter set or header of the above tile, so that an adaptive parameter set corresponding to the adaptive parameter set identifier can be used in the tile.

[0069] The header of the above brick includes information regarding an adaptive parameter set identifier, so that an adaptive parameter set corresponding to the adaptive parameter set identifier can be used in the brick.

[0070] The above picture can be divided into one or more rows of tiles and one or more columns of tiles.

[0071] The above subpicture may be divided into one or more tile rows and one or more tile columns within the picture. The above subpicture is an area having a rectangular / square shape within the picture and may include one or more CTUs. Additionally, at least one tile / brick / slice may be included within a single subpicture.

[0072] The above tile is an area within the picture that has a rectangular or square shape and may include one or more CTUs. Additionally, the tile may be divided into one or more bricks.

[0073] The above brick may refer to one or more CTU rows within a tile. A tile may be divided into one or more bricks, and each brick may have at least one CTU row. A tile that is not divided into two or more may also refer to a brick.

[0074] The above slice may include one or more tiles within the picture and one or more bricks within the tile.

[0075] Parsing: This refers to determining the value of a syntax element by entropy decoding a bitstream, or it may refer to entropy decoding itself.

[0076] Symbol: May represent at least one of the following: a syntactic element of the unit to be encoded / decoded, a coding parameter, or a value of a transform coefficient. Additionally, the symbol may represent the target of entropy encoding or the result of entropy decoding.

[0077] Prediction Mode: This may be information indicating a mode of encoding / decoding by intra-frame prediction or a mode of encoding / decoding by inter-frame prediction.

[0078] Prediction Unit: This refers to the basic unit used when performing predictions, such as cross-frame prediction, intra-frame prediction, cross-frame reward, intra-frame reward, and motion reward. A single prediction unit may be divided into multiple partitions or multiple sub-prediction units of smaller sizes. Multiple partitions may also serve as basic units for performing prediction or reward. A partition created by the division of a prediction unit may also be a prediction unit.

[0079] Prediction Unit Partition: This can refer to a form in which prediction units are divided.

[0080] Reference Picture List: This may refer to a list containing one or more reference pictures used for cross-frame prediction or motion compensation. The types of reference picture lists may include LC (List Combined), L0 (List 0), L1 (List 1), L2 (List 2), L3 (List 3), etc., and one or more reference picture lists may be used for cross-frame prediction.

[0081] Inter Prediction Indicator: May indicate the inter-frame prediction direction (unidirectional prediction, bidirectional prediction, etc.) of the current block. Alternatively, it may indicate the number of reference images used when generating the prediction blocks for the current block. Alternatively, it may indicate the number of prediction blocks used when performing inter-frame prediction or motion compensation for the current block.

[0082] Prediction list utilization flag: Indicates whether a prediction block is generated using at least one reference image within a specific reference image list. A prediction list utilization flag can be used to derive a prediction indicator between frames, and conversely, a prediction list utilization flag can be used to derive a prediction indicator between frames. For example, if the prediction list utilization flag indicates a first value of 0, it may indicate that a prediction block is not generated using a reference image within the reference image list, and if it indicates a second value of 1, it may indicate that a prediction block can be generated using the reference image list.

[0083] Reference Picture Index: This can refer to an index in a reference picture list that points to a specific reference picture.

[0084] Reference Picture: This may refer to an image referenced by a specific block for inter-frame prediction or motion compensation. Alternatively, the reference picture may be an image containing a reference block referenced by the current block for inter-frame prediction or motion compensation. Hereinafter, the terms "reference picture" and "reference image" may be used interchangeably with the same meaning.

[0085] Motion Vector: This can be a 2D vector used for cross-frame prediction or motion compensation. A motion vector can represent the offset between the block to be encoded / decoded and the reference block. For example, (mvX, mvY) can represent a motion vector. mvX can represent the horizontal component, and mvY can represent the vertical component.

[0086] Search Range: The search range may be a 2-dimensional area where a search for motion vectors takes place during cross-frame prediction. For example, the size of the search range may be MxN. M and N may each be positive integers.

[0087] Motion Vector Candidate: This may refer to a block that serves as a prediction candidate when predicting a motion vector, or the motion vector of that block. Additionally, a motion vector candidate may be included in the motion vector candidate list.

[0088] Motion Vector Candidate List: This can refer to a list composed of one or more motion vector candidates.

[0089] Motion Vector Candidate Index: May refer to an indicator pointing to a motion vector candidate within the motion vector candidate list. May be the index of a Motion Vector Predictor.

[0090] Motion Information: This may refer to information including at least one of motion vectors, reference image indices, cross-frame prediction indicators, as well as prediction list utilization flags, reference image list information, reference images, motion vector candidates, motion vector candidate indices, merge candidates, merge indices, etc.

[0091] Merge Candidate List: Can refer to a list composed of one or more merge candidates.

[0092] Merge Candidate: This may refer to spatial merge candidates, temporal merge candidates, combined merge candidates, combined positive prediction merge candidates, zero merge candidates, etc. A merge candidate may include motion information such as cross-frame prediction indicators, reference image indices for each list, motion vectors, prediction list utilization flags, and cross-frame prediction indicators.

[0093] Merge Index: This may refer to an indicator pointing to a merge candidate within a merge candidate list. Additionally, the merge index may indicate the block that induced the merge candidate among the blocks restored spatially or temporally adjacent to the current block. Furthermore, the merge index may indicate at least one of the movement information possessed by the merge candidate.

[0094] Transform Unit: This may refer to a basic unit for performing residual signal encoding / decoding, such as transform, inverse transform, quantization, inverse quantization, and transform coefficient encoding / decoding. A single transform unit may be divided into multiple sub-transform units of smaller sizes. Here, the transform / inverse transform may include at least one of a first-order transform / inverse transform and a second-order transform / inverse transform.

[0095] Scaling: This can refer to the process of multiplying a factor by a quantized level. Transformation coefficients can be generated as a result of scaling the quantized level. Scaling can also be called dequantization.

[0096] Quantization Parameter: This may refer to a value used to generate a quantized level using a transform factor in quantization. Alternatively, it may refer to a value used to generate a transform factor by scaling the quantized level in inverse quantization. The quantization parameter may be a value mapped to the quantization step size.

[0097] Delta Quantization Parameter: This may refer to the difference between the predicted quantization parameter and the quantization parameter of the unit to be encoded / decoded.

[0098] Scan: This can refer to a method of sorting the order of units, blocks, or coefficients within a matrix. For example, sorting a 2D array into a 1D array is called a scan. Alternatively, sorting a 1D array into a 2D array can also be called a scan or inverse scan.

[0099] Transform Coefficient: This may refer to the coefficient value generated after performing a transformation in the encoder. Alternatively, it may refer to the coefficient value generated after performing at least one of entropy decoding and inverse quantization in the decoder. Quantized levels or quantized transform coefficient levels obtained by applying quantization to the transform coefficient or residual signal may also be included in the meaning of transform coefficient.

[0100] Quantized Level: This may refer to a value generated by performing quantization on transform coefficients or residual signals in an encoder. Alternatively, it may refer to the value subject to inverse quantization before it is performed in a decoder. Similarly, the quantized transform coefficient level resulting from transform and quantization may also be included within the meaning of quantized level.

[0101] Non-zero Transform Coefficient: This may refer to a transform coefficient whose magnitude is not zero, a transform coefficient level whose magnitude is not zero, or a quantized level.

[0102] Quantization Matrix: This refers to a matrix used in the quantization or inverse quantization process to improve the subjective or objective quality of an image. A quantization matrix can also be called a scaling list.

[0103] Quantization Matrix Coefficient: This can refer to each element within the quantization matrix. Quantization matrix coefficients can also be called matrix coefficients.

[0104] Default Matrix: This may refer to a predetermined quantization matrix predefined in the encoder and decoder.

[0105] Non-default Matrix: This can refer to a quantization matrix that is not predefined in the encoder and decoder and is signaled by the user.

[0106] Statistic value: A statistical value for at least one variable, encoding parameter, constant, etc., having specific values ​​that can be computed, may be at least one of the average value, weighted average value, weighted sum value, minimum value, maximum value, mode, median value, and interpolation value of said specific values.

[0107] FIG. 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied.

[0108] The encoding device (100) may be an encoder, a video encoding device, or an image encoding device. The video may include one or more images. The encoding device (100) may sequentially encode one or more images.

[0109] Referring to FIG. 1, the encoding device (100) may include a motion prediction unit (111), a motion compensation unit (112), an intra prediction unit (120), a switch (115), a subtractor (125), a converter (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse converter (170), an adder (175), a filter unit (180), and a reference picture buffer (190).

[0110] The encoding device (100) can perform encoding on an input image in intra mode and / or inter mode. Additionally, the encoding device (100) can generate a bitstream containing encoded information through encoding of the input image and can output the generated bitstream. The generated bitstream can be stored on a computer-readable recording medium or streamed via a wired / wireless transmission medium. When intra mode is used as the prediction mode, the switch (115) can be switched to intra, and when inter mode is used as the prediction mode, the switch (115) can be switched to inter. Here, intra mode may refer to an intra-frame prediction mode, and inter mode may refer to an inter-frame prediction mode. The encoding device (100) can generate a prediction block for an input block of the input image. Additionally, after the prediction block is generated, the encoding device (100) can encode a residual block using the difference (residual) of the input block and the prediction block. The input image may be referred to as the current image that is the target of the current encoding. The input block may be referred to as the current block or the block to be encoded, which is the target of the current encoding.

[0111] When the prediction mode is an intra mode, the intra prediction unit (120) may use a sample of a block that has already been encoded / decoded around the current block as a reference sample. The intra prediction unit (120) may perform spatial prediction for the current block using the reference sample and generate prediction samples for the input block through spatial prediction. Here, intra prediction may mean intra-frame prediction.

[0112] When the prediction mode is an inter mode, the motion prediction unit (111) can search for the region that best matches the input block from the reference image during the motion prediction process and derive a motion vector using the searched region. At this time, the search region can be used as the region. The reference image can be stored in the reference picture buffer (190). Here, the reference image can be stored in the reference picture buffer (190) when encoding / decoding of the reference image is processed.

[0113] The motion compensation unit (112) can generate a prediction block for the current block by performing motion compensation using a motion vector. Here, inter-prediction may mean inter-frame prediction or motion compensation.

[0114] The motion prediction unit (111) and motion compensation unit (112) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform inter-frame prediction or motion compensation, based on the encoding unit, it can determine whether the motion prediction and motion compensation method of the prediction unit included in the corresponding encoding unit is a Skip Mode, Merge Mode, Advanced Motion Vector Prediction (AMVP) Mode, or Current Picture Reference Mode, and can perform inter-frame prediction or motion compensation according to each mode.

[0115] The subtractor (125) can generate a residual block using the difference between the input block and the prediction block. The residual block may also be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal in block units.

[0116] The transformation unit (130) can generate a transform coefficient by performing a transform on the remaining block and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the remaining block. When a transform skip mode is applied, the transformation unit (130) may skip the transform on the remaining block.

[0117] A quantized level can be generated by applying quantization to a conversion coefficient or a residual signal. In the following embodiments, the quantized level may also be referred to as a conversion coefficient.

[0118] The quantization unit (140) can generate a quantized level by quantizing a transformation coefficient or residual signal according to a quantization parameter, and can output the generated quantized level. At this time, the quantization unit (140) can quantize the transformation coefficient using a quantization matrix.

[0119] The entropy encoding unit (150) can generate a bitstream and output a bitstream by performing entropy encoding according to a probability distribution on values ​​calculated by the quantization unit (140) or coding parameter values ​​calculated during the encoding process. The entropy encoding unit (150) can perform entropy encoding on information regarding a sample of an image and information for decoding an image. For example, information for decoding an image may include syntax elements, etc.

[0120] When entropy coding is applied, a small number of bits are allocated to symbols with a high probability of occurrence and a large number of bits are allocated to symbols with a low probability of occurrence, thereby representing the symbols and reducing the size of the bit sequence for the symbols to be encoded. The entropy coding unit (150) may use encoding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) for entropy coding. For example, the entropy coding unit (150) may perform entropy coding using a Variable Length Coding (VLC) table. In addition, the entropy encoding unit (150) may perform arithmetic encoding using the derived binarization method, probability model, and context model after deriving a binarization method of the target symbol and a probability model of the target symbol / bin.

[0121] The entropy encoding unit (150) can convert a 2-dimensional block form coefficient into a 1-dimensional vector form through a transform coefficient scanning method to encode a transform coefficient level (quantized level).

[0122] Coding parameters may include not only information (flags, indices, etc.) that is encoded in the encoder and signaled to the decoder, such as syntax elements, but also information derived during the encoding or decoding process, and may refer to information required when encoding or decoding images. For example, unit / block size, unit / block depth, unit / block partitioning information, unit / block shape, unit / block partitioning structure, whether to partition in quadtree form, whether to partition in binary tree form, binary tree partitioning direction (horizontal or vertical), binary tree partitioning type (symmetrical or asymmetrical), whether to partition in triad tree form, triad tree partitioning direction (horizontal or vertical), triad tree partitioning type (symmetrical or asymmetrical), whether to partition in complex tree form, complex tree partitioning direction (horizontal or vertical), complex tree partitioning type (symmetrical or asymmetrical), complex tree partitioning tree (binary tree or triad tree), prediction mode (intra-frame prediction or inter-frame prediction), intra-frame luminance prediction mode / direction, intra-frame chrominance prediction mode / direction, intra-frame partitioning information, inter-frame partitioning information, encoded block partitioning flag, predicted block partitioning flag, transform block partitioning flag, reference sample filtering method, reference sample filter tab, reference sample filter coefficients, predicted block filtering method, predicted block filter tab, predicted block Filter coefficients, prediction block boundary filtering method, prediction block boundary filter tab, prediction block boundary filter coefficients, intra-frame prediction mode, inter-frame prediction mode, motion information, motion vector, motion vector difference, reference image index, inter-frame prediction direction, inter-frame prediction indicator, prediction list utilization flag, reference image list, reference image, motion vector prediction index, motion vector prediction candidate, motion vector candidate list, whether to use merge mode, merge index, merge candidate, merge candidate list, whether to use skip mode,Interpolation filter type, Interpolation filter tab, Interpolation filter coefficients, Motion vector magnitude, Motion vector representation accuracy, Transform type, Transform magnitude, Info on whether to use 1st-order transform, Info on whether to use 2nd-order transform, 1st-order transform index, 2nd-order transform index, Info on presence of residual signal, Coded Block Pattern, Coded Block Flag, Quantization parameters, Residual quantization parameters, Quantization matrix, In-frame loop filter application status, In-frame loop filter coefficients, In-frame loop filter tab, In-frame loop filter shape / form, Deblocking filter application status, Deblocking filter coefficients, Deblocking filter tab, Deblocking filter strength, Deblocking filter shape / form, Adaptive sample offset application status, Adaptive sample offset value, Adaptive sample offset category, Adaptive sample offset type, Adaptive loop filter application status, Adaptive loop filter coefficients, Adaptive loop filter tab, Adaptive loop filter shape / form, Binarization / Debinarization method, Context model determination method, Context model update method, Regular mode execution status, Bypass mode execution Status, Context Bin, Bypass Bin, Important Factor Flag, Last Important Factor Flag, Factor Group Unit Encoding Flag, Last Important Factor Position, Flag for whether the factor value is greater than 1, Flag for whether the factor value is greater than 2, Flag for whether the factor value is greater than 3, Remaining Factor Value Information, Sign Information, Recovered Luminance Sample, Recovered Chromaticity Sample, Residual Luminance Sample, Residual Chromaticity Sample, Luminance Conversion Factor, Chromaticity Conversion Factor, Luminance Quantized Level, Chromaticity Quantized Level, Conversion Factor Level Scanning Method, Size of Decoder Side Motion Vector Search Area, Shape of Decoder Side Motion Vector Search Area, Number of Decoder Side Motion Vector Searches, CTU Size Information, Minimum Block Size Information, Maximum Block Size Information, Maximum Block Depth Information, Minimum Block Depth Information, Image Display / Output Order, Slice Identification Information, Slice Type,At least one value or a combined form of slice splitting information, tile group identification information, tile group type, tile group splitting information, tile identification information, tile type, tile splitting information, picture type, input sample bit depth, restored sample bit depth, residual sample bit depth, transform factor bit depth, quantized level bit depth, information about the luminance signal, and information about the chrominance signal may be included in the encoding parameter.

[0123] Here, signaling a flag or index may mean that in an encoder, the corresponding flag or index is entropy encoded and included in a bitstream, and in a decoder, the corresponding flag or index is entropy decoded from the bitstream.

[0124] When the encoding device (100) performs encoding through inter-prediction, the encoded current image can be used as a reference image for another image to be processed later. Accordingly, the encoding device (100) can restore or decode the encoded current image again, and can store the restored or decoded image as a reference image in the reference picture buffer (190).

[0125] The quantized level can be dequantized in the dequantization unit (160) and inverse transformed in the inverse transform unit (170). The dequantized and / or inverse transformed coefficients can be added to the prediction block through the adder (175). A reconstructed block can be generated by adding the dequantized and / or inverse transformed coefficients and the prediction block. Here, the dequantized and / or inverse transformed coefficients refer to coefficients for which at least one of dequantization and inverse transformation has been performed, and may refer to the reconstructed residual block.

[0126] The restoration block may pass through a filter section (180). The filter section (180) may apply at least one of a deblocking filter, a Sample Adaptive Offset (SAO), an Adaptive Loop Filter (ALF), etc., to the restoration sample, restoration block, or restoration image. The filter section (180) may also be referred to as an in-loop filter.

[0127] Deblocking filters can remove block distortion occurring at the boundaries between blocks. To determine whether to perform deblocking, the decision to apply the filter to the current block can be made based on samples contained in a few columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering intensity.

[0128] To compensate for encoding errors using a sample adaptive offset, an appropriate offset value can be added to the sample value. The sample adaptive offset can correct the offset from the original image on a sample-by-sample basis for the deblocked image. One method may be to divide the samples included in the image into a certain number of regions, determine the region to be offset, and apply the offset to that region, or to apply the offset by considering the edge information of each sample.

[0129] An adaptive loop filter can perform filtering based on a comparison of the reconstructed image and the original image. After dividing the samples included in the image into predetermined groups, a filter to be applied to each group can be determined, thereby performing filtering differently for each group. Information regarding whether to apply an adaptive loop filter can be signaled per coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may vary depending on each block.

[0130] The restored block or restored image that has passed through the filter unit (180) can be stored in the reference picture buffer (190). The restored block that has passed through the filter unit (180) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (180). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0131] FIG. 2 is a block diagram showing the configuration according to one embodiment of a decoding device to which the present invention is applied.

[0132] The decoding device (200) may be a decoder, a video decoding device, or an image decoding device.

[0133] Referring to FIG. 2, the decoding device (200) may include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), a motion compensation unit (250), an adder (255), a filter unit (260), and a reference picture buffer (270).

[0134] The decoding device (200) can receive a bitstream output from the encoding device (100). The decoding device (200) can receive a bitstream stored in a computer-readable recording medium or a bitstream stream streamed through a wired / wireless transmission medium. The decoding device (200) can perform decoding on the bitstream in intra mode or inter mode. Additionally, the decoding device (200) can generate a restored image or a decoded image through decoding and can output the restored image or the decoded image.

[0135] If the prediction mode used for decoding is intra mode, the switch can be switched to intra. If the prediction mode used for decoding is inter mode, the switch can be switched to inter.

[0136] The decoding device (200) can decode the input bitstream to obtain a reconstructed residual block and generate a prediction block. Once the reconstructed residual block and the prediction block are obtained, the decoding device (200) can generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may be referred to as the current block.

[0137] The entropy decoding unit (210) can generate symbols by performing entropy decoding according to the probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the inverse process of the entropy encoding method described above.

[0138] The entropy decoding unit (210) can convert a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a conversion coefficient scanning method to decode a conversion coefficient level (quantized level).

[0139] The quantized level can be inversely quantized in the inverse quantization unit (220) and inversely transformed in the inverse transformation unit (230). The quantized level can be generated as a restored residual block as a result of inverse quantization and / or inverse transformation being performed. At this time, the inverse quantization unit (220) can apply a quantization matrix to the quantized level.

[0140] When intra mode is used, the intra prediction unit (240) can generate a prediction block by performing a spatial prediction on the current block using sample values ​​of already decoded blocks around the block to be decoded.

[0141] When an inter mode is used, the motion compensation unit (250) can generate a prediction block by performing motion compensation on the current block using a motion vector and a reference image stored in the reference picture buffer (270). The motion compensation unit (250) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. To perform motion compensation, it can determine whether the motion compensation method of the prediction unit included in the corresponding encoding unit is a skip mode, merge mode, AMVP mode, or current picture reference mode based on the encoding unit, and can perform motion compensation according to each mode.

[0142] The adder (255) can generate a restored block by adding the restored residual block and the prediction block. The filter unit (260) can apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the restored block or the restored image. The filter unit (260) can output the restored image. The restored block or the restored image can be stored in a reference picture buffer (270) and used for inter-prediction. The restored block that has passed through the filter unit (260) may be part of the reference image. That is to say, the reference image may be a restored image composed of the restored blocks that have passed through the filter unit (260). The stored reference image may subsequently be used for inter-frame prediction or motion compensation.

[0143] FIG. 3 is a diagram schematically illustrating the segmentation structure of an image when encoding and decoding an image. FIG. 3 schematically illustrates an embodiment in which a single unit is divided into a plurality of sub-units.

[0144] To efficiently segment the image, a coding unit (CU) may be used in encoding and decoding. The coding unit may be used as the basic unit of image encoding / decoding. Additionally, the coding unit may be used as a unit to distinguish between intra-frame prediction mode and inter-frame prediction mode during image encoding / decoding. The coding unit may be the basic unit used for the processes of prediction, transform, quantization, inverse transform, inverse quantization, or encoding / decoding of transform coefficients.

[0145] Referring to FIG. 3, the image (300) is sequentially divided into Largest Coding Units (LCUs), and the division structure is determined in LCU units. Here, LCU can be used with the same meaning as Coding Tree Units (CTUs). The division of a unit may refer to the division of a block corresponding to the unit. The block division information may include information regarding the depth of the unit. The depth information may indicate the number and / or degree of division of the unit. A unit may be hierarchically divided into multiple sub-units based on a tree structure and having depth information. That is to say, the unit and the sub-units generated by the division of the unit may correspond to a node and a child node of the node, respectively. Each divided sub-unit may have depth information. The depth information may be information indicating the size of the CU and may be stored for each CU. Since the unit depth indicates the number and / or degree of division of the unit, the division information of the sub-unit may include information regarding the size of the sub-unit.

[0146] The partitioning structure may refer to the distribution of coding units (CUs) within the CTU (310). This distribution may be determined by whether to partition a single CU into multiple CUs (positive integers of 2 or more, including 2, 4, 8, 16, etc.). The width and height of the CUs generated by partitioning may be half the width and height of the CUs before partitioning, respectively, or may have a size smaller than the width and height of the CUs before partitioning, depending on the number of partitions. The CUs may be recursively partitioned into multiple CUs. Through recursive partitioning, at least one of the width and height of the partitioned CUs may be reduced compared to at least one of the width and height of the CUs before partitioning. The partitioning of the CUs may be performed recursively up to a predefined depth or a predefined size. For example, the depth of the CTU may be 0, and the depth of the Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, the CTU may be a coding unit having the maximum coding unit size as described above, and the SCU may be a coding unit having the minimum coding unit size. Division begins from the CTU (310), and the depth of the CU increases by 1 each time the horizontal and / or vertical size of the CU is reduced by division. For example, for each depth, the CU that is not divided may have a size of 2Nx2N. Also, for the CU that is divided, the CU of size 2Nx2N may be divided into 4 CUs of size NxN. The size of N may be reduced by half each time the depth increases by 1.

[0147] Additionally, information regarding whether a CU is divided can be expressed through the division information of the CU. The division information may be 1 bit of information. All CUs except the SCU may include division information. For example, if the value of the division information is a first value, the CU may not be divided, and if the value of the division information is a second value, the CU may be divided.

[0148] Referring to FIG. 3, a CTU with a depth of 0 can be a 64x64 block. 0 can be the minimum depth. A SCU with a depth of 3 can be an 8x8 block. 3 can be the maximum depth. CUs of 32x32 blocks and 16x16 blocks can be represented as depth 1 and depth 2, respectively.

[0149] For example, if a single encoding unit is divided into four encoding units, the width and height of the four divided encoding units may each have half the size of the encoding unit before division. For example, if a 32x32 encoding unit is divided into four encoding units, the four divided encoding units may each have a size of 16x16. When a single encoding unit is divided into four encoding units, the encoding unit can be said to have been divided into a quad-tree form (quad-tree partition).

[0150] For example, if a single encoding unit is divided into two encoding units, the width or height of the two divided encoding units may be half the size of the encoding unit before division. For example, if a 32x32 encoding unit is divided vertically into two encoding units, the two divided encoding units may each have a size of 16x32. For example, if an 8x32 encoding unit is divided horizontally into two encoding units, the two divided encoding units may each have a size of 8x16. When a single encoding unit is divided into two encoding units, the encoding unit can be said to have been partitioned into a binary tree form (binary-tree partition).

[0151] For example, when a single encoding unit is divided into three encoding units, the encoding unit can be divided into three encoding units by dividing the horizontal or vertical dimensions of the encoding unit before division in a ratio of 1:2:1. For example, if a 16x32 encoding unit is divided horizontally into three encoding units, the three divided encoding units may have dimensions of 16x8, 16x16, and 16x8, respectively, starting from the top. For example, if a 32x32 encoding unit is divided vertically into three encoding units, the three divided encoding units may have dimensions of 8x32, 16x32, and 8x32, respectively, starting from the left. When a single encoding unit is divided into three encoding units, the encoding unit can be said to have been partitioned in the form of a ternary-tree (ternary-tree partition).

[0152] The CTU (320) of Fig. 3 is an example of a CTU to which quadtree splitting, binary tree splitting and 3-part tree splitting are all applied.

[0153] As described above, to partition a CTU, at least one of quadtree partitioning, binary tree partitioning, and triad tree partitioning may be applied. Each partitioning may be applied based on a predetermined priority. For example, quadtree partitioning may be applied preferentially to the CTU. A coding unit that can no longer be quadtree partitioned may correspond to a leaf node of a quadtree. A coding unit corresponding to a leaf node of a quadtree may become a root node of a binary tree and / or a triad tree. That is, a coding unit corresponding to a leaf node of a quadtree may be binary tree partitioned, triad tree partitioned, or not partitioned further. At this time, by ensuring that quadtree partitioning is not performed again on the coding unit created by binary tree partitioning or triad tree partitioning of the coding unit corresponding to a leaf node of a quadtree, the partitioning of the block and / or signaling of partitioning information can be effectively performed.

[0154] The division of a coding unit corresponding to each node of a quadtree can be signaled using quad division information. Quad division information having a first value (e.g., '1') can indicate that the corresponding coding unit is quadtree divided. Quad division information having a second value (e.g., '0') can indicate that the corresponding coding unit is not quadtree divided. Quad division information may be a flag having a predetermined length (e.g., 1 bit).

[0155] There may be no priority between binary tree splitting and triad tree splitting. That is, encoding units corresponding to the leaf nodes of a quadtree can be binary tree split or triad tree split. Additionally, encoding units generated by binary tree splitting or triad tree splitting may be binary tree split or triad tree split again, or may not be split any further.

[0156] A partition in which there is no priority between binary tree partitioning and triad tree partitioning can be referred to as a multi-type tree partition. That is, the encoding unit corresponding to the leaf node of a quadtree can become the root node of a multi-type tree. The partition of the encoding unit corresponding to each node of the multi-type tree can be signaled using at least one of the partition status information, partition direction information, and partition tree information of the multi-type tree. For the partition of the encoding unit corresponding to each node of the multi-type tree, the partition status information, partition direction information, and partition tree information may be signaled sequentially.

[0157] Information on whether a composite tree is split with a first value (e.g., '1') may indicate that the corresponding encoding unit is split into a composite tree. Information on whether a composite tree is split with a second value (e.g., '0') may indicate that the corresponding encoding unit is not split into a composite tree.

[0158] When a encoding unit corresponding to each node of a composite tree is split, the corresponding encoding unit may further include splitting direction information. The splitting direction information may indicate the splitting direction of the composite tree split. Splitting direction information having a first value (e.g., '1') may indicate that the corresponding encoding unit is split in the vertical direction. Splitting direction information having a second value (e.g., '0') may indicate that the corresponding encoding unit is split in the horizontal direction.

[0159] When a encoding unit corresponding to each node of a composite tree is partitioned, the encoding unit may further include partition tree information. The partition tree information may indicate the tree used for the composite tree partition. Partition tree information having a first value (e.g., '1') may indicate that the encoding unit is partitioned into a binary tree. Partition tree information having a second value (e.g., '0') may indicate that the encoding unit is partitioned into a triad tree.

[0160] The splitting information, the splitting tree information, and the splitting direction information may each be flags having a predetermined length (e.g., 1 bit).

[0161] At least one of quad splitting information, information on whether a composite tree is split, splitting direction information, and splitting tree information can be entropy encoded / decoded. For the entropy encoding / decoding of the above information, information from a neighboring encoding unit adjacent to the current encoding unit may be used. For example, the splitting form (segmentation status, splitting tree, and / or splitting direction) of the left encoding unit and / or the upper encoding unit is highly likely to be similar to the splitting form of the current encoding unit. Therefore, context information for the entropy encoding / decoding of the information of the current encoding unit can be derived based on the information of the neighboring encoding unit. At this time, the information of the neighboring encoding unit may include at least one of the quad splitting information, information on whether a composite tree is split, splitting direction information, and splitting tree information of the corresponding encoding unit.

[0162] In another embodiment, among binary tree partitioning and three-part tree partitioning, binary tree partitioning may be performed first. That is, binary tree partitioning is applied first, and a encoding unit corresponding to a leaf node of the binary tree may be set as the root node of the three-part tree. In this case, quadtree partitioning and binary tree partitioning may not be performed for the encoding unit corresponding to a node of the three-part tree.

[0163] A encoding unit that is no longer divided by quadtree splitting, binary tree splitting, and / or ternary tree splitting can be a unit of encoding, prediction, and / or conversion. That is, the encoding unit may no longer be divided for prediction and / or conversion. Therefore, a splitting structure, splitting information, etc., for splitting the encoding unit into a prediction unit and / or conversion unit may not exist in the bitstream.

[0164] However, if the size of the encoding unit serving as the unit of division is larger than the size of the maximum conversion block, the encoding unit may be recursively divided until it becomes equal to or smaller than the size of the maximum conversion block. For example, if the size of the encoding unit is 64x64 and the size of the maximum conversion block is 32x32, the encoding unit may be divided into four 32x32 blocks for conversion. For example, if the size of the encoding unit is 32x64 and the size of the maximum conversion block is 32x32, the encoding unit may be divided into two 32x32 blocks for conversion. In this case, whether the encoding unit is divided for conversion is not separately signaled, but may be determined by comparing the width or height of the encoding unit with the width or height of the maximum conversion block. For example, if the width of the encoding unit is larger than the width of the maximum conversion block, the encoding unit may be divided vertically into two. In addition, if the vertical dimension of the encoding unit is greater than the vertical dimension of the maximum conversion block, the encoding unit can be divided horizontally into two halves.

[0165] Information regarding the maximum and / or minimum size of the encoding unit and information regarding the maximum and / or minimum size of the conversion block may be signaled or determined at an upper level of the encoding unit. The upper level may be, for example, a sequence level, a picture level, a tile level, a tile group level, a slice level, etc. For example, the minimum size of the encoding unit may be determined to be 4x4. For example, the maximum size of the conversion block may be determined to be 64x64. For example, the minimum size of the conversion block may be determined to be 4x4.

[0166] Information regarding the minimum size of an encoding unit corresponding to a leaf node of a quadtree (quadtree minimum size) and / or information regarding the maximum depth from the root node to a leaf node of a composite tree (composite tree maximum depth) may be signaled or determined at an upper level of the encoding unit. The upper level may be, for example, a sequence level, a picture level, a slice level, a tile group level, a tile level, etc. Information regarding the quadtree minimum size and / or information regarding the composite tree maximum depth may be signaled or determined for each of the in-frame slice and the inter-frame slice.

[0167] Difference information regarding the size of the CTU and the maximum size of the transform block may be signaled or determined at an upper level of the encoding unit. The upper level may be, for example, a sequence level, a picture level, a slice level, a tile group level, a tile level, etc. Information regarding the maximum size of the encoding unit corresponding to each node of the binary tree (binary tree maximum size) may be determined based on the size of the encoding tree unit and the difference information. The maximum size of the encoding unit corresponding to each node of the triad tree (triad tree maximum size) may have different values ​​depending on the type of slice. For example, in the case of an in-frame slice, the triad tree maximum size may be 32x32. Also, for example, in the case of an inter-frame slice, the triad tree maximum size may be 128x128. For example, the minimum size of the encoding unit corresponding to each node of the binary tree (binary tree minimum size) and / or the minimum size of the encoding unit corresponding to each node of the triad tree (triad tree minimum size) can be set as the minimum size of the encoding block.

[0168] As another example, the maximum size of a binary tree and / or the maximum size of a triad tree can be signaled or determined at the slice level. Also, the minimum size of a binary tree and / or the minimum size of a triad tree can be signaled or determined at the slice level.

[0169] Based on the size and depth information of the various blocks mentioned above, quad splitting information, information on whether a composite tree is split, splitting tree information and / or splitting direction information, etc., may or may not exist in the bitstream.

[0170] For example, if the size of the encoding unit is not larger than the minimum size of the quadtree, the encoding unit does not include quad splitting information, and the said quad splitting information can be inferred as a second value.

[0171] For example, if the size (width and height) of a encoding unit corresponding to a node of a composite tree is larger than the maximum size (width and height) of a binary tree and / or the maximum size (width and height) of a three-part tree, the encoding unit may not be divided into a binary tree and / or a three-part tree. Accordingly, information on whether the composite tree is divided is not signaled and can be inferred as a second value.

[0172] Alternatively, if the size (width and height) of the encoding unit corresponding to the node of the composite tree is equal to the minimum size (width and height) of the binary tree, or if the size (width and height) of the encoding unit is equal to twice the minimum size (width and height) of the 3-partition tree, the encoding unit may not be divided into a binary tree and / or a 3-partition tree. Accordingly, information regarding whether the composite tree is divided is not signaled and can be inferred as a second value. This is because if the encoding unit is divided into a binary tree and / or a 3-partition tree, an encoding unit smaller than the minimum size of the binary tree and / or the minimum size of the 3-partition tree is generated.

[0173] Alternatively, binary tree splitting or three-part tree splitting may be limited based on the size of a virtual pipeline data unit (hereinafter referred to as the pipeline buffer size). For example, if a encoding unit is divided into sub-coding units that are not suitable for the pipeline buffer size by binary tree splitting or three-part tree splitting, said binary tree splitting or three-part tree splitting may be limited. The pipeline buffer size may be the size of a maximum transform block (e.g., 64X64). For example, when the pipeline buffer size is 64X64, the following splitting may be limited.

[0174] - 3-partition tree partitioning for NxM (N and / or M are 128) encoding units

[0175] - Horizontal binary tree partitioning for 128xN (N <= 64) encoding units

[0176] - Vertical binary tree partitioning for Nx128 (N <= 64) encoding units

[0177] Alternatively, if the depth of the encoding unit within the composite tree corresponding to the node of the composite tree is equal to the maximum depth of the composite tree, the encoding unit may not be divided into a binary tree and / or a three-way tree. Accordingly, information regarding whether the composite tree is divided is not signaled and can be inferred as a second value.

[0178] Alternatively, information on whether the composite tree is divided can be signaled only when at least one of vertical binary tree splitting, horizontal binary tree splitting, vertical three-way splitting, and horizontal three-way splitting is possible for the encoding unit corresponding to the node of the composite tree. Otherwise, the encoding unit may not be divided into a binary tree and / or a three-way splitting. Accordingly, information on whether the composite tree is divided is not signaled and can be inferred as a second value.

[0179] Alternatively, the division direction information may be signaled only when both vertical binary tree division and horizontal binary tree division are possible for the encoding unit corresponding to the node of the composite tree, or when both vertical 3-division tree division and horizontal 3-division tree division are possible. Otherwise, the division direction information is not signaled and may be inferred as a value indicating a direction in which division is possible.

[0180] Alternatively, the split tree information may be signaled only when both vertical binary tree splitting and vertical triad tree splitting are possible for the encoding unit corresponding to the node of the composite tree, or when both horizontal binary tree splitting and horizontal triad tree splitting are possible. Otherwise, the split tree information is not signaled and may be inferred as a value indicating a splittable tree.

[0181] Figure 4 is a diagram illustrating an example of an in-screen prediction process.

[0182] The arrows from the center to the outer edge of Fig. 4 may indicate the prediction directions of the prediction modes within the screen.

[0183] In-frame encoding and / or decoding may be performed using reference samples from neighboring blocks of the current block. Neighboring blocks may be restored neighboring blocks. For example, in-frame encoding and / or decoding may be performed using the values ​​of reference samples or encoding parameters contained in the restored neighboring blocks.

[0184] A prediction block may refer to a block generated as a result of performing a prediction within the screen. A prediction block may correspond to at least one of CU, PU, ​​and TU. The unit of a prediction block may be at least one of the sizes of CU, PU, ​​and TU. A prediction block may be a square-shaped block with sizes such as 2x2, 4x4, 16x16, 32x32, or 64x64, or a rectangular-shaped block with sizes such as 2x8, 4x8, 2x16, 4x16, and 8x16.

[0185] In-frame prediction can be performed according to the in-frame prediction mode for the current block. The number of in-frame prediction modes that the current block may have can be a predefined fixed value or a value determined differently based on the attributes of the prediction block. For example, the attributes of the prediction block may include the size and shape of the prediction block.

[0186] The number of in-screen prediction modes may be fixed at N regardless of the block size. Or, for example, the number of in-screen prediction modes may be 3, 5, 9, 17, 34, 35, 36, 65, or 67. Or, the number of in-screen prediction modes may vary depending on the block size and / or the type of color component. For example, the number of in-screen prediction modes may differ depending on whether the color component is a luminance signal or a chroma signal. For example, as the block size increases, the number of in-screen prediction modes may increase. Or, the number of in-screen prediction modes for a luminance component block may be greater than the number of in-screen prediction modes for a chroma component block.

[0187] The in-frame prediction mode may be a non-directional mode or a directional mode. The non-directional mode may be a DC mode or a Planar mode, and the angular mode may be a prediction mode having a specific direction or angle. The in-frame prediction mode may be represented by at least one of a mode number, a mode value, a mode number, a mode angle, or a mode direction. The number of in-frame prediction modes may be one or more M, including the non-directional and directional modes. A step of checking whether samples included in the restored surrounding blocks can be used as reference samples for the current block to predict the current block in-frame may be performed. If there are samples that cannot be used as reference samples for the current block, the sample value of the sample that cannot be used as a reference sample may be replaced with a value obtained by copying and / or interpolating at least one sample value among the samples included in the restored surrounding blocks, and then used as a reference sample for the current block.

[0188] Figure 7 is a diagram illustrating reference samples available for in-screen prediction.

[0189] As illustrated in FIG. 7, at least one of reference sample lines 0 to 3 may be used for in-frame prediction of the current block. In FIG. 7, samples of segment A and segment F may be padded with the nearest samples of segment B and segment E, respectively, instead of being taken from restored neighboring blocks. Index information indicating the reference sample lines to be used for in-frame prediction of the current block may be signaled. For example, in FIG. 7, reference sample line indicators 0, 1, and 2 may be signaled as index information indicating reference sample lines 0, 1, and 2. If the top boundary of the current block is the boundary of the CTU, only reference sample line 0 may be available. Therefore, in this case, the index information may not be signaled. If reference sample lines other than reference sample line 0 are used, filtering for the prediction block described below may not be performed.

[0190] When making an in-screen prediction, a filter may be applied to at least one of the reference sample or the prediction sample based on at least one of the in-screen prediction mode and the size of the current block.

[0191] In Planner mode, when generating a prediction block for the current block, the sample value of the target sample can be generated using the weighted sum of the top and left reference samples of the current sample and the top-right and bottom-left reference samples of the current block, depending on the position of the target sample within the prediction block. Additionally, in DC mode, when generating a prediction block for the current block, the average value of the top and left reference samples of the current block can be used. Furthermore, in Directional mode, a prediction block can be generated using the top, left, top-right, and / or bottom-left reference samples of the current block. Real-valued interpolation may also be performed to generate the prediction sample value.

[0192] In the case of in-frame prediction between color components, a prediction block for the current block of the second color component can be generated based on the corresponding restoration block of the first color component. For example, the first color component may be a luminance component, and the second color component may be a chrominance component. For in-frame prediction between color components, parameters of a linear model between the first color component and the second color component may be derived based on a template. The template may include upper and / or left peripheral samples of the current block and corresponding upper and / or left peripheral samples of the restoration block of the first color component. For example, the parameters of the linear model may be derived using the sample value of the first color component having the maximum value among the samples in the template and the corresponding sample value of the second color component, and the sample value of the first color component having the minimum value among the samples in the template and the corresponding sample value of the second color component. Once the parameters of the linear model are derived, the corresponding restoration block can be applied to the linear model to generate a prediction block for the current block. Depending on the image format, subsampling may be performed on the peripheral samples of the restoration block of the first color component and the corresponding restoration block. For example, if one sample of the second color component corresponds to four samples of the first color component, one corresponding sample can be calculated by subsampling the four samples of the first color component. In this case, parameter derivation of the linear model and intra-frame prediction between color components can be performed based on the subsampled corresponding sample. Whether to perform intra-frame prediction between color components and / or the range of the template can be signaled as an intra-frame prediction mode.

[0193] The current block can be divided into two or four sub-blocks in the horizontal or vertical direction. The divided sub-blocks can be restored sequentially. That is, an in-frame prediction can be performed on the sub-blocks to generate sub-predicted blocks. Additionally, inverse quantization and / or inverse transformation can be performed on the sub-blocks to generate sub-residual blocks. A restored sub-block can be generated by adding the sub-predicted block to the sub-residual block. The restored sub-block can be used as a reference sample for the in-frame prediction of the lower-ranked sub-blocks. A sub-block may be a block containing a predetermined number (e.g., 16) or more samples. Thus, for example, if the current block is an 8x4 block or a 4x8 block, the current block can be divided into two sub-blocks. Also, if the current block is a 4x4 block, the current block cannot be divided into sub-blocks. If the current block has other sizes, the current block can be divided into four sub-blocks. Information regarding whether the sub-block-based in-screen prediction is performed and / or the division direction (horizontal or vertical) may be signaled. The sub-block-based in-screen prediction may be restricted to be performed only when using reference sample line 0. When the sub-block-based in-screen prediction is performed, filtering for the prediction block described below may not be performed.

[0194] A final prediction block can be generated by performing filtering on the predicted prediction blocks within the screen. The filtering can be performed by applying a predetermined weight to the filtering target sample, the left reference sample, the top reference sample, and / or the top-left reference sample. The weight and / or reference samples (range, position, etc.) used for the filtering can be determined based on at least one of the block size, the prediction mode within the screen, and the position of the filtering target sample within the prediction block. The filtering can be performed only in the case of a predetermined prediction mode within the screen (e.g., DC, planar, vertical, horizontal, diagonal, and / or adjacent diagonal mode). The adjacent diagonal mode may be a mode obtained by adding or subtracting k from the diagonal mode. For example, k may be a positive integer less than or equal to 8.

[0195] The in-frame prediction mode of the current block can be entropy encoded / decoded by predicting it from the in-frame prediction mode of blocks existing in the vicinity of the current block. If the in-frame prediction modes of the current block and the surrounding blocks are identical, information indicating that the in-frame prediction modes of the current block and the surrounding blocks are identical can be signaled using predetermined flag information. Additionally, indicator information regarding the in-frame prediction mode that is identical to the in-frame prediction mode of the current block among multiple in-frame prediction modes of surrounding blocks can be signaled. If the in-frame prediction modes of the current block and the surrounding blocks are different, the in-frame prediction mode information of the current block can be entropy encoded / decoded by performing entropy encoding / decoding based on the in-frame prediction modes of the surrounding blocks.

[0196] Figure 5 is a diagram illustrating an example of an inter-frame prediction process.

[0197] The rectangle shown in Fig. 5 can represent an image. Additionally, the arrow in Fig. 5 can indicate the prediction direction. Each image can be classified into I-picture (Intra Picture), P-picture (Predictive Picture), B-picture (Bi-predictive Picture), etc., depending on the encoding type.

[0198] Picture I can be encoded / decoded through intra-frame prediction without inter-frame prediction. Picture P can be encoded / decoded through inter-frame prediction using only reference images existing in a unidirectional direction (e.g., forward or reverse). Picture B can be encoded / decoded through inter-frame prediction using reference images existing in both directions (e.g., forward and reverse). Additionally, in the case of Picture B, it can be encoded / decoded through inter-frame prediction using reference images existing in both directions, or through inter-frame prediction using reference images existing in either the forward or reverse direction. Here, the bidirectional direction may be the forward and reverse directions. Here, when inter-frame prediction is used, the encoder may perform inter-frame prediction or motion compensation, and the decoder may perform corresponding motion compensation.

[0199] Below, the inter-screen prediction according to the embodiment is described in detail.

[0200] Inter-frame prediction or motion compensation can be performed using reference images and motion information.

[0201] Motion information for the current block can be derived during inter-frame prediction by each of the encoding device (100) and the decoding device (200). Motion information can be derived using motion information of restored surrounding blocks, motion information of a collocated block, and / or a block adjacent to the collocated block. A collocated block may be a block corresponding to the spatial position of the current block within an already restored collocated picture. Here, the collocated picture may be one picture among at least one reference picture included in a reference picture list.

[0202] The method of deriving motion information may vary depending on the prediction mode of the current block. For example, prediction modes applied for inter-frame prediction may include AMVP mode, merge mode, skip mode, merge mode with motion vector differences, sub-block merge mode, geometric division mode, inter-intra combined prediction mode, and affine inter mode. Here, the merge mode can be referred to as the motion merge mode.

[0203] For example, when AMVP is applied as a prediction mode, a motion vector candidate list can be generated by determining at least one of the motion vector of a restored surrounding block, the motion vector of a call block, the motion vector of a block adjacent to the call block, and the (0, 0) motion vector as a motion vector candidate. Motion vector candidates can be derived using the generated motion vector candidate list. Motion information of the current block can be determined based on the derived motion vector candidates. Here, the motion vector of the call block or the motion vector of a block adjacent to the call block can be referred to as a temporal motion vector candidate, and the motion vector of a restored surrounding block can be referred to as a spatial motion vector candidate.

[0204] The encoding device (100) can calculate the Motion Vector Difference (MVD) between the motion vector of the current block and a motion vector candidate, and can entropy-encode the MVD. Additionally, the encoding device (100) can generate a bitstream by entropy-encoding a motion vector candidate index. The motion vector candidate index can indicate the optimal motion vector candidate selected from among the motion vector candidates included in the motion vector candidate list. The decoding device (200) entropy-decodes the motion vector candidate index from the bitstream and can select a motion vector candidate for the block to be decoded from among the motion vector candidates included in the motion vector candidate list using the entropy-decoded motion vector candidate index. Additionally, the decoding device (200) can derive the motion vector of the block to be decoded through the sum of the entropy-decoded MVD and the motion vector candidate.

[0205] Meanwhile, the encoding device (100) can entropy-encode the resolution information of the calculated MVD. The decoding device (200) can adjust the resolution of the entropy-decoded MVD using the MVD resolution information.

[0206] Meanwhile, the encoding device (100) can calculate the Motion Vector Difference (MVD) between the motion vector of the current block and motion vector candidates based on an affine model, and can entropy encode the MVD. The decoding device (200) can derive the affine control motion vector of the block to be decoded by deriving the affine control motion vector of the block to be decoded through the sum of the entropy decoded MVD and the affine control motion vector candidates, thereby deriving motion vectors in sub-block units.

[0207] The bitstream may include a reference image index indicating a reference image. The reference image index may be entropy encoded and signaled from the encoding device (100) to the decoding device (200) via the bitstream. The decoding device (200) may generate a prediction block for a block to be decoded based on the induced motion vector and the reference image index information.

[0208] Another example of a method for deriving motion information is merge mode. Merge mode may refer to the merging of motions for multiple blocks. Merge mode may refer to a mode in which motion information of the current block is derived from the motion information of surrounding blocks. When merge mode is applied, a merge candidate list can be generated using the restored motion information of surrounding blocks and / or the motion information of the call block. Motion information may include at least one of 1) a motion vector, 2) a reference image index, and 3) an inter-frame prediction indicator. The prediction indicator may be unidirectional (L0 prediction, L1 prediction) or bidirectional.

[0209] The merge candidate list may represent a list in which motion information is stored. The motion information stored in the merge candidate list may be at least one of motion information of neighboring blocks adjacent to the current block (spatial merge candidate), motion information of a block collocated with the current block in a reference image (temporal merge candidate), new motion information generated by a combination of motion information already existing in the merge candidate list, motion information of a block encoded / decoded prior to the current block (history-based merge candidate), and zero merge candidate.

[0210] The encoding device (100) can generate a bitstream by entropy encoding at least one of a merge flag and a merge index and then signal it to the decoding device (200). The merge flag may be information indicating whether to perform a merge mode on a block-by-block basis, and the merge index may be information regarding which block among the surrounding blocks adjacent to the current block will be merged with. For example, the surrounding blocks of the current block may include at least one of the left adjacent block, the top adjacent block, and the temporally adjacent block of the current block.

[0211] Meanwhile, the encoding device (100) can entropy-encode correction information for correcting the motion vector among the motion information of the merge candidate and signal it to the decoding device (200). The decoding device (200) can correct the motion vector of the merge candidate selected by the merge index based on the correction information. Here, the correction information may include at least one of correction status information, correction direction information, and correction magnitude information. As described above, the prediction mode for correcting the motion vector of the merge candidate based on the signaled correction information can be referred to as a merge mode having a motion vector difference.

[0212] Skip mode may be a mode that applies the motion information of surrounding blocks directly to the current block. When skip mode is used, the encoding device (100) may entropy-encode information regarding which block's motion information to use as the motion information of the current block and signal it to the decoding device (200) via a bitstream. At this time, the encoding device (100) may not signal to the decoding device (200) any syntax elements regarding at least one of motion vector difference information, encoding block flags, and transform coefficient levels (quantized levels).

[0213] The subblock merge mode may refer to a mode that derives motion information at the subblock level of a coding block (CU). When the subblock merge mode is applied, a subblock merge candidate list may be generated using motion information of the subblock collocated to the current subblock in the reference image (subblock-based temporal merge candidate) and / or affine control point motion vector merge candidate.

[0214] The geometric partitioning mode may refer to a mode in which the current block is partitioned in a predetermined direction to derive motion information for each, each prediction sample is derived using the derived motion information, and each prediction sample is derived by weighted summing the derived prediction samples to derive a prediction sample of the current block.

[0215] The inter-intra combined prediction mode may refer to a mode that derives the prediction sample of the current block by weighting the prediction sample generated by inter-frame prediction and the prediction sample generated by intra-frame prediction.

[0216] The decoding device (200) can self-correct the derived motion information. The decoding device (200) can derive the motion information having the minimum SAD into corrected motion information by searching a predefined area based on the reference block indicated by the derived motion information.

[0217] The decoding device (200) can compensate for prediction samples derived through inter-frame prediction using optical flow.

[0218] Figure 6 is a diagram illustrating the process of transformation and quantization.

[0219] As illustrated in FIG. 6, a quantized level may be generated by performing a transformation and / or quantization process on the residual signal. The residual signal may be generated as the difference between the original block and the prediction block (in-frame prediction block or inter-frame prediction block). Here, the prediction block may be a block generated by in-frame prediction or inter-frame prediction. Here, the transformation may include at least one of a first transformation and a second transformation. Transformation coefficients may be generated by performing a first transformation on the residual signal, and second transformation coefficients may be generated by performing a second transformation on the transformation coefficients.

[0220] The primary transform may be performed using at least one of a plurality of predefined transform methods. For example, the plurality of predefined transform methods may include a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), or a Karhunen-Loeve Transform (KLT)-based transform. A secondary transform may be performed on the transform coefficients generated after the primary transform is performed. The transform method applied during the primary transform and / or secondary transform may be determined based on at least one of the encoding parameters of the current block and / or surrounding blocks. Alternatively, transform information indicating the transform method may be signaled. The DCT-based transform may include, for example, DCT2, DCT-8, etc. The DST-based transform may include, for example, DST-7.

[0222] Quantized levels can be generated by performing quantization on the result of a first transformation and / or a second transformation, or on the residual signal. The quantized levels can be scanned according to at least one of an up-right diagonal scan, a vertical scan, and a horizontal scan based on at least one of an in-frame prediction mode or a block size / shape. For example, the coefficients of a block can be converted into a one-dimensional vector form by scanning them using an up-right diagonal scan. Depending on the size of the transformed block and / or the in-frame prediction mode, a vertical scan that scans the two-dimensional block shape coefficients in the column direction, or a horizontal scan that scans the two-dimensional block shape coefficients in the row direction, may be used instead of an up-right diagonal scan. The scanned quantized levels can be entropy-encoded and included in a bitstream.

[0223] In the decoder, the bitstream can be entropy decoded to generate quantized levels. The quantized levels can be inverse scanned and aligned into a two-dimensional block shape. At this time, at least one of an upper-right diagonal scan, a vertical scan, and a horizontal scan can be performed as a method of inverse scanning.

[0224] Inverse quantization can be performed on the quantized level, and depending on whether a second inverse transform is performed, a second inverse transform can be performed, and depending on whether a first inverse transform is performed on the result of the second inverse transform, a first inverse transform can be performed to generate a restored residual signal.

[0225] Inverse mapping of the dynamic range can be performed on the luminance component restored through intra-frame prediction or inter-frame prediction before in-loop filtering. The dynamic range can be divided into 16 equal pieces, and a mapping function for each piece can be signaled. The mapping function can be signaled at the slice level or the tile group level. An inverse mapping function for performing the inverse mapping can be derived based on the mapping function. In-loop filtering, saving of the reference picture, and motion compensation are performed in the inversely mapped area, and the prediction block generated through inter-frame prediction can be used to generate the restoration block after being converted to the mapped area by mapping using the mapping function. However, since intra-frame prediction is performed in the mapped area, the prediction block generated by intra-frame prediction can be used to generate the restoration block without mapping / inverse mapping.

[0226] If the current block is a residual block of a chrominance component, the residual block can be converted into an inversely mapped area by performing scaling on the chrominance component of the mapped area. The availability of the scaling can be signaled at the slice level or the tile group level. The scaling may be applied only when the mapping for the luminance component is available and the division of the luminance component and the division of the chrominance component follow the same tree structure. The scaling may be performed based on the average of the sample values ​​of the luminance prediction block corresponding to the chrominance block. In this case, if the current block uses cross-frame prediction, the luminance prediction block may refer to the mapped luminance prediction block. By referencing a lookup table using the index of the piece to which the average of the sample values ​​of the luminance prediction block belongs, the value required for the scaling can be derived. Finally, by scaling the residual block using the derived value, the residual block can be converted into an inversely mapped area. Subsequent restoration of color difference component blocks, intra-frame prediction, inter-frame prediction, in-loop filtering, and saving of the reference picture can be performed in the inversely mapped area.

[0227] Information indicating whether mapping / inverse mapping of the above luminance component and color difference component is available can be signaled through a sequence parameter set.

[0228] The predicted block of the current block can be generated based on a block vector representing the displacement between the current block and the reference block within the current picture. In this way, the prediction mode that generates the predicted block by referencing the current picture can be named the Intra Block Copy (IBC) mode. The IBC mode can be applied to an MxN (M <= 64, N <= 64) encoding unit. The IBC mode may include skip mode, merge mode, AMVP mode, etc. In the case of skip mode or merge mode, a merge candidate list is constructed, and a merge index is signaled to identify a single merge candidate. The block vector of the identified merge candidate can be used as the block vector of the current block. The merge candidate list may include at least one of the following: a spatial candidate, a history-based candidate, a candidate based on the average of two candidates, or a zero-merge candidate. In the case of AMVP mode, a difference block vector may be signaled. Additionally, the predicted block vector can be derived from the left neighbor block and the top neighbor block of the current block. An index regarding which neighbor block to use can be signaled. The predicted block in IBC mode may be limited to a block within a previously restored area that is included in the current CTU or the left CTU. For example, the value of the block vector may be restricted so that the predicted block of the current block is located within the three 64x64 block areas that precede the 64x64 block to which the current block belongs in terms of encoding / decoding order. By restricting the value of the block vector in this way, memory consumption and device complexity associated with the implementation of IBC mode can be reduced.

[0230] Hereinafter, a method for encoding / decoding inter-frame prediction information according to the present invention will be described.

[0232] FIG. 8 is a flowchart illustrating an inter-frame prediction information encoding / decoding method according to an embodiment of the present invention.

[0233] Referring to FIG. 8, the inter-frame prediction information encoding / decoding can be composed of [D1] a prediction mode information encoding / decoding step and [D2] an inter-frame prediction information encoding / decoding step. Below, each step will be described in detail.

[0235] [D1] Prediction mode information encoding / decoding step

[0236] In performing encoding and decoding of inter-frame prediction information for a current encoding / decoding target block (hereinafter referred to as the “current block”), encoding and decoding of prediction mode information can be performed using at least one of a skip indicator (e.g., cu_skip_flag), a prediction mode indicator (e.g., pred_mode_flag), and a prediction mode IBC indicator (e.g., pred_mode_ibc_flag).

[0237] Prediction mode information according to one embodiment of the present invention may indicate whether to perform a skip mode of the current block, an inter-frame prediction mode, an intra-frame prediction mode, an intra-frame block copy-based prediction mode, and a palette mode. Additionally, the prediction mode information may include at least one of a skip indicator, a prediction mode indicator, and a prediction mode IBC indicator.

[0238] The skip indicator (cu_skip_flag) may indicate that there is no residual signal for the current block. Accordingly, entropy encoding / decoding, quantization / in-quantization, and transform / in-transformation may not be performed on the residual signal. The skip indicator may be a single bit with a value of '0' or '1'. Meanwhile, the skip indicator may indicate whether to perform inter-frame prediction or intra-frame block copy-based prediction for the current block.

[0239] For example, if the skip indicator 'cu_skip_flag' decoded in the decoder is '1', cross-frame prediction information (e.g., merge_idx) can be decoded.

[0240] For example, if the skip indicator 'cu_skip_flag' decoded in the decoder is '1' and the current slice type is 'I' (in-frame) slice, then in-frame block copy-based predictive skip information can be decoded.

[0241] If the slice type of the current block is an 'I' slice and the width or height of the target block is greater than 64, the skip indicator may not be entropy decoded. In other words, this may mean that for blocks under the above conditions, an in-screen block copy-based predictive skip mode is not allowed.

[0242] For example, if the skip indicator 'cu_skip_flag' decoded in the decoder is '1', entropy decoding for the residual signal may not be performed.

[0243] For example, if the skip indicator 'cu_skip_flag' decoded in the decoder is '1', inverse quantization and inverse transform may not be performed.

[0244] For example, if the skip indicator 'cu_skip_flag' decoded in the decoder is '0', the prediction mode indicator or the prediction mode IBC indicator can be decoded.

[0246] The prediction mode indicator (pred_mode_flag) may indicate whether to perform cross-frame prediction or intra-frame prediction for the current block. The prediction mode indicator may be a single bit with a value of 0 or 1.

[0247] For example, if the prediction mode indicator decoded in the decoder is '0', entropy decoding of inter-frame prediction mode related information can be performed.

[0248] For example, if the prediction mode indicator decoded by the decoder is '1', entropy decoding of the prediction mode-related information within the screen can be performed.

[0250] The prediction mode IBC indicator (pred_mode_ibc_flag) may indicate whether the current block is predicted based on in-screen block copy. The prediction mode indicator may be a single bit with a value of 0 or 1.

[0251] For example, if the in-frame block copy prediction mode indicator (pred_mode_ibc_flag) decoded by the decoder is '1', entropy decoding for the in-frame block copy prediction mode can be performed.

[0252] For example, if the intra-frame block copy prediction mode indicator (pred_mode_ibc_flag) decoded in the decoder is '0', entropy decoding can be performed on information related to inter-frame prediction or intra-frame prediction mode.

[0253] If the width or height of the current block is greater than 64, the above-mentioned in-frame block copy prediction mode indicator may not be entropy decoded. In other words, if the width or height of the target block is greater than 64, it may mean that in-frame block copy-based prediction is not allowed.

[0254] In performing entropy encoding / decoding of prediction mode information, entropy encoding / decoding can be performed differently adaptively by utilizing prediction information of spatial neighbor blocks. The prediction information of spatial neighbor blocks may be at least one of the skip indicator, prediction mode indicator, prediction mode IBC indicator, and residual signal presence indicator (cbf) of the spatial neighbor blocks.

[0255] For example, in encoding / decoding the CABAC entropy of a skip indicator, different probability models can be used depending on the number of cases where the skip indicator of a spatial neighboring block is '1'.

[0256] For example, in entropy encoding / decoding of skip indicators, CABAC can be performed if the number of cases where the skip indicator of a spatial neighboring block is '1' is greater than a specific threshold, and bypass coding can be performed otherwise.

[0258] [D2] Inter-frame prediction information encoding / decoding step

[0259] Encoding and decoding of inter-frame prediction information for the current block can be performed.

[0261] The inter-frame prediction mode may include at least one of a general merge mode that infers the motion information of the current block from the motion information of surrounding blocks and an Inter mode (AMVP mode, Advance Motion Vector Prediction mode) that derives the motion information of the current block by combining entropy-decoded motion vector difference information and a predicted motion vector.

[0262] The general merge mode can be indicated by the value of the general merge mode indicator (e.g., general_merge_flag).

[0263] For example, if the entropy-decoded 'general_merge_flag' is the first value '1', it may mean that the current block is decoded in general merge mode, and may indicate that the current block is decoded in at least one of the following merge modes: in-frame block copy-based merge mode, regular merge mode, merge mode with motion vector difference, sub-block merge mode, in-frame combined merge mode, and triangular split mode.

[0264] For example, if the entropy-decrypted 'general_merge_flag' is a second value of '0', it may mean that the current block is decrypted in AMVP mode or in-frame block copy-based inter mode.

[0265] When the skip indicator 'cu_skip_flag' decoded in the decoder is the first value '1', the general merge mode indicator (general_merge_flag) can be inferred as the first value '1' without entropy decoding.

[0266] Prediction modes applied to the above general merge mode may include an intra-frame block copy-based merge mode, a regular merge mode, a merge mode with motion vector difference (MMVD mode, Merge with MVD mode), a subblock merge mode, an intra-frame combined merge mode (or inter-intra combined prediction mode, CIIP mode, Combined Inter Intra Prediction mode), and a triangular partitioning mode (or geometric partitioning mode, GPM, Geometric Partitioning Mode).

[0267] Meanwhile, the prediction modes applied to the above skip mode may include an in-screen block copy-based merge mode, a regular merge mode, a merge mode with motion vector differences, a sub-block merge mode, a triangular division mode (or geometric division mode), etc.

[0269] Regular merge mode can use the movement information of a merge candidate selected by the decrypted merge index as the movement information of the current block. Here, regular merge mode may refer to the merge mode described in FIG. 5.

[0270] Regular merge mode can be indicated by a regular merge mode indicator (e.g., regular_merge_flag) value, said indicator can be entropy encoded / decoded.

[0271] For example, if the entropy-decoded 'regular_merge_flag' is the first value '1', it may mean that the current block is decoded in regular merge mode. If the 'regular_merge_flag' is the first value '1', merge index information may be entropy-encoded / decoded.

[0272] For example, if the entropy-decoded 'regular_merge_flag' is a second value of '0', it may mean that the current block is not in regular merge mode, but in at least one merge mode such as a merge mode with motion vector difference, sub-block merge mode, intra-frame combined merge mode, or geometric division mode.

[0273] The regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding if the general merge mode indicator (general_merge_flag) has a first value of '1' and satisfies at least one of the following conditions (condition 1 or condition 2). In other words, it can indicate that the current block is always in regular merge mode.

[0274] Condition 1: If the entropy-decoded indicator values ​​below at the upper level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), Sub-picture, Tile Group, Slice Header, etc., all have the second value '0',

[0275] Indicator for whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0276] Indicator for whether to use sub-block temporal motion vector prediction (e.g., sps_sbtmvp_enabled_flag)

[0277] Indicator for whether to use affine transformation model prediction (e.g., sps_affine_enabled_flag)

[0278] Indicator for whether to use combined prediction within the screen (e.g., sps_ciip_enabled_flag)

[0279] Indicator for whether to use triangle splitting merge mode (e.g., sps_triangle_enabled_flag)

[0280] Meanwhile, if the above indicator values ​​are the second value '0', it may mean that the corresponding merge mode is not used.

[0282] Condition 2: When the merge mode enabler (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at higher levels, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-pictures, tile groups, and slice headers, has a second value of '0', and the product of the current block's width and height is 32

[0284] As another example, the above regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding if the general merge mode indicator (general_merge_flag) has a first value of '1' and satisfies at least one of the following conditions (condition 1 or condition 2). In other words, it can indicate that the current block is always in regular merge mode.

[0285] Condition 1: When the entropy-decoded indicator values ​​below at the upper level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), Sub-Picture, Tile Group, Slice Header, etc., all have a second value of '0', and the maximum candidate count information for sub-block merge mode (MaxNumSubblockMergeCand) is 0 (MaxNumSubblockMergeCand = 0),

[0286] Indicator for whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0287] Indicator for whether to use combined prediction within the screen (e.g., sps_ciip_enabled_flag)

[0288] Indicator for whether to use triangle splitting merge mode (e.g., sps_triangle_enabled_flag)

[0290] Condition 2: When the merge mode enabler (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at higher levels, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-pictures, tile groups, and slice headers, has a second value of '0', and the product of the current block's width and height is 32

[0291] As another example, the above regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding if the general merge mode indicator (general_merge_flag) has a first value of '1' and satisfies at least one of the following conditions (condition 1 or condition 2). In other words, it can indicate that the current block is always in regular merge mode.

[0292] Condition 1: When the entropy-decoded indicator values ​​below at the upper level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), Sub-picture, Tile Group, Slice Header, etc., all have the second value '0', and the maximum candidate count information (MaxNumTriangleMergeCand) for the Triangle Merge Mode is 0 (MaxNumTriangleMergeCand = 0),

[0293] Indicator for whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0294] Indicator for whether to use sub-block temporal motion vector prediction (e.g., sps_sbtmvp_enabled_flag)

[0295] Indicator for whether to use affine transformation model prediction (e.g., sps_affine_enabled_flag)

[0296] Indicator for whether to use combined prediction within the screen (e.g., sps_ciip_enabled_flag)

[0298] Condition 2: When the merge mode enabler (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at higher levels, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-pictures, tile groups, and slice headers, has a second value of '0', and the product of the current block's width and height is 32

[0300] As another example, the above regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding if the general merge mode indicator (general_merge_flag) has a first value of '1' and satisfies at least one of the following conditions (condition 1 or condition 2). In other words, it can indicate that the current block is always in regular merge mode.

[0301] Condition 1: When the entropy-decoded indicator values ​​below at the upper level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), Sub-Picture, Tile Group, Slice Header, etc., all have a second value of '0', and the maximum candidate count information for Sub-Block Merge Mode (MaxNumSubblockMergeCand) is '0' (MaxNumSubblockMergeCand = 0), and the maximum candidate count information for Triangle Merge Mode (MaxNumTriangleMergeCand) is '0' (MaxNumTriangleMergeCand = 0),

[0302] Indicator for whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0303] Indicator for whether to use combined prediction within the screen (e.g., sps_ciip_enabled_flag)

[0305] Condition 2: When the merge mode enabler (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at higher levels, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-pictures, tile groups, and slice headers, has a second value of '0', and the product of the current block's width and height is 32

[0307] As another example, the above regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding when the general merge mode indicator (general_merge_flag) has a first value of '1' and the sub-block merge mode indicator (e.g., merge_subblock_flag) has a second value of '0' indicating that it is not a sub-block merge mode. In other words, it can always indicate that the current block is in regular merge mode.

[0309] Meanwhile, the regular merge mode indicator (e.g., regular_merge_flag) can indicate a regular merge mode or a merge mode with motion vector differences.

[0310] For example, if the entropy-decoded 'regular_merge_flag' is the first value '1', it may mean that the current block is decoded in regular merge mode or merge mode with motion vector difference.

[0312] The in-screen block copy-based skip / merge mode can be determined based on the entropy-decoded prediction mode information (skip indicator or prediction mode IBC indicator), and the movement information of the merge candidate selected by the decoded merge index can be used as the movement information of the current block, i.e., the block vector.

[0313] In the case corresponding to the block copy-based skip / merge mode within the above screen, only the merge index information can be entropy decoded.

[0315] FIGS. 9 to 12 are drawings for explaining the syntax of inter-frame prediction information in merge mode according to various embodiments of the present invention.

[0316] Figure 9 is an example of a syntax table regarding inter-frame prediction mode information that is entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0317] As shown in FIG. 9, the regular merge mode indicator (regular_merge_flag) can be entropy decoded only when at least one of the entropy-decoded indicator values ​​of 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', 'sps_ciip_enabled_flag' or 'sps_triangle_enabled_flag' at a higher level (e.g., sequence level) is '1', that is, only when the use of the corresponding mode is allowed. Otherwise, entropy decoding is not performed, and the regular merge mode indicator value can always be inferred as '1'.

[0318] If the value of at least one of the indicators 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', 'sps_ciip_enabled_flag', or 'sps_triangle_enabled_flag' is '1', the entropy decoding of the regular merge mode indicator may additionally be determined based on the width and height of the current block.

[0319] For example, as shown in Fig. 9, the regular merge mode indicator can be entropy decoded only when the product of the width and height of the current block is not 32. In other words, when the product of the width and height is 32, it is not entropy decoded and can always be inferred as '1'. That is, it can be inferred that the current block is in regular merge mode.

[0320] When at least one of the indicator values ​​among 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', 'sps_ciip_enabled_flag' or 'sps_triangle_enabled_flag' is '1', additionally when 'sps_mmvd_enabled_flag' is the first value '1', the regular merge mode indicator can always be entropy decoded regardless of the width and height of the current block as shown in FIG. 9.

[0322] Figure 10 is another example of a syntax table regarding inter-frame prediction mode information that is entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0323] Unlike in Fig. 9, the entropy decoding of the 'sps_sbtmvp_enabled_flag' and 'sps_affine_enabled_flag' indicators, which are entropy decoded at a higher level (e.g., sequence level), can be determined based on the maximum number of entropy decoded subblock merge mode indicators (MaxNumSubblockMergeCand) at other higher levels (e.g., sequence level, picture level, or slice header).

[0324] The above MaxNumSubblockMergeCand can have a value from 0 to 5 and can be derived as follows by the 'five_minus_max_num_subblock_merge_cand' information.

[0325] MaxNumSubblockMergeCand = 5- five_minus_max_num_subblock_merge_cand

[0326] If the above MaxNumSubblockMergeCand has a value of '0', it may mean that the subblock merge mode is not used for the current slice.

[0327] As shown in FIG. 10, if the maximum number information of the sub-block merge mode of the current block is greater than 0, or if at least one indicator value among 'sps_mmvd_enabled_flag', 'sps_ciip_enabled_flag' or 'sps_triangle_enabled_flag' entropy decoded at the upper level is '1', entropy decoding for the regular merge mode indicator (regular_merge_flag) can be performed.

[0329] Figure 11 is another example of a syntax table for inter-frame prediction mode information that is entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0330] Unlike in FIG. 9, the entropy decoding of the 'sps_triangle_enabled_flag' indicator can be determined based on the maximum number of entropy decoded triangular merge mode indicators (MaxNumTriangleMergeCand) at another higher level (e.g., slice header), rather than directly using the entropy decoded 'sps_triangle_enabled_flag' indicator at a higher level (e.g., sequence level). Here, the triangular merge mode may be a geometric merge mode.

[0331] The above MaxNumTriangleMergeCand can be derived as follows by the 'max_num_merge_cand_minus_max_num_triangle_cand' information decoded at the slice level.

[0332] MaxNumTriangleMergeCand = MaxNumMergeCand- max_num_merge_cand_minus_max_num_triangle_cand

[0333] The above MaxNumMergeCand may represent the number of merge candidates derived from 'six_minus_max_num_merge_cand' decoded at the slice level.

[0334] If the above MaxNumTriangleMergeCand has a value of '0', it may mean that the triangle split merge mode is not used for the current slice.

[0335] As shown in FIG. 11, if the maximum number information of the triangular partitioning merge mode of the current block is greater than 1, or if at least one of the indicator values ​​among 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', and 'sps_ciip_enabled_flag' entropy decoded at the upper level is '1', then entropy decoding for the regular merge mode indicator (regular_merge_flag) can be performed.

[0337] Figure 12 is another example of a syntax table for inter-frame prediction mode information that is entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0338] Unlike in FIG. 9, the entropy decoding of the regular merge mode indicator (regular_merge_flag) can be determined based on the maximum number information of the sub-block merge mode (MaxNumSubblockMergeCand) and the maximum number information of the triangular split merge mode (MaxNumTriangleMergeCand) that are entropy decoded at other upper levels (e.g., sequence level, picture level, or slice header) without directly using the entropy decoded 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', and 'sps_triangle_enabled_flag' indicators at upper levels (e.g., sequence level, picture level, or slice header).

[0339] The above MaxNumSubblockMergeCand can have a value from 0 to 5 and can be derived as follows by the 'five_minus_max_num_subblock_merge_cand' information.

[0340] MaxNumSubblockMergeCand = 5- five_minus_max_num_subblock_merge_cand

[0341] If the above MaxNumSubblockMergeCand has a value of '0', it may mean that the subblock unit merge mode is not used for the current block.

[0342] The above MaxNumTriangleMergeCand can be derived as follows by the 'max_num_merge_cand_minus_max_num_triangle_cand' information decoded at the slice level.

[0343] MaxNumTriangleMergeCand = MaxNumMergeCand- max_num_merge_cand_minus_max_num_triangle_cand

[0344] The above MaxNumMergeCand represents the number of merge candidates derived from 'six_minus_max_num_merge_cand' decoded at the slice level.

[0345] If the above MaxNumTriangleMergeCand has a value of '0', it may mean that the triangle split merge mode is not used for the current slice.

[0346] As shown in FIG. 12, if the maximum number of triangular partitioning merge modes of the current block is greater than 1, or the maximum number of sub-block merge modes is greater than 0, or if at least one of the indicator values ​​of 'sps_mmvd_enabled_flag' and 'sps_ciip_enabled_flag' entropy decoded at the upper level is '1', then entropy decoding for the regular merge mode indicator (regular_merge_flag) can be performed.

[0348] Figure 13 is an example of a syntax table for an indicator of whether to use non-regular merge mode that is entropy decoded in a sequence parameter set.

[0349] When the value of the non-regular merge mode enabler (e.g., sps_non_regular_merge_enabled_flag) is the first value '1', at least one of the following indicators can be entropy decoded: the merge mode enabler with motion vector difference, the sub-block temporal motion vector prediction enabler, the affine model transformation-based prediction enabler, the intra-frame combined merge mode enabler, and the triangulation merge mode enabler.

[0350] When the value of the non-regular merge mode usage indicator is the second value '0', the merge mode usage indicator with motion vector difference, the sub-block unit temporal motion vector prediction usage indicator, the affine model transformation-based prediction usage indicator, the intra-frame combined merge mode usage indicator, and the triangulation merge mode usage indicator can be inferred as the second value '0', which indicates that the prediction modes are not used, without entropy decoding the corresponding prediction modes.

[0352] Figure 14 is another example of a syntax table regarding inter-frame prediction mode information that is entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0353] The entropy decoding of the regular merge mode indicator (regular_merge_flag) can be determined based on the entropy decoded non-regular merge mode indicator values ​​at higher levels, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc.

[0354] For example, the regular merge mode indicator (regular_merge_flag) can be entropy decoded only when the value of the non-regular merge mode enabler (e.g., sps_non_regular_merge_enabled_flag) is the first value '1' and at least one of two conditions is satisfied (the decoded sps_mmvd_enabled_flag' at the upper level is the first value '1', and the product of the width and height of the current block is not 32).

[0356] The merge mode with motion vector difference, sub-block merge mode, in-screen combined merge mode, and triangular split mode are described in detail below.

[0358] The Merge mode with motion vector difference (MMVD mode) can use the motion vector among the motion information of a merge candidate selected by a decoded merge index as the motion information of the current block after correcting it based on correction information. Here, the correction information may include at least one of correction direction information (mmvd_direction_idx) and correction magnitude information (mmvd_distance_idx). Motion vector difference information can be derived through the correction information, and the motion information obtained by adding the derived motion vector difference information to the motion vector among the motion information of a merge candidate selected by the merge index can be used as the motion information of the current block.

[0359] The merge mode with the above motion vector difference can be indicated by a value of a merge mode indicator (e.g., mmvd_merge_flag) with motion vector difference, and said indicator can be entropy encoded / decoded.

[0360] For example, if the entropy-decoded 'mmvd_merge_flag' is the first value '1', it may mean that the current block is decoded in a merge mode with motion vector difference. If the above 'mmvd_merge_flag' is the first value '1', the 'mmvd_cand_flag' corresponding to merge index information, correction direction information (mmvd_distance_idx), and correction magnitude information (mmvd_direction_idx) may be entropy-decoded.

[0361] For example, if the entropy-decoded 'mmvd_merge_flag' is a second value of '0', it may mean that the current block is decoded into at least one other merge mode, such as regular merge mode, sub-block merge mode, intra-frame combined merge mode, or triangular merge mode, rather than a merge mode with motion vector difference.

[0362] For example, if the entropy-decoded 'mmvd_merge_flag' is the second value '0', it may mean that the current block is decoded in regular merge mode rather than merge mode with motion vector difference.

[0363] The above indicator may not perform entropy decoding based on the value of the indicator for whether to use entropy decoded motion vector difference merge mode at a higher level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc.

[0364] For example, the 'mmvd_merge_flag' can be entropy decoded only when the indicator for whether to use merge mode with entropy decoded motion vector differences in SPS (e.g., sps_mmvd_enabled_flag) has a first value of '1' indicating the use of merge mode with motion vector differences.

[0365] The above indicator (mmvd_merge_flag) can be set to a value of '0' if entropy decoding has not been performed. In other words, it indicates that a merge mode with motion vector difference is not applied to the current block being decoded.

[0367] The subblock merge mode may refer to a mode that induces motion information at the subblock level of the current block. When the subblock merge mode is applied, a subblock merge candidate list can be generated using motion information of the subblocks collocated to the current subblock in the reference image (subblock-based temporal merge candidate) and / or affine control point motion vector merge candidate.

[0368] The above sub-block merge mode can be indicated by a sub-block merge mode indicator (e.g., merge_subblock_flag) value, and said indicator can be entropy encoded / decoded.

[0369] For example, if the entropy-decrypted 'merge_subblock_flag' is the first value '1', it may mean that the current block is decrypted in subblock merge mode. If the 'merge_sublock_flag' is the first value '1', the subblock merge index information (merge_subblock_idx) may be entropy-decrypted.

[0370] For example, if the entropy-decoded 'merge_subblock_flag' is a second value of '0', it may mean that the current block is not in sub-block merge mode, but is decoded in at least one other merge mode, such as regular merge mode, merge mode with motion vector difference, in-frame combined merge mode, or triangular merge mode.

[0371] The above indicator may not perform entropy decoding based on at least one of the indicators for whether to use entropy decoded sub-block unit temporal motion vector prediction at a higher level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc., and the indicator for whether to use prediction based on an affine transform model.

[0372] For example, in the case where the indicator for whether to use entropy-decoded sub-block unit temporal motion vector prediction in SPS (e.g., sps_sbtmvp_enabled_flag) has a first value of '1' indicating that sub-block unit temporal motion vector is used, the indicator 'merge_subblock_flag' can be entropy-decoded.

[0373] For example, in the case where the indicator for whether to use entropy-decoded affine transform model prediction in SPS (e.g., sps_affine_enabled_flag) has a first value of '1' indicating that the affine transform model is used, the indicator 'merge_subblock_flag' can be entropy-decoded.

[0374] For example, if both 'sps_sbtmvp_enabled_flag' and 'sps_affine_enabled_flag' in the SPS have a second value of '0' indicating that sub-block unit temporal motion vector prediction and affine transform model prediction are not used, the indicator 'merge_subblock_flag' may not be entropy decoded.

[0375] The above indicator may perform entropy decoding based on the maximum number information (MaxNumSubblockMergeCand) of the subblock merge mode. For example, if MaxNumSubblockMergeCand is '0', the above indicator 'merge_subblock_flag' may not be entropy decoded.

[0376] The above indicator (merge_subblock_flag) can be set to a value of '0' if entropy has not been decoded. In other words, it indicates that the sub-block merge mode is not applied to the block currently being decoded.

[0378] The Intra-Intra Combined Merge Mode (or Inter-Intra Combined Prediction Mode, CIIP mode, Combined Inter Intra Prediction mode) may refer to a mode that derives the prediction sample of the current block by weighting the prediction sample generated by inter-frame prediction and the prediction sample generated by intra-frame prediction.

[0379] The combined merge mode within the screen above may be indicated by the value of the combined merge mode indicator (e.g., ciip_flag) within the screen, and said indicator may be entropy encoded / decoded.

[0380] For example, if the entropy-decoded 'ciip_flag' is the first value '1', it may mean that the current block is decoded in a merge mode combined within the frame. If the 'ciip_flag' is the first value '1', the merge index information (merge_idx) for generating cross-frame prediction samples may be entropy-decoded.

[0381] For example, if the entropy-decoded 'ciip_flag' is a second value of '0', it may mean that the current block is decoded into at least one other merge mode, such as a regular merge mode, a merge mode with motion vector differences, a sub-block merge mode, or a triangular merge mode, rather than an in-frame combined merge mode.

[0382] The above indicator may not perform entropy decoding based on the value of the indicator for whether to use a combined merge mode within an entropy-decoded frame at a higher level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc.

[0383] For example, 'ciip_flag' can be entropy decoded only when the entropy decoded in-frame combined merge mode usage indicator in SPS (e.g., sps_ciip_enabled_flag) has a first value of '1' indicating the use of in-frame combined merge mode.

[0384] If the above indicator is not entropy decoded, the value of the in-screen combined merge mode indicator may be set based on at least one of the in-screen combined merge mode enable / disable indicator (sps_ciip_enabled_flag), regular merge mode indicator (regular_merge_flag), skip indicator (cu_skip_flag), and the size of the current block.

[0386] The triangular partitioning merge mode (or geometric partitioning mode, GPM) may refer to a mode that divides the current block to derive motion information for each, derives prediction samples for each using the derived motion information for each, and derives prediction samples for the current block by weighted summing the derived prediction samples. Here, the geometric partitioning mode may refer to a mode that divides the current block into asymmetric sub-blocks for prediction.

[0387] The above triangular split merge mode can be indicated by a triangular split merge mode indicator (e.g., triangle_merge_flag) value, and said indicator can be entropy encoded / decoded.

[0388] For example, if the entropy-decoded 'triangle_merge_flag' is the first value '1', it may mean that the current block is decoded in triangular merge mode.

[0389] For example, if the entropy-decoded 'triangle_merge_flag' is a second value of '0', it may mean that the current block is decoded into at least one other merge mode, such as regular merge mode, merge mode with motion vector difference, sub-block merge mode, or intra-frame combined merge mode, rather than triangular merge mode.

[0390] The above indicator can perform entropy decoding based on the value of the indicator for whether to use entropy-decoded triangular partitioning merge mode at a higher level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc.

[0391] For example, the 'triangle_merge_flag' can be entropy decoded only when the entropy decoded triangular partition merge mode enabler in SPS (e.g., sps_triangle_enabled_flag) has a first value of '1' indicating the use of triangular partition merge mode.

[0392] If the above indicator is not entropy decoded, it can be set to a value of '0'. In other words, it can indicate that the triangular partition merge mode is not applied to the current block to be decoded.

[0393] Meanwhile, the execution of the triangular split merge mode can be determined based on the value of the merge mode indicator (e.g., ciip_flag) within the screen. For example, if the merge mode indicator (ciip_flag) within the screen is '0', which is a second value that does not indicate the merge mode within the screen, the current block can be determined to be in the triangular split merge mode, and the triangular split merge prediction information can be entropy decoded.

[0394] Additionally, the geometric division merge mode may be determined based on the value of a merge mode indicator (e.g., ciip_flag) within the screen. For example, if the merge mode indicator (ciip_flag) within the screen is '0', which is a second value that does not indicate a merge mode within the screen, the geometric division merge prediction information may be entropy decoded.

[0396] The triangular split merge mode can be indicated by the triangular split merge mode indicator value, and the indicator may set the value of the indicator (e.g., MergeTriangleFlag) of the current block to '1' when the following conditions are satisfied without performing entropy encoding / decoding. Here, the triangular split merge mode may mean the geometric split merge mode. That is, the indicator for whether to use the triangular split merge mode, the maximum number of triangular split merge modes, and the triangular split merge mode indicator may mean the indicator for whether to use the geometric split merge mode, the maximum number of geometric split merge modes, and the geometric split merge mode indicator, respectively.

[0397] If at least one of the following conditions is not satisfied, the 'MergeTriangleFlag' value can be set to '0'. In other words, this may mean that the triangulation merge mode is not allowed for the current block.

[0398] If the value of the indicator for whether to use entropy-decoded triangular partition merge mode at the upper level (e.g., sps_triangle_enabled_flag) is '1',

[0399] If the slice type of the current block is B slice,

[0400] If the value of the general merge mode indicator (general_merge_flag) is '1',

[0401] If the maximum allowable number of candidates (MaxNumTriangleMergeCand) of the triangle merge mode is greater than or equal to 2,

[0402] If the product of the current block's width and height is greater than or equal to 64,

[0403] If the regular merge mode indicator (regular_merge_flag) value is '0',

[0404] If the merge mode indicator (mmvd_merge_flag) value with motion vector difference is '0',

[0405] If the sub-block merge mode indicator (merge_subblock_flag) value is '0',

[0406] If the value of the combined merge mode indicator (ciip_flag) within the screen is '0',

[0407] When the above MergeTriangleFlag is the first value '1', the split direction information indicator (merge_triangle_split_dir) and the merge index information (merge_triangle_idx0, merge_triangle_idx1) for each of the two sub-blocks can be entropy-encoded / decoded.

[0408] When the maximum number of allowed candidates (MaxNumTriangleMergeCand) of the triangle split merge mode is 2, 'merge_triangle_idx1' can use the remaining candidates excluding the candidate indicated by 'merge_triangle_idx0' without entropy encoding / decoding.

[0410] If the encoded / decoded general merge mode (general_merge_flag) for the current block is the first value '1' and all the merge mode indicator values ​​below are the second value '0', then the merge index information (merge_idx) for the current block can be inferred as '0' without entropy decoding.

[0411] Regular merge mode indicator (e.g., regular_merge_flag)

[0412] Merge mode indicator with motion vector difference (e.g., mmvd_merge_flag)

[0413] Subblock merge mode indicator (e.g., merge_subblock_flag)

[0414] In-screen combined merge mode indicators (e.g., ciip_flag)

[0415] Triangle splitting merge mode indicator (e.g., MergeTriangleFlag)

[0416] If the above conditions apply and the general merge mode (general_merge_flag) is the first value '1' and the merge index information is inferred to be '0', the movement information of the first candidate among the movement information stored in the merge candidate list can be used as the movement information of the current block.

[0418] The motion information stored in the merge candidate list may be at least one of the motion information of a neighboring block adjacent to the current block (spatial merge candidate), motion information of a block collocated with the current block in a reference image (temporal merge candidate), new motion information generated by a combination of motion information already existing in the merge candidate list, motion information of a block encoded / decoded prior to the current block (history-based merge candidate), and zero merge candidate.

[0420] As another example, if the encoded / decoded general merge mode (general_merge_flag) for the current block is the first value '1' and all the merge mode indicator values ​​below are the second value '0', the current block can be set to fallback merge mode, and after configuring the merge list with only zero merge candidates, the movement information of the first candidate can be used as the movement information of the current block.

[0421] Regular merge mode indicator (regular_merge_flag)

[0422] Merge mode indicator with motion vector difference (mmvd_merge_flag)

[0423] Subblock merge mode indicator (merge_subblock_flag)

[0424] In-screen combined merge mode indicator (ciip_flag)

[0425] Triangle Split Merge Mode Indicator (MergeTriangleFlag)

[0426] In the case where the above conditions apply and the general merge mode (general_merge_flag) is the first value '1' and the merge index information is inferred to be '0', additional correction may not be performed based on the motion vector derived from the index information.

[0427] If the above conditions apply and the general merge mode (general_merge_flag) is the first value '1', and the current block is set to fallback merge mode, additional correction may not be performed.

[0428] The above additional correction may refer to a process of correcting motion information with the minimum SAD (Decoder Motion Vector Refinement, DMVR) by searching a pre-defined area based on the reference block indicated by the motion information derived from the merge index information '0' in the decoder.

[0429] The above additional correction may refer to a process (Bi-Directional Optical Flow, BDOF) that compensates for prediction samples derived through inter-frame prediction using optical flow based on motion information derived from merge index information '0' in the decoder.

[0431] Prediction modes applied to Inter mode (or AMVP mode, Advance Motion Vector Prediction mode) may include an in-frame block copy-based Inter mode, a general AMVP mode, an affine Inter mode using an affine transformation model, a Symmetric Motion Vector Difference (SMVD) mode that derives the L1 direction MVD based on the L0 direction MVD, and an AMVR mode (Adaptive Motion Vector Resolution) that adjusts the resolution of the MVD.

[0433] The in-screen block copy intermode can be determined based on the entropy-decoded prediction mode information (prediction mode IBC indicator, e.g., pred_mode_ibc_flag), and the motion vector obtained by adding the decoded block vector difference (BVD) to the motion vector prediction candidate selected by the decoded motion vector index (e.g., mvp_l0_flag) can be used as the motion vector of the current block, i.e., the block vector.

[0435] In general AMVP mode, the motion vector obtained by summing the decoded L0 / L1 motion vector differences to the motion vector prediction candidates selected by the decoded motion vector index (e.g., mvp_lX_flag, X=0 or 1) for each of the L0 direction and L1 direction can be used as the L0 / L1 motion vector of the current block.

[0436] An inter-frame prediction indicator (inter_pred_idc) indicating whether L0, L1 unidirectional inter-frame prediction or L0 / L1 bidirectional inter-frame prediction is performed, a reference image index (ref_idx) for the L0 and L1 directions, and motion vector difference value information for the horizontal / vertical directions for the L0 and L1 directions can be entropy encoded / decoded.

[0437] If the motion vector difference value in at least one direction among the horizontal or vertical motion vector difference values ​​for the L0 and L1 directions is not '0', the adaptive motion vector difference indicator (amvr_flag) can be entropy decoded, and if the value of the indicator is '1', the indicator (amvr_precision_flag) for 1 / 16-pixel, 1 / 2-pixel, 1-pixel, or 4-pixel resolution can be entropy decoded.

[0438] For example, if 'amver_flag' has a first value of '1' and 'amvr_precision_flag' has a second value of '0', it may mean that the motion vector difference has a resolution of 1 (integer) pixel. If 'amver_flag' has a first value of '1' and 'amvr_precision_flag' has a first value of '1', it may mean that the motion vector difference has a resolution of 4 (integer) pixels. If 'amvr_flag' has a second value of '0', the motion vector difference has a resolution of 1 / 4 pixel.

[0440] The affine inter mode can use the motion vector obtained by summing the decoded L0 / L1 affine control point motion vector difference to the affine control point motion vector prediction candidate selected by the decoded motion vector index (e.g., mvp_lX_flag, X=0 or 1) for each of the L0 direction and L1 direction as the L0 / L1 affine control point motion vector of the current block.

[0441] An affine intermode indicator (inter_affine_flag) indicating whether the current block is in affine intermode can be entropy encoded / decoded.

[0442] When the above indicator (inter_affine_flag) is the first value '1', affine type information (cu_affine_type_flag) indicating whether it is a 4-parameter using 2 affine control point motion vectors or a 6-parameter using 3 affine control point motion vectors can be entropy encoded / decoded.

[0443] If 'cu_affine_type_flag' is the first value '1', it may mean a 6-parameter using three affine control point motion vectors.

[0444] If 'cu_affine_type_flag' is the second value '0', it may mean a 4-parameter using two affine control point motion vectors.

[0445] The above affine type information (cu_affine_type_flag) can determine whether to entropy encode / decode based on an indicator (e.g., sps_affine_type_flag) for using six entropy-decoded parameters at a higher level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc.

[0446] For example, if the entropy-decrypted sps_affine_type_flag in SPS is '1' and the affine intermode indicator 'inter_affine_flag' of the block to be decrypted is 1, then cu_affine_type_flag can be decrypted.

[0447] If the above affine type information does not exist, it can be inferred as the second value, '0', i.e., the 4-parameter.

[0448] Based on an inter-predation indicator (inter_pred_idc) indicating whether L0, L1 unidirectional inter-frame prediction or L0 / L1 bidirectional inter-frame prediction is performed, a reference image index (ref_idx) for L0 and L1 directions, and affine type information, motion vector difference value information for two or three L0 and L1 directions in horizontal / vertical directions can be entropy encoded / decoded.

[0450] If the motion vector difference value in at least one direction among the two or three L0 and L1 directions for horizontal or vertical motion vector difference values ​​is not '0', the adaptive motion vector difference indicator (amvr_flag) can be entropy decoded, and if the value of the indicator is '1', the indicator for 1-pixel or 1 / 16-pixel resolution (amvr_precision_flag) can be entropy decoded.

[0451] If 'amver_flag' is the first value '1' and 'amvr_precision_flag' is the second value '0', it may mean that the motion vector difference value is 1 / 16 pixel resolution.

[0452] If 'amver_flag' is the first value '1' and 'amvr_precision_flag' is the first value '1', it may mean that the motion vector difference value is 1 (integer) pixel resolution.

[0453] If 'amvr_flag' is the second value '0', the motion vector difference value may mean 1 / 4 pixel resolution.

[0455] The above adaptive motion vector difference indicator (amvr_flag) can determine whether to entropy encode / decode based on an indicator for whether to use entropy-decoded affine adaptive motion vector difference at a higher level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc. (e.g., sps_affine_amvr_enabled_flag).

[0456] As shown in FIG. 15, the above indicator for whether to use the affine adaptive motion vector difference (sps_affine_amvr_enabled_flag) can be entropy encoded / decoded only when the value of the indicator for whether to use the affine transform model prediction (sps_affine_enabled_flag) is the first value, '1'.

[0457] For example, if the 'sps_affine_amvr_enabled_flag' encoded / decoded in SPS has a first value of '1' and the current block is an affine intermode indicator 'inter_affine_flag' has a first value of '1', and at least one of the motion vector difference values ​​for the horizontal or vertical directions for two or three L0 and L1 directions is not '0', then the adaptive motion vector difference indicator (amvr_flag) can be entropy encoded / decoded.

[0459] In SMVD mode, reference image index information derived at the slice level can be used without performing entropy encoding / decoding of the reference image index (ref_idx) information for the L0 and L1 directions, respectively. Additionally, the horizontal / vertical motion vector difference values ​​in the L1 direction can be used as the horizontal / vertical motion vector difference values ​​in the L1 direction by reflecting the entropy encoded / decoded horizontal / vertical motion vector difference values ​​in the L0 direction, respectively, without performing entropy encoding / decoding of the horizontal / vertical motion vector difference values ​​in the L1 direction.

[0460] A symmetric MVD mode indicator (sym_mvd_flag) indicating whether the current block is in SMVD mode can be entropy encoded / decoded.

[0461] The above indicator (sym_mvd_flag) can determine whether to entropy encode / decode based on at least one of the following: an indicator (sps_smvd_enabled_flag) for whether to use an entropy-decoded symmetric MVD mode at a higher level such as a Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), sub-picture, tile group, slice header, etc.; an inter-pred_idc indicator indicating whether to predict L0, L1 unidirectional inter-frame prediction or L0 / L1 bidirectional inter-frame prediction; and mvd_l1_zero_flag indicating whether to entropy encode / decode in the slice header and use the L1 direction motion vector difference value as '0' without entropy decoding.

[0462] For example, if the decoded 'sps_smvd_enabled_flag' in SPS is '1', which is the first value allowing symmetric MVD, and the inter-frame prediction indicator of the current block is bidirectional prediction, and the decoded 'mvd_l1_zero_flag' is '0', which is the second value, then the indicator (sym_mvd_flag) can be entropy encoded / decoded.

[0463] For example, if the decoded 'sps_smvd_enabled_flag' in SPS is '1', which is the first value allowing symmetric MVD, and the inter-frame prediction indicator of the current block is bidirectional prediction, and the decoded 'mvd_l1_zero_flag' is '1', which is the first value, then the indicator (sym_mvd_flag) may not be entropy encoded / decoded.

[0464] If the above 'sym_mvd_flag' value does not exist, the corresponding indicator value is inferred to be '0', which may mean that symmetric MVD is not allowed for the current block.

[0466] FIG. 16 is a diagram illustrating the syntax of inter-screen prediction information in merge mode according to one embodiment of the present invention.

[0467] FIG. 16 is an example of a syntax table for inter-frame prediction mode information that is entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0468] The subblock merge indicator (merge_subblock_flag) can determine whether to perform entropy decoding based on at least one of the maximum number of entropy-decoded subblock merge modes (MaxNumSubblockMergeCand) at a higher level (e.g., sequence level or picture level), the width of the current block, and the height of the current block.

[0469] For example, if MaxNumSubblockMergeCand is greater than '0' and both the width and height of the current block are greater than or equal to 8, the subblock merge indicator can be entropy decoded.

[0470] The regular_merge_flag can determine whether entropy decoding is performed based on at least one of the sub-block merge mode indicator (merge_subblock_flag), skip indicator (cu_skip_flag), indicator for enabling in-frame combined prediction (sps_ciip_enabled_flag), current block width (cbWidth), current block height (cbHeight), current block area (cbWidth*cbHeight), slice type (slice_type), and the maximum number of triangulation merge candidates (MaxNumTriangleMergeCand).

[0471] For example, if the sub-block merge mode indicator (merge_subblock_flag) is '1', which is the first value indicating the sub-block merge mode, the regular merge mode indicator (regular_merge_flag) may not be entropy decoded.

[0472] For example, if the in-frame combined prediction enabler (sps_ciip_enabled_flag) is '1', a first value indicating that in-frame combined prediction is enabled, and the skip enabler (cu_skip_flag) is '0', a second value indicating that it is not in skip mode, the regular merge mode enabler (regular_merge_flag) can be entropy decoded.

[0474] For example, if the slice type is B slice and the maximum number of triangular merge candidates is greater than or equal to a predefined value (e.g., 2), the regular merge mode indicator (regular_merge_flag) can be entropy decoded. If the maximum number of triangular merge candidates is less than the predefined value, it may mean that the triangular merge mode is not allowed. For example, if the slice type is B slice and the triangular merge mode is allowed, the regular merge mode indicator (regular_merge_flag) can be entropy decoded.

[0475] For example, if the slice type indicates an I slice or a P slice and the skip indicator (cu_skip_flag) is '1', which is the first value indicating the skip mode, the regular merge mode indicator (regular_merge_flag) may not be entropy decoded.

[0476] For example, if the slice type indicates an I slice or a P slice and the in-frame combined prediction enabler (sps_ciip_enabled_flag) is a second value of '0' which does not indicate that in-frame combined prediction is enabled, the regular merge mode enabler (regular_merge_flag) may not be entropy decoded.

[0477] For example, if at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value (e.g., 128), the regular merge mode indicator (regular_merge_flag) may not be entropy decoded.

[0478] In addition, if conditions such as those described in FIG. 16 are satisfied, the regular merge mode indicator (regular_merge_flag) can be entropy decoded.

[0479] The merge mode indicator with motion vector difference (mmvd_merge_flag) can be entropy decoded if the regular merge mode indicator is '1', which is the first value indicating regular merge mode, and the merge mode enable indicator with motion vector difference (sps_mmvd_enabled_flag) is '1', which is the first value indicating use of merge mode with motion vector difference.

[0480] The entropy decoding of the in-frame combined merge mode indicator (ciip_flag) can be determined based on at least one of the regular merge mode indicator (regular_merge_flag), the skip indicator (cu_skip_flag), the in-frame combined prediction enablement indicator (sps_ciip_enabled_flag), the width of the current block (cbWidth), the height of the current block (cbHeight), the slice type (slice_type), and the maximum number of triangulation merge candidates (MaxNumTriangleMergeCand). The in-frame combined merge mode indicator (ciip_flag) can be entropy decoded if the regular merge mode indicator is '0', which is a second value indicating that it is not a regular merge mode.

[0481] If the maximum number of triangular merge candidates (MaxNumTriangleMergeCand) is less than 2, the in-screen combined merge mode indicator (ciip_flag) may not be entropy decoded. If the maximum number of triangular merge candidates (MaxNumTriangleMergeCand) is less than 2, it may mean that the triangular merge mode is not allowed.

[0482] If the slice type indicates an I slice or a P slice, the in-screen combined merge mode indicator (ciip_flag) may not be entropy decoded.

[0483] The merge index information (merge_idx) can be entropy decoded when the merge mode indicator (ciip_flag) within the screen is '1', which is the first value indicating the merge mode within the screen, and the maximum number of merge candidates (MaxNumMergeCand) is greater than 1.

[0484] If the in-screen merge mode indicator (ciip_flag) is not entropy decoded, the value of the in-screen merge mode indicator may be set based on at least one of the in-screen merge mode enable / disable indicator (sps_ciip_enabled_flag), regular merge mode indicator (regular_merge_flag), skip indicator (cu_skip_flag), and the size of the current block.

[0485] Therefore, even if the merge mode indicator (ciip_flag) within the screen is not entropy decoded, the value of the merge mode indicator (ciip_flag) within the screen can be set to the first value '1', so the merge index information (merge_idx) can be entropy decoded even if the merge mode indicator within the screen is not entropy decoded.

[0486] The execution of the triangular split merge mode can be determined based on the value of the merge mode indicator (ciip_flag) within the screen. For example, if the merge mode indicator (ciip_flag) within the screen is '0', which is a second value that does not indicate a merge mode within the screen, the triangular split merge prediction information (merge_tranagle_split_dir, merge_triangle_idx0, merge_triagle_idx1) can be entropy decoded. Here, the triangular split merge mode may be a geometric split merge mode.

[0488] FIG. 17 is a diagram illustrating syntax elements representing various inter-screen prediction modes according to one embodiment of the present invention.

[0489] If the value of the regular_merge_flag is '1', it may indicate that a regular merge mode or a merge mode with motion vector differences is used to derive cross-frame prediction information. If the value of the regular_merge_flag is '0', it may indicate that an intra-frame combined merge mode or a geometric division merge mode is used to derive cross-frame prediction information.

[0490] If the value of the merge mode indicator with motion vector difference (mmvd_merge_flag) is '1', it indicates that the merge mode with motion vector difference is used to derive cross-frame prediction information. If the value of the merge mode indicator with motion vector difference (mmvd_merge_flag) is '0', it indicates that the regular merge mode is used to derive cross-frame prediction information.

[0491] Meanwhile, the merge mode indicator with motion vector difference (mmvd_merge_flag) can be set to a value of '0' if entropy decoding is not performed. In other words, it indicates that the merge mode with motion vector difference is not applied to the current block being decoded and that it is in regular merge mode.

[0492] The intra-frame combined merge mode indicator (ciip_flag) can indicate whether the combination of cross-frame prediction and intra-frame prediction is applied.

[0493] Meanwhile, if the in-screen merge mode indicator (ciip_flag) is not entropy decoded, the value of the in-screen merge mode indicator may be set based on at least one of the in-screen merge mode enable / disable indicator (sps_ciip_enabled_flag), regular merge mode indicator (regular_merge_flag), skip indicator (cu_skip_flag), and the size of the current block.

[0495] FIG. 18 is a flowchart illustrating an image decoding method according to one embodiment of the present invention.

[0496] Referring to FIG. 18, the decoder can decode the sub-block merge mode indicator of the current block (e.g., merge_subblock_flag) (S1801).

[0497] And, if the sub-block merge mode indicator indicates that it is not a sub-block merge mode (S1802-No), the decoder can decode the regular merge mode indicator of the current block (e.g., regular_merge_flag) (S1803).

[0498] Here, the regular merge mode indicator may indicate that, when the first value (e.g., '1') is a regular merge mode or a merge mode with motion vector differences is used to derive inter-frame prediction information of the current block, and when the second value (e.g., '0') is a frame-combined merge mode or a geometric division merge mode is used to derive inter-frame prediction information of the current block.

[0499] The above regular merge mode indicator may determine whether to decode based on the in-screen combined prediction usage indicator of the current block (e.g., sps_ciip_enabled_flag).

[0500] The above regular merge mode indicator may determine whether to decrypt based on the skip indicator of the current block (e.g., cu_skip_flag).

[0501] The above regular merge mode indicator may determine whether to decrypt based on the slice type (e.g., slice_type) of the current block.

[0502] The above regular merge mode indicator may not be decoded if at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value (e.g., 128).

[0503] And, if the sub-block merge mode indicator indicates a sub-block merge mode (S1802-Yes), the decoder can decrypt the sub-block merge mode information of the current block (S1804). Here, the sub-block merge mode information may be a sub-block merge index (e.g., merge_subblock_idx).

[0504] Meanwhile, when the regular merge mode indicator is a first value (e.g., '1'), the decoder can decode a merge mode indicator (mmvd_merge_flag) having the motion vector difference of the current block. And, when the regular merge mode indicator is a second value (e.g., '0'), the decoder can decode a merge mode indicator (e.g., ciip_flag) combined within the screen of the current block.

[0505] If the merge mode indicator with the above motion vector difference is not entropy decoded, the merge mode indicator with the above motion vector difference may be considered as a predefined value (e.g., '0') representing a regular merge mode that is not a merge mode with the motion vector difference.

[0506] If the merge mode indicator combined within the screen is not entropy decoded, the merge mode indicator combined within the screen may be set based on at least one of a regular merge mode indicator, a merge mode usage indicator combined within the screen, a skip indicator, and the size of the current block.

[0508] FIG. 19 is a flowchart illustrating an image encoding method according to one embodiment of the present invention.

[0509] Referring to FIG. 19, the encoder can encode the subblock merge mode indicator of the current block (e.g., merge_subblock_flag) (S1901).

[0510] And, if the encoder indicates that the sub-block merge mode indicator is not a sub-block merge mode (S1902-No), it can encode the regular merge mode indicator (regular_merge_flag) of the current block (S1903).

[0511] Here, the regular merge mode indicator may indicate that, when the first value (e.g., '1') is a regular merge mode or a merge mode with motion vector differences is used to derive inter-frame prediction information of the current block, and when the second value (e.g., '0') is a frame-combined merge mode or a geometric division merge mode is used to derive inter-frame prediction information of the current block.

[0512] The above regular merge mode indicator may be encoded based on the in-screen combined prediction usage indicator of the current block (e.g., sps_ciip_enabled_flag).

[0513] The above regular merge mode indicator may be encoded based on the skip indicator of the current block (e.g., cu_skip_flag).

[0514] The above regular merge mode indicator may determine whether to encode based on the slice type (e.g., slice_type) of the current block.

[0515] The above regular merge mode indicator may not be encoded if at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value (e.g., 128).

[0516] And, if the sub-block merge mode indicator indicates a sub-block merge mode (S1902-Yes), the encoder can encode the sub-block merge mode information of the current block (S1904). Here, the sub-block merge mode information may be a sub-block merge index (e.g., merge_subblock_idx).

[0517] Meanwhile, if the regular merge mode indicator is a first value (e.g., '1'), the encoder can encode a merge mode indicator (mmvd_merge_flag) having the motion vector difference of the current block. And, if the regular merge mode indicator is a second value (e.g., '0'), the encoder can encode a merge mode indicator (e.g., ciip_flag) within the screen of the current block.

[0518] A computer-readable non-transient recording medium according to the present invention can store a bitstream generated by the image encoding method described in FIG. 19.

[0519] Specifically, in a non-transient computer-readable recording medium storing a bitstream generated by an image encoding method, the image encoding method may include the step of encoding a sub-block merge mode indicator of a current block; the step of encoding a regular merge mode indicator of the current block when the sub-block merge mode indicator indicates that it is not a sub-block merge mode; and the step of encoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates a sub-block merge mode.

[0521] The above embodiments can be performed in the same way in the encoder and decoder.

[0522] An image can be encoded / decoded using at least one of the above embodiments or a combination of at least one.

[0523] The order of applying the above embodiments may differ between the encoder and the decoder, and the order of applying the above embodiments may be the same between the encoder and the decoder.

[0524] The above example can be performed for each of the luminance and color difference signals, and the above example can be performed in the same way for the luminance and color difference signals.

[0525] The shape of the block to which the above embodiments of the present invention are applied may be square or non-square.

[0526] At least one of the syntax elements (flags, indices, etc.) that are entropy-encoded in the encoder and entropy-decoded in the decoder may use at least one of the following binaryization methods.

[0527] Binarization method for truncated rice

[0528] K-th order Exp_Golomb binarization method

[0529] Restricted K-th order Exp_Golomb binarization method

[0530] Fixed-length binarization method

[0531] Unary binarization method

[0532] Truncated Unary Binarization Method

[0533] Truncated Binary Binarization Method

[0534] The above embodiments of the present invention may be applied according to the size of at least one of an encoding block, a prediction block, a conversion block, a block, a current block, an encoding unit, a prediction unit, a conversion unit, a unit, and a current unit. The size here may be defined as a minimum size and / or a maximum size for the application of the above embodiments, or it may be defined as a fixed size for the application of the above embodiments. Furthermore, the above embodiments may be applied as a first embodiment at a first size, and as a second embodiment at a second size. That is, the above embodiments may be applied in combination according to the size. Additionally, the above embodiments of the present invention may be applied only when the size is greater than or equal to the minimum size and less than or equal to the maximum size. That is, the above embodiments may be applied only when the block size falls within a certain range.

[0535] For example, the above embodiments may be applied only when the current block size is 8x8 or larger. For example, the above embodiments may be applied only when the current block size is 4x4. For example, the above embodiments may be applied only when the current block size is 16x16 or smaller. For example, the above embodiments may be applied only when the current block size is 16x16 or larger and 64x64 or smaller.

[0536] The embodiments of the present invention may be applied according to a temporal layer. A separate identifier is signaled to identify the temporal layer to which the embodiments are applicable, and the embodiments may be applied to the temporal layer specified by the identifier. The identifier may be defined as the lowest layer and / or the highest layer to which the embodiments are applicable, or it may be defined as indicating a specific layer to which the embodiments are applied. Additionally, a fixed temporal layer to which the embodiments are applied may be defined.

[0537] For example, the above embodiments may be applied only when the temporal layer of the current image is the lowest layer. For example, the above embodiments may be applied only when the temporal layer identifier of the current image is 1 or greater. For example, the above embodiments may be applied only when the temporal layer of the current image is the highest layer.

[0538] The slice type or tile group type to which the above embodiments of the present invention are applied is defined, and the above embodiments of the present invention may be applied according to the said slice type or tile group type.

[0539] In the embodiments described above, methods are described based on flowcharts as a series of steps or units; however, the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps as described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, that other steps may be included, or that one or more steps of the flowcharts may be omitted without affecting the scope of the present invention.

[0540] The embodiments described above include examples of various aspects. While it is not possible to describe all possible combinations for representing various aspects, those skilled in the art will recognize that other combinations are possible. Accordingly, the present invention shall be deemed to include all other substitutions, modifications, and changes falling within the scope of the following claims.

[0541] The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0542] Although the present invention has been described above with specific details such as specific components, limited embodiments, and drawings, this is provided only to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments, and a person skilled in the art to which the invention belongs can make various modifications and variations from this description.

[0543] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all modifications equivalent to or equivalent to the claims set forth below, as well as the claims described below, shall be considered to fall within the scope of the concept of the present invention.

Claims

Claim 1 A video decoding method comprising: a step of decoding a sub-block merge mode indicator of a current block; and a step of decoding a regular merge mode indicator of a current block when the sub-block merge mode indicator indicates that a sub-block merge mode is not applied to the current block, wherein when the sub-block merge mode indicator indicates that the sub-block merge mode is applied to the current block, the sub-block merge mode information of the current block is decoded, and when the regular merge mode indicator indicates that a merge mode having a regular merge mode and a motion vector difference is not applied to the current block, the merge mode indicator within the frame is selectively decoded from the bitstream. Claim 2 A video decoding method according to claim 1, wherein when the regular merge mode indicator is a first value, one of the regular merge mode or the merge mode having the motion vector difference is used to derive the inter-frame prediction information of the current block, and when the regular merge mode indicator is a second value, one of the intra-frame combined merge mode or the geometric division merge mode is used to derive the inter-frame prediction information of the current block. Claim 3 A video decoding method according to claim 1, characterized in that whether to decode the regular merge mode indicator of the current block from the bitstream is determined based on the in-frame combined prediction allowance indicator. Claim 4 A video decoding method according to claim 1, characterized in that whether to decode the regular merge mode indicator of the current block from the bitstream is determined based on the skip indicator of the current block. Claim 5 A video decoding method according to claim 1, characterized in that whether to decode the regular merge mode indicator of the current block from the bitstream is determined based on the type of slice to which the current block belongs. Claim 6 A video decoding method according to claim 1, characterized in that when at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value, the regular merge mode indicator is not decoded from the bitstream. Claim 7 An image decoding method according to claim 2, characterized in that when the regular merge mode indicator is the first value, a merge mode indicator having the motion vector difference of the current block is selectively decoded. Claim 8 A video decoding method according to claim 7, characterized in that when the merge mode indicator having the motion vector difference is not decoded, the value of the merge mode indicator having the motion vector difference is inferred to indicate that the merge mode having the motion vector difference is not applied. Claim 9 An image decoding method according to claim 1, characterized in that even if the decoding of the merge mode indicator combined within the screen is omitted, the value is inferred to indicate that the merge mode combined within the screen is applied to the current block, and the merge index information is explicitly decoded from the bitstream. Claim 10 A video encoding method comprising: a step of encoding a sub-block merge mode indicator of a current block; and a step of encoding a regular merge mode indicator of a current block when the sub-block merge mode indicator is encoded with a value indicating that a sub-block merge mode is not applied to the current block, wherein when the sub-block merge mode indicator is encoded with a value indicating that the sub-block merge mode is applied to the current block, sub-block merge mode information of the current block is encoded, and when the regular merge mode indicator is encoded with a value indicating that a merge mode having a regular merge mode and a motion vector difference is not applied to the current block, an in-frame combined merge mode indicator is selectively encoded in a bitstream. Claim 11 A video encoding method according to claim 10, wherein when the regular merge mode indicator is encoded with a first value, one of the regular merge mode or the merge mode having the motion vector difference is used to derive inter-frame prediction information of the current block, and when the regular merge mode indicator is encoded with a second value, one of the intra-frame combined merge mode or the geometric division merge mode is used to derive inter-frame prediction information of the current block. Claim 12 A video encoding method according to claim 10, characterized in that it is determined whether to encode the regular merge mode indicator of the current block into the bitstream based on the combined prediction allowance indicator within the screen. Claim 13 A video encoding method characterized in that, in claim 10, whether to encode the regular merge mode indicator of the current block into the bitstream is determined based on the skip indicator of the current block. Claim 14 A video encoding method characterized in that, in claim 10, whether to encode the regular merge mode indicator of the current block into the bitstream is determined based on the type of slice to which the current block belongs. Claim 15 A video encoding method according to claim 10, characterized in that when at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value, the regular merge mode indicator of the current block is not encoded in the bitstream. Claim 16 An image encoding method according to claim 11, characterized in that when the regular merge mode indicator is the first value, a merge mode indicator having the motion vector difference of the current block is encoded. Claim 17 An apparatus for transmitting compressed video data comprises: a processor for acquiring the compressed video data; and a transmitter for transmitting the compressed video data, wherein the compressed video data comprises: a step of encoding a sub-block merge mode indicator of a current block; and a step of encoding a regular merge mode indicator of the current block when the sub-block merge mode indicator is encoded with a value indicating that the sub-block merge mode is not applied to the current block, wherein when the sub-block merge mode indicator is encoded with a value indicating that the sub-block merge mode is applied to the current block, sub-block merge mode information of the current block is encoded, and when the regular merge mode indicator is encoded with a value indicating that the regular merge mode and a merge mode having motion vector difference are not applied to the current block, an in-frame combined merge mode indicator is selectively encoded in a bitstream. Claim 18 A video encoding method according to claim 10, characterized in that even if the encoding of the merge mode indicator combined within the screen is omitted, the value is inferred to indicate that the merge mode combined within the screen is applied to the current block, and the merge index information is explicitly encoded in the bitstream.

Images